matplotlib-users Mailing List for matplotlib

 
I'm using matplotlib plots embedded in a wxpython application to display
a stream of data. The application works effectively but I have noticed
that memory usage rises substantially over time (if left long enough the
process can go from 50MB to near 1GB of RAM). After some
experimentation, I have tracked the problem down to the canvas.draw()
call needed to refresh my matplotlib plos. Thinking that the problem may
be due to how I coded up my plots I looked to see if the problem was
present in the examples given at
 
http://matplotlib.sourceforge.net/examples/user_interfaces
 
and found that the problem seem to be present in some the examples
posted there if I added a timer to cause the canvas draw to be
repeatedly performed. 
 
 
The example below is based on 
 
http://matplotlib.sourceforge.net/examples/user_interfaces/embedding_in_
wx2.html
 
and has been instrumented to report memory usage when running in
Windows. My tests indicate that working set memory rises by 11 to 12
Kbytes at each plot. 
 
Some what to my surprise, I found the same issue when I ran an even
simpler version on a linux mint virtual box running on my win7 system.
(I have not explored this much so somebody else might want to test
things on a non virtual linux box) I assume this implies that python is
probably keeping something alive that was expected to be automatically
cleaned up at each draw. I have made some effort to localize the problem
further but right now I don't have the necessary time to get heavily
into the inner workings of matplotlib or wxpython 
 
Has any body else seen this problem before, and if so, has any body a
work around that allows rapid replotting - I would guess that I could
periodically flush the figure and re build my plots but this would cause
periodic glitches in plotting? 
 
 
Dominic
 
 
System details
 
Win7 ultimate service pack1
Python 2.6.6 (but I've seen the problem with python2.5 and 2.7)
Matplotlib 1.1.1 (but also in earlier versions)
Wxpython 2.9.4.0 msw (classic) (put also on 2.8.12.0 unicode)
 
 
Example 
------------------------------------------------------------------------
-----------------------------------------------------------
import wxversion
wxversion.ensureMinimal('2.8')
 
from numpy import arange, sin, pi
 
import matplotlib
 
# uncomment the following to use wx rather than wxagg
#matplotlib.use('WX')
#from matplotlib.backends.backend_wx import FigureCanvasWx as
FigureCanvas
 
# comment out the following to use wx rather than wxagg
matplotlib.use('WXAgg')
from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg as
FigureCanvas
 
from matplotlib.backends.backend_wx import NavigationToolbar2Wx
 
from matplotlib.figure import Figure
import win32process
import wx
 
class CanvasFrame(wx.Frame):
 
 def __init__(self):
 wx.Frame.__init__(self,None,-1,
 'CanvasFrame',size=(550,350))
 
 #self.SetBackgroundColour(wx.NamedColor("WHITE")) # 2.9 does not
like
 
 self.figure = Figure()
 self.axes = self.figure.add_subplot(111)
 t = arange(0.0,3.0,0.01)
 s = sin(2*pi*t)
 
 self.axes.plot(t,s)
 self.canvas = FigureCanvas(self, -1, self.figure)
 
 self.sizer = wx.BoxSizer(wx.VERTICAL)
 self.sizer.Add(self.canvas, 1, wx.LEFT | wx.TOP | wx.GROW)
 self.SetSizer(self.sizer)
 self.Fit()
 
 self.add_toolbar() # comment this out for no toolbar
 self.timer = wx.Timer(self, -1)
 
 self.Bind(wx.EVT_TIMER, self.OnUpdate)
 self.delay = 200
 self.timer.Start(self.delay, oneShot = False)
 self.proc_id = win32process.GetCurrentProcess()
 adict = win32process.GetProcessMemoryInfo(self.proc_id)
 self.memory_old = adict["WorkingSetSize"]
 self.i = 0
 
 def add_toolbar(self):
 self.toolbar = NavigationToolbar2Wx(self.canvas)
 self.toolbar.Realize()
 if wx.Platform == '__WXMAC__':
 # Mac platform (OSX 10.3, MacPython) does not seem to cope
with
 # having a toolbar in a sizer. This work-around gets the
buttons
 # back, but at the expense of having the toolbar at the top
 self.SetToolBar(self.toolbar)
 else:
 # On Windows platform, default window size is incorrect, so
set
 # toolbar width to figure width.
 tw, th = self.toolbar.GetSizeTuple()
 fw, fh = self.canvas.GetSizeTuple()
 # By adding toolbar in sizer, we are able to put it at the
bottom
 # of the frame - so appearance is closer to GTK version.
 # As noted above, doesn't work for Mac.
 self.toolbar.SetSize(wx.Size(fw, th))
 self.sizer.Add(self.toolbar, 0, wx.LEFT | wx.EXPAND)
 # update the axes menu on the toolbar
 self.toolbar.update()
 
 
 def OnUpdate(self, evt):
 self.OnPaint( evt)
 self.i +=1
 the_count =25
 if self.i %the_count ==0: 
 adict_now = win32process.GetProcessMemoryInfo(self.proc_id)
 memory_now = adict_now["WorkingSetSize"]
 delta = (memory_now - self.memory_old )/ (the_count *1024.0)
#convert to kbytes per update
 self.memory_old = memory_now
 print "memory now = %i delta = %f\n" %(memory_now, delta)
 
 def OnPaint(self, event):
 self.canvas.draw()
 
class App(wx.App):
 
 def OnInit(self):
 'Create the main window and insert the custom frame'
 frame = CanvasFrame()
 frame.Show(True)
 
 return True
 
app = App(0)
app.MainLoop()

On Tue, Sep 11, 2012 at 3:34 PM, Neal Becker <ndb...@gm...> wrote:
> OK, I've attached my sanitized example
>
ImportError: No module named pandas.
Can you provide an example that doesn't depend on pandas?
> Benjamin Root wrote:
>
>> On Tue, Sep 11, 2012 at 9:29 AM, Neal Becker
>> <ndb...@gm...> wrote:
>>
>>> I tried a scatterplot with legend(loc='best'), but the legend
>>> appears on the upper right, covering a data point. There is nothing
>>> anywhere
>>> in the graph on the upper left, which is where 'best' should go.
>>>
>>>
>> A small, self-contained example would be most useful. The logic for 'best'
>> isn't the greatest, but it should work in most cases. If you could provide
>> an example of where this blatantly breaks, it might uncover a bug.
>>
>> Cheers!
>> Ben Root
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
-- 
Damon McDougall
http://www.damon.is-a-geek.com
B2.39
Mathematics Institute
University of Warwick
Coventry
West Midlands
CV4 7AL
United Kingdom

OK, I've attached my sanitized example
Benjamin Root wrote:
> On Tue, Sep 11, 2012 at 9:29 AM, Neal Becker
> <ndb...@gm...> wrote:
> 
>> I tried a scatterplot with legend(loc='best'), but the legend
>> appears on the upper right, covering a data point. There is nothing
>> anywhere
>> in the graph on the upper left, which is where 'best' should go.
>>
>>
> A small, self-contained example would be most useful. The logic for 'best'
> isn't the greatest, but it should work in most cases. If you could provide
> an example of where this blatantly breaks, it might uncover a bug.
> 
> Cheers!
> Ben Root

On Tue, Sep 11, 2012 at 9:29 AM, Neal Becker <ndb...@gm...> wrote:
> I tried a scatterplot with legend(loc='best'), but the legend
> appears on the upper right, covering a data point. There is nothing
> anywhere
> in the graph on the upper left, which is where 'best' should go.
>
>
A small, self-contained example would be most useful. The logic for 'best'
isn't the greatest, but it should work in most cases. If you could provide
an example of where this blatantly breaks, it might uncover a bug.
Cheers!
Ben Root

I tried a scatterplot with legend(loc='best'), but the legend
appears on the upper right, covering a data point. There is nothing anywhere
in the graph on the upper left, which is where 'best' should go.

On Sunday, September 9, 2012, Eric Firing wrote:
> Regarding the need to pre-allocate: yes, matlab is slicker in this
> regard, and every now and then there is discussion about implementing
> equivalent behavior in numpy, or in an add-on module.
>
Technically, you don ́t have to preallocate the memory, but every time you
expand your array, the whole of it will be copied to a new contiguous piece
of memory. This works, but it can be slow if you do it many times, or you
are handling big data.
Using a Python list is not much more flexible, but you could use carrays
(for chunked arrays), that are structures NumPyArrays-compatible (AFAIK)
that are stored non contiguosly in memory. Adding data only needs to move
one of those small pieces, instead of the whole array.