matplotlib-users Mailing List for matplotlib

On 11/21/10 9:43 AM, Gael Varoquaux wrote:
> Yes, I can confirm that adding processEvents in places to facilitate
> redraws is a good way to lead to segfaults. I have seen this in many
> places other than matplotlib. The reason Tk does not have this problem is
> that it deals with event loops in a fundementally different way that Wx
> or Qt.
wx has "Yield()" which sounds a lot like QT's processEvents. But it also 
has "SafeYield()" which can (in theory, anyway) be called safetly within 
an event handler.
Does QT have anything similar?
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

On Mon, Nov 22, 2010 at 11:32 AM, Eric Firing <ef...@ha...> wrote:
> On 11/22/2010 06:15 AM, Benjamin Root wrote:
> > On Fri, Nov 19, 2010 at 3:14 PM, Caleb Constantine
> > <cad...@gm... <mailto:cad...@gm...>> wrote:
> >
> > On Thu, Nov 18, 2010 at 4:50 PM, Benjamin Root <ben...@ou...
> > <mailto:ben...@ou...>> wrote:
> > >
> > > Caleb,
> > >
> > > Interesting analysis. One possible source of a leak would be
> > some sort of dangling reference that still hangs around even though
> > the plot objects have been cleared. By the time of the matplotlib
> > 1.0.0 release, we did seem to clear out pretty much all of these,
> > but it is possible there are still some lurking about. We should
> > probably run your script against the latest svn to see how the
> > results compare.
> > >
> > > Another possibility might be related to numpy. However this is
> > the draw statement, so I don't know how much numpy is used in there.
> > The latest refactor work in numpy has revealed some memory leaks
> > that have existed, so who knows?
> > >
> > > Might be interesting to try making equivalent versions of this
> > script using different backends, and different package versions to
> > possibly isolate the source of the memory leak.
> > >
> > > Thanks for your observations,
> > > Ben Root
> > >
> >
> > Sorry for the double post; it seems the first is not displaying
> > correctly on SourceForge.
> >
> > I conducted a couple more experiments taking into consideration
> > suggestions
> > made in responses to my original post (thanks for the response).
> >
> > First, I ran my original test (as close to it as possible anyway)
> > using the
> > Agg back end for 3 hours, plotting 16591 times (about 1.5Hz). Memory
> > usage
> > increased by 86MB. That's about 5.3K per redraw. Very similar to my
> > original
> > experiment. As suggested, I called gc.collect() after each iteration.
> It
> > returned 67 for every iteration (no increase), although
> len(gc.garbage)
> > reported 0 each iteration.
> >
> > Second, I ran a test targeting TkAgg for 3 hours, plotting 21374
> > times. Memory
> > usage fluctuated over time, but essentially did not increase:
> > starting at
> > 32.54MB and ending at 32.79MB. gc.collect() reported 0 after each
> > iteration
> > as did len(gc.garbage).
> >
> > Attached are images of plots showing change in memory usage over
> > time for each
> > experiment.
> >
> > Any comments would be appreciated.
> >
> > Following is the code for each experiment.
> >
> > Agg
> > -----
> >
> > from random import random
> > from datetime import datetime
> > import os
> > import gc
> > import time
> > import win32api
> > import win32con
> > import win32process
> >
> > import numpy
> >
> > import matplotlib
> > matplotlib.use("Agg")
> > from matplotlib.figure import Figure
> > from matplotlib.backends.backend_agg import FigureCanvasAgg as
> > FigureCanvas
> >
> > def get_process_memory_info(process_id):
> > memory = {}
> > process = None
> > try:
> > process = win32api.OpenProcess(
> >
> win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
> > False, process_id);
> > if process is not None:
> > return win32process.GetProcessMemoryInfo(process)
> > finally:
> > if process:
> > win32api.CloseHandle(process)
> > return memory
> >
> > meg = 1024.0 * 1024.0
> >
> > figure = Figure(dpi=None)
> > canvas = FigureCanvas(figure)
> > axes = figure.add_subplot(1,1,1)
> >
> > def draw(channel, seconds):
> > axes.clear()
> > axes.plot(channel, seconds)
> > canvas.print_figure('test.png')
> >
> > channel = numpy.sin(numpy.arange(1000) * random())
> > seconds = numpy.arange(len(channel))
> > testDuration = 60 * 60 * 3
> > startTime = time.time()
> >
> > print "starting memory: ", \
> > get_process_memory_info(os.getpid())["WorkingSetSize"]/meg
> >
> > while (time.time() - startTime) < testDuration:
> > draw(channel, seconds)
> >
> > t = datetime.now()
> > memory = get_process_memory_info(os.getpid())
> > print "time: {0}, working: {1:f}, collect: {2}, garbage:
> > {3}".format(
> > t,
> > memory["WorkingSetSize"]/meg,
> > gc.collect(),
> > len(gc.garbage) )
> >
> > time.sleep(0.5)
> >
> >
> > TkAgg
> > ---------
> > from random import random
> > from datetime import datetime
> > import sys
> > import os
> > import gc
> > import time
> > import win32api
> > import win32con
> > import win32process
> >
> > import numpy
> >
> > import matplotlib
> > matplotlib.use("TkAgg")
> > from matplotlib.figure import Figure
> > from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg \
> > as FigureCanvas
> >
> > import Tkinter as tk
> >
> > def get_process_memory_info(process_id):
> > memory = {}
> > process = None
> > try:
> > process = win32api.OpenProcess(
> >
> win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
> > False, process_id);
> > if process is not None:
> > return win32process.GetProcessMemoryInfo(process)
> > finally:
> > if process:
> > win32api.CloseHandle(process)
> > return memory
> >
> > meg = 1024.0 * 1024.0
> >
> > rootTk = tk.Tk()
> > rootTk.wm_title("TKAgg Memory Leak")
> >
> > figure = Figure()
> > canvas = FigureCanvas(figure, master=rootTk)
> > axes = figure.add_subplot(1,1,1)
> >
> > def draw(channel, seconds):
> > axes.clear()
> > axes.plot(channel, seconds)
> >
> > channel = numpy.sin(numpy.arange(1000) * random())
> > seconds = numpy.arange(len(channel))
> >
> > testDuration = 60 * 60 * 3
> > startTime = time.time()
> >
> > print "starting memory: ", \
> > get_process_memory_info(os.getpid())["WorkingSetSize"]/meg
> >
> > draw(channel, seconds)
> > canvas.show()
> > canvas.get_tk_widget().pack(side=tk.TOP, fill=tk.BOTH, expand=1)
> >
> > rate = 500
> >
> > def on_tick():
> > canvas.get_tk_widget().after(rate, on_tick)
> >
> > if (time.time() - startTime) >= testDuration:
> > return
> >
> > draw(channel, seconds)
> >
> > t = datetime.now()
> > memory = get_process_memory_info(os.getpid())
> > print "time: {0}, working: {1:f}, collect: {2}, garbage:
> > {3}".format(
> > t,
> > memory["WorkingSetSize"]/meg,
> > gc.collect(),
> > len(gc.garbage) )
> >
> > canvas.get_tk_widget().after(rate, on_tick)
> > tk.mainloop()
> >
> >
> > Interesting results. I would like to try these tests on a Linux machine
> > to see if there is a difference, but I don't know what the equivalent
> > functions would be to some of the win32 calls. Does anybody have a
> > reference for such things?
>
> Do you need win32 calls, or do you just need to read the memory usage?
> If the latter, see cbook.report_memory().
>
> Eric
>
> >
> > Ben Root
>
>
I tried out the script using cbook.report_memory() with and without the
patch. The patch certainly made the leak *much* slower. I am still finding
a very slow leak at approximately 0.03226 MiB per 5 minutes in the resident
set size.
Ben Root

On Mon, Nov 22, 2010 at 5:56 PM, Benjamin Root <ben...@ou...> wrote:
>
> Miguel,
>
> This is a known issue with mplot3d. The issue is that the polygons are
> essentially abstracted 2-D objects that are using the same backend
> architecture as the core 2-D plotting software. I think your example script
> is probably the *best* example of the problem that I have seen and does a
> very good job of illustrating the problem.
>
> The crux of the problem is that each polygon is assigned a z-order for
> layering, and this z-order is determined by (i think) the center-of-mass
> location of the polygon and how it relates the location of the centers of
> mass of the other objects. In most graphing situations, this is a good
> enough hack, but there are too many use-cases where it falls apart.
>
> Because we are limited to a single z-order value for each object, and
> compositing is done through a 2-D rendering engine, this problem will likely
> not get solved any time soon, unfortunately. The fix would probably require
> a complete rewrite of the mpl drawing engine or maybe the utilization of
> OpenGL?
>
> Sorry I can't be more helpful in fixing your problem, it has been a
> aggravating issue for me as well. Thank you for your excellent example
> script.
>
> Ben Root
>
>
Thank you for your quick (and kind) reply.
I understand that matplotlib is originally 2d - I was hoping, with this new
3d toolkit, to ditch mayavi and use matplotlib for everything :), but that's
ok.
Thanks again,
Miguel

On Mon, Nov 22, 2010 at 11:44 AM, Miguel Costa <mig...@gm...>wrote:
> Hello all.
>
> I'm attempting to use bar3d to plot a packing solution but from some
> viewpoints the perspective is wrong (boxes seem overlapped), and with a
> large number of boxes the plot is always incoherent.
>
> (I'm using matplotlib 1.0.0 on fedora 14)
>
> Does anyone know how to fix this?
>
> Cheers,
> Miguel
>
>
Miguel,
This is a known issue with mplot3d. The issue is that the polygons are
essentially abstracted 2-D objects that are using the same backend
architecture as the core 2-D plotting software. I think your example script
is probably the *best* example of the problem that I have seen and does a
very good job of illustrating the problem.
The crux of the problem is that each polygon is assigned a z-order for
layering, and this z-order is determined by (i think) the center-of-mass
location of the polygon and how it relates the location of the centers of
mass of the other objects. In most graphing situations, this is a good
enough hack, but there are too many use-cases where it falls apart.
Because we are limited to a single z-order value for each object, and
compositing is done through a 2-D rendering engine, this problem will likely
not get solved any time soon, unfortunately. The fix would probably require
a complete rewrite of the mpl drawing engine or maybe the utilization of
OpenGL?
Sorry I can't be more helpful in fixing your problem, it has been a
aggravating issue for me as well. Thank you for your excellent example
script.
Ben Root

Hello all.
I'm attempting to use bar3d to plot a packing solution but from some
viewpoints the perspective is wrong (boxes seem overlapped), and with a
large number of boxes the plot is always incoherent.
(I'm using matplotlib 1.0.0 on fedora 14)
Does anyone know how to fix this?
Cheers,
Miguel
Example code:
-----------------------------------------------------------------------
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
# setup figure
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.set_xlim3d(0, 10)
ax.set_ylim3d(0, 10)
ax.set_zlim3d(0, 10)
# boxes 
x = [0,2,0,2,0]
y = [0,0,2,2,0]
z = [0,0,0,0,2]
dx = [2,2,2,2,3]
dy = [2,2,2,2,3]
dz = [2,2,2,2,3]
colors = ['b','b','b','b','g']
# draw
ax.bar3d(x, y, z, dx, dy, dz, color=colors, alpha=1.0)
# perspective
ax.view_init(azim=48.0, elev=20.0) # problem
#ax.view_init(azim=48.0, elev=45.0) # no problem
# show
plt.show()

On 11/22/2010 06:15 AM, Benjamin Root wrote:
> On Fri, Nov 19, 2010 at 3:14 PM, Caleb Constantine
> <cad...@gm... <mailto:cad...@gm...>> wrote:
>
> On Thu, Nov 18, 2010 at 4:50 PM, Benjamin Root <ben...@ou...
> <mailto:ben...@ou...>> wrote:
> >
> > Caleb,
> >
> > Interesting analysis. One possible source of a leak would be
> some sort of dangling reference that still hangs around even though
> the plot objects have been cleared. By the time of the matplotlib
> 1.0.0 release, we did seem to clear out pretty much all of these,
> but it is possible there are still some lurking about. We should
> probably run your script against the latest svn to see how the
> results compare.
> >
> > Another possibility might be related to numpy. However this is
> the draw statement, so I don't know how much numpy is used in there.
> The latest refactor work in numpy has revealed some memory leaks
> that have existed, so who knows?
> >
> > Might be interesting to try making equivalent versions of this
> script using different backends, and different package versions to
> possibly isolate the source of the memory leak.
> >
> > Thanks for your observations,
> > Ben Root
> >
>
> Sorry for the double post; it seems the first is not displaying
> correctly on SourceForge.
>
> I conducted a couple more experiments taking into consideration
> suggestions
> made in responses to my original post (thanks for the response).
>
> First, I ran my original test (as close to it as possible anyway)
> using the
> Agg back end for 3 hours, plotting 16591 times (about 1.5Hz). Memory
> usage
> increased by 86MB. That's about 5.3K per redraw. Very similar to my
> original
> experiment. As suggested, I called gc.collect() after each iteration. It
> returned 67 for every iteration (no increase), although len(gc.garbage)
> reported 0 each iteration.
>
> Second, I ran a test targeting TkAgg for 3 hours, plotting 21374
> times. Memory
> usage fluctuated over time, but essentially did not increase:
> starting at
> 32.54MB and ending at 32.79MB. gc.collect() reported 0 after each
> iteration
> as did len(gc.garbage).
>
> Attached are images of plots showing change in memory usage over
> time for each
> experiment.
>
> Any comments would be appreciated.
>
> Following is the code for each experiment.
>
> Agg
> -----
>
> from random import random
> from datetime import datetime
> import os
> import gc
> import time
> import win32api
> import win32con
> import win32process
>
> import numpy
>
> import matplotlib
> matplotlib.use("Agg")
> from matplotlib.figure import Figure
> from matplotlib.backends.backend_agg import FigureCanvasAgg as
> FigureCanvas
>
> def get_process_memory_info(process_id):
> memory = {}
> process = None
> try:
> process = win32api.OpenProcess(
> win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
> False, process_id);
> if process is not None:
> return win32process.GetProcessMemoryInfo(process)
> finally:
> if process:
> win32api.CloseHandle(process)
> return memory
>
> meg = 1024.0 * 1024.0
>
> figure = Figure(dpi=None)
> canvas = FigureCanvas(figure)
> axes = figure.add_subplot(1,1,1)
>
> def draw(channel, seconds):
> axes.clear()
> axes.plot(channel, seconds)
> canvas.print_figure('test.png')
>
> channel = numpy.sin(numpy.arange(1000) * random())
> seconds = numpy.arange(len(channel))
> testDuration = 60 * 60 * 3
> startTime = time.time()
>
> print "starting memory: ", \
> get_process_memory_info(os.getpid())["WorkingSetSize"]/meg
>
> while (time.time() - startTime) < testDuration:
> draw(channel, seconds)
>
> t = datetime.now()
> memory = get_process_memory_info(os.getpid())
> print "time: {0}, working: {1:f}, collect: {2}, garbage:
> {3}".format(
> t,
> memory["WorkingSetSize"]/meg,
> gc.collect(),
> len(gc.garbage) )
>
> time.sleep(0.5)
>
>
> TkAgg
> ---------
> from random import random
> from datetime import datetime
> import sys
> import os
> import gc
> import time
> import win32api
> import win32con
> import win32process
>
> import numpy
>
> import matplotlib
> matplotlib.use("TkAgg")
> from matplotlib.figure import Figure
> from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg \
> as FigureCanvas
>
> import Tkinter as tk
>
> def get_process_memory_info(process_id):
> memory = {}
> process = None
> try:
> process = win32api.OpenProcess(
> win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
> False, process_id);
> if process is not None:
> return win32process.GetProcessMemoryInfo(process)
> finally:
> if process:
> win32api.CloseHandle(process)
> return memory
>
> meg = 1024.0 * 1024.0
>
> rootTk = tk.Tk()
> rootTk.wm_title("TKAgg Memory Leak")
>
> figure = Figure()
> canvas = FigureCanvas(figure, master=rootTk)
> axes = figure.add_subplot(1,1,1)
>
> def draw(channel, seconds):
> axes.clear()
> axes.plot(channel, seconds)
>
> channel = numpy.sin(numpy.arange(1000) * random())
> seconds = numpy.arange(len(channel))
>
> testDuration = 60 * 60 * 3
> startTime = time.time()
>
> print "starting memory: ", \
> get_process_memory_info(os.getpid())["WorkingSetSize"]/meg
>
> draw(channel, seconds)
> canvas.show()
> canvas.get_tk_widget().pack(side=tk.TOP, fill=tk.BOTH, expand=1)
>
> rate = 500
>
> def on_tick():
> canvas.get_tk_widget().after(rate, on_tick)
>
> if (time.time() - startTime) >= testDuration:
> return
>
> draw(channel, seconds)
>
> t = datetime.now()
> memory = get_process_memory_info(os.getpid())
> print "time: {0}, working: {1:f}, collect: {2}, garbage:
> {3}".format(
> t,
> memory["WorkingSetSize"]/meg,
> gc.collect(),
> len(gc.garbage) )
>
> canvas.get_tk_widget().after(rate, on_tick)
> tk.mainloop()
>
>
> Interesting results. I would like to try these tests on a Linux machine
> to see if there is a difference, but I don't know what the equivalent
> functions would be to some of the win32 calls. Does anybody have a
> reference for such things?
Do you need win32 calls, or do you just need to read the memory usage? 
If the latter, see cbook.report_memory().
Eric
>
> Ben Root

On Thu, Nov 18, 2010 at 2:30 AM, Eric Emsellem <eem...@es...> wrote:
> Ok problem(s) solved, thanks a lot for the efficient help (this also taught
> me how to go through the code more thoroughly)
>
> * for the record: I had a pylab.py in the site-packages directory, probably
> a left-over from some other installation, which was interfering with the
> pylab.py which should be the one under matplotlib/. I just removed it now
> and it works well.
>
> * and you were right, the funny "try this" was a left over of many tests I
> had done to track down the problem (mea culpa on that one)
>
> thanks again, all works beautifully (and consistently) now.
>
> cheers
> Eric
>
Glad to be of help. Most of the time, odd behaviors like this tend to be
the result of conflicting installations. Digging through them tends to be
difficult, but a very educating experience.
Ben Root

On Fri, Nov 19, 2010 at 3:14 PM, Caleb Constantine <cad...@gm...>wrote:
> On Thu, Nov 18, 2010 at 4:50 PM, Benjamin Root <ben...@ou...> wrote:
> >
> > Caleb,
> >
> > Interesting analysis. One possible source of a leak would be some sort
> of dangling reference that still hangs around even though the plot objects
> have been cleared. By the time of the matplotlib 1.0.0 release, we did seem
> to clear out pretty much all of these, but it is possible there are still
> some lurking about. We should probably run your script against the latest
> svn to see how the results compare.
> >
> > Another possibility might be related to numpy. However this is the draw
> statement, so I don't know how much numpy is used in there. The latest
> refactor work in numpy has revealed some memory leaks that have existed, so
> who knows?
> >
> > Might be interesting to try making equivalent versions of this script
> using different backends, and different package versions to possibly isolate
> the source of the memory leak.
> >
> > Thanks for your observations,
> > Ben Root
> >
>
> Sorry for the double post; it seems the first is not displaying
> correctly on SourceForge.
>
> I conducted a couple more experiments taking into consideration suggestions
> made in responses to my original post (thanks for the response).
>
> First, I ran my original test (as close to it as possible anyway) using the
> Agg back end for 3 hours, plotting 16591 times (about 1.5Hz). Memory usage
> increased by 86MB. That's about 5.3K per redraw. Very similar to my
> original
> experiment. As suggested, I called gc.collect() after each iteration. It
> returned 67 for every iteration (no increase), although len(gc.garbage)
> reported 0 each iteration.
>
> Second, I ran a test targeting TkAgg for 3 hours, plotting 21374 times.
> Memory
> usage fluctuated over time, but essentially did not increase: starting at
> 32.54MB and ending at 32.79MB. gc.collect() reported 0 after each iteration
> as did len(gc.garbage).
>
> Attached are images of plots showing change in memory usage over time for
> each
> experiment.
>
> Any comments would be appreciated.
>
> Following is the code for each experiment.
>
> Agg
> -----
>
> from random import random
> from datetime import datetime
> import os
> import gc
> import time
> import win32api
> import win32con
> import win32process
>
> import numpy
>
> import matplotlib
> matplotlib.use("Agg")
> from matplotlib.figure import Figure
> from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
>
> def get_process_memory_info(process_id):
> memory = {}
> process = None
> try:
> process = win32api.OpenProcess(
> win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
> False, process_id);
> if process is not None:
> return win32process.GetProcessMemoryInfo(process)
> finally:
> if process:
> win32api.CloseHandle(process)
> return memory
>
> meg = 1024.0 * 1024.0
>
> figure = Figure(dpi=None)
> canvas = FigureCanvas(figure)
> axes = figure.add_subplot(1,1,1)
>
> def draw(channel, seconds):
> axes.clear()
> axes.plot(channel, seconds)
> canvas.print_figure('test.png')
>
> channel = numpy.sin(numpy.arange(1000) * random())
> seconds = numpy.arange(len(channel))
> testDuration = 60 * 60 * 3
> startTime = time.time()
>
> print "starting memory: ", \
> get_process_memory_info(os.getpid())["WorkingSetSize"]/meg
>
> while (time.time() - startTime) < testDuration:
> draw(channel, seconds)
>
> t = datetime.now()
> memory = get_process_memory_info(os.getpid())
> print "time: {0}, working: {1:f}, collect: {2}, garbage: {3}".format(
> t,
> memory["WorkingSetSize"]/meg,
> gc.collect(),
> len(gc.garbage) )
>
> time.sleep(0.5)
>
>
> TkAgg
> ---------
> from random import random
> from datetime import datetime
> import sys
> import os
> import gc
> import time
> import win32api
> import win32con
> import win32process
>
> import numpy
>
> import matplotlib
> matplotlib.use("TkAgg")
> from matplotlib.figure import Figure
> from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg \
> as FigureCanvas
>
> import Tkinter as tk
>
> def get_process_memory_info(process_id):
> memory = {}
> process = None
> try:
> process = win32api.OpenProcess(
> win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
> False, process_id);
> if process is not None:
> return win32process.GetProcessMemoryInfo(process)
> finally:
> if process:
> win32api.CloseHandle(process)
> return memory
>
> meg = 1024.0 * 1024.0
>
> rootTk = tk.Tk()
> rootTk.wm_title("TKAgg Memory Leak")
>
> figure = Figure()
> canvas = FigureCanvas(figure, master=rootTk)
> axes = figure.add_subplot(1,1,1)
>
> def draw(channel, seconds):
> axes.clear()
> axes.plot(channel, seconds)
>
> channel = numpy.sin(numpy.arange(1000) * random())
> seconds = numpy.arange(len(channel))
>
> testDuration = 60 * 60 * 3
> startTime = time.time()
>
> print "starting memory: ", \
> get_process_memory_info(os.getpid())["WorkingSetSize"]/meg
>
> draw(channel, seconds)
> canvas.show()
> canvas.get_tk_widget().pack(side=tk.TOP, fill=tk.BOTH, expand=1)
>
> rate = 500
>
> def on_tick():
> canvas.get_tk_widget().after(rate, on_tick)
>
> if (time.time() - startTime) >= testDuration:
> return
>
> draw(channel, seconds)
>
> t = datetime.now()
> memory = get_process_memory_info(os.getpid())
> print "time: {0}, working: {1:f}, collect: {2}, garbage: {3}".format(
> t,
> memory["WorkingSetSize"]/meg,
> gc.collect(),
> len(gc.garbage) )
>
> canvas.get_tk_widget().after(rate, on_tick)
> tk.mainloop()
>
Interesting results. I would like to try these tests on a Linux machine to
see if there is a difference, but I don't know what the equivalent functions
would be to some of the win32 calls. Does anybody have a reference for such
things?
Ben Root

Caleb,
Thanks for doing all of this investigation and providing something easy 
to reproduce.
With the help of valgrind, I believe I've tracked it down to a bug in 
PyCXX, the Python/C++ interface tool matplotlib uses.
I have attached a patch that seems to remove the leak for me, but as I'm 
not a PyCXX expert, I'm not comfortable with committing it to the 
repository just yet. *I'm hoping you and/or some other developers could 
test it on their systems (a fully clean re-build is required) and report 
any problems back. *I also plan to raise this question on the PyCXX 
mailing list to get any thoughts they may have.
Cheers,
Mike
On 11/19/2010 04:14 PM, Caleb Constantine wrote:
> On Thu, Nov 18, 2010 at 4:50 PM, Benjamin Root<ben...@ou...> wrote:
>> Caleb,
>>
>> Interesting analysis. One possible source of a leak would be some sort of dangling reference that still hangs around even though the plot objects have been cleared. By the time of the matplotlib 1.0.0 release, we did seem to clear out pretty much all of these, but it is possible there are still some lurking about. We should probably run your script against the latest svn to see how the results compare.
>>
>> Another possibility might be related to numpy. However this is the draw statement, so I don't know how much numpy is used in there. The latest refactor work in numpy has revealed some memory leaks that have existed, so who knows?
>>
>> Might be interesting to try making equivalent versions of this script using different backends, and different package versions to possibly isolate the source of the memory leak.
>>
>> Thanks for your observations,
>> Ben Root
>>
> Sorry for the double post; it seems the first is not displaying
> correctly on SourceForge.
>
> I conducted a couple more experiments taking into consideration suggestions
> made in responses to my original post (thanks for the response).
>
> First, I ran my original test (as close to it as possible anyway) using the
> Agg back end for 3 hours, plotting 16591 times (about 1.5Hz). Memory usage
> increased by 86MB. That's about 5.3K per redraw. Very similar to my original
> experiment. As suggested, I called gc.collect() after each iteration. It
> returned 67 for every iteration (no increase), although len(gc.garbage)
> reported 0 each iteration.
>
> Second, I ran a test targeting TkAgg for 3 hours, plotting 21374 times. Memory
> usage fluctuated over time, but essentially did not increase: starting at
> 32.54MB and ending at 32.79MB. gc.collect() reported 0 after each iteration
> as did len(gc.garbage).
>
> Attached are images of plots showing change in memory usage over time for each
> experiment.
>
> Any comments would be appreciated.
>
> Following is the code for each experiment.
>
> Agg
> -----
>
> from random import random
> from datetime import datetime
> import os
> import gc
> import time
> import win32api
> import win32con
> import win32process
>
> import numpy
>
> import matplotlib
> matplotlib.use("Agg")
> from matplotlib.figure import Figure
> from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
>
> def get_process_memory_info(process_id):
> memory = {}
> process = None
> try:
> process = win32api.OpenProcess(
> win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
> False, process_id);
> if process is not None:
> return win32process.GetProcessMemoryInfo(process)
> finally:
> if process:
> win32api.CloseHandle(process)
> return memory
>
> meg = 1024.0 * 1024.0
>
> figure = Figure(dpi=None)
> canvas = FigureCanvas(figure)
> axes = figure.add_subplot(1,1,1)
>
> def draw(channel, seconds):
> axes.clear()
> axes.plot(channel, seconds)
> canvas.print_figure('test.png')
>
> channel = numpy.sin(numpy.arange(1000) * random())
> seconds = numpy.arange(len(channel))
> testDuration = 60 * 60 * 3
> startTime = time.time()
>
> print "starting memory: ", \
> get_process_memory_info(os.getpid())["WorkingSetSize"]/meg
>
> while (time.time() - startTime)< testDuration:
> draw(channel, seconds)
>
> t = datetime.now()
> memory = get_process_memory_info(os.getpid())
> print "time: {0}, working: {1:f}, collect: {2}, garbage: {3}".format(
> t,
> memory["WorkingSetSize"]/meg,
> gc.collect(),
> len(gc.garbage) )
>
> time.sleep(0.5)
>
>
> TkAgg
> ---------
> from random import random
> from datetime import datetime
> import sys
> import os
> import gc
> import time
> import win32api
> import win32con
> import win32process
>
> import numpy
>
> import matplotlib
> matplotlib.use("TkAgg")
> from matplotlib.figure import Figure
> from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg \
> as FigureCanvas
>
> import Tkinter as tk
>
> def get_process_memory_info(process_id):
> memory = {}
> process = None
> try:
> process = win32api.OpenProcess(
> win32con.PROCESS_QUERY_INFORMATION|win32con.PROCESS_VM_READ,
> False, process_id);
> if process is not None:
> return win32process.GetProcessMemoryInfo(process)
> finally:
> if process:
> win32api.CloseHandle(process)
> return memory
>
> meg = 1024.0 * 1024.0
>
> rootTk = tk.Tk()
> rootTk.wm_title("TKAgg Memory Leak")
>
> figure = Figure()
> canvas = FigureCanvas(figure, master=rootTk)
> axes = figure.add_subplot(1,1,1)
>
> def draw(channel, seconds):
> axes.clear()
> axes.plot(channel, seconds)
>
> channel = numpy.sin(numpy.arange(1000) * random())
> seconds = numpy.arange(len(channel))
>
> testDuration = 60 * 60 * 3
> startTime = time.time()
>
> print "starting memory: ", \
> get_process_memory_info(os.getpid())["WorkingSetSize"]/meg
>
> draw(channel, seconds)
> canvas.show()
> canvas.get_tk_widget().pack(side=tk.TOP, fill=tk.BOTH, expand=1)
>
> rate = 500
>
> def on_tick():
> canvas.get_tk_widget().after(rate, on_tick)
>
> if (time.time() - startTime)>= testDuration:
> return
>
> draw(channel, seconds)
>
> t = datetime.now()
> memory = get_process_memory_info(os.getpid())
> print "time: {0}, working: {1:f}, collect: {2}, garbage: {3}".format(
> t,
> memory["WorkingSetSize"]/meg,
> gc.collect(),
> len(gc.garbage) )
>
> canvas.get_tk_widget().after(rate, on_tick)
> tk.mainloop()
>
>
> ------------------------------------------------------------------------------
> Beautiful is writing same markup. Internet Explorer 9 supports
> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2& L3.
> Spend less time writing and rewriting code and more time creating great
> experiences on the web. Be a part of the beta today
> http://p.sf.net/sfu/msIE9-sfdev2dev
>
>
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

On Mon, Nov 22, 2010 at 6:59 AM, Tobias Winchen <
wi...@ph...> wrote:
> Dear all,
>
> > On Wednesday 17 November 2010 Benjamin Root wrote:
> >
> > Nevermind about the example script, I tested it myself and verified the
> > problem. The fix was simple. We were missing a square root for the
> > denominator of the formula for x and y. Before I submit this patch, can
> > anybody with more knowledge about the hammer projection verify that the
> > equations on the wikipedia page are correct? I would like to verify the
> > transformation before making this change.
> >
>
> The patch seems correct - see
>
>
> http://www.progonos.com/furuti/MapProj/Dither/CartHow/HowAiHaW3/howAiHaW3.html
>
> or
>
> The Glossary of Mapping Sciences 1994,
>
> http://books.google.com/books?id=jPVxSDzVRP0C&lpg=PA59&ots=n4hOaubxqo&dq=hammer%20projection%20equation&pg=PA241#v=onepage&q&f=true
>
> Best regards,
>
> Tobias
>
>
Tobias and Matthias, thanks for the confirmation. I have updated the
maintenance branch and the trunk with revisions 8808 and 8809.
Ben Root

On Mon, Nov 22, 2010 at 7:56 AM, andres luhamaa <and...@gm...>wrote:
> Hello!
> I see strange behaviour in plotting bars with dates on x-axis. Example
> attached, problem seems to arise only after duplicating x-axis and with
> all-zero data. good.png is with non-zero data and bad.png with zero data.
>
> Regards,
> Andres
>
>
I noticed some issues recently with another bug report regarding bar
charts. It seems that if the first bar (maybe even the last bar) is of zero
height, then some code somewhere doesn't properly choose the correct axes
limit. This error here might be a degenerate form of that bug.
If anything, this indicates that the bug must be somewhere between the auto
tickers and bar(). I have checked bar() itself, and I don't see any sort of
problem. There was a possible issue at around line 4694 in axes.py (at the
end of the bar() method), but I don't think that is the cause.
Ben Root

Hello!
I see strange behaviour in plotting bars with dates on x-axis. Example 
attached, problem seems to arise only after duplicating x-axis and with 
all-zero data. good.png is with non-zero data and bad.png with zero data.
Regards,
Andres

Dear all,
> On Wednesday 17 November 2010 Benjamin Root wrote:
> 
> Nevermind about the example script, I tested it myself and verified the
> problem. The fix was simple. We were missing a square root for the
> denominator of the formula for x and y. Before I submit this patch, can
> anybody with more knowledge about the hammer projection verify that the
> equations on the wikipedia page are correct? I would like to verify the
> transformation before making this change.
> 
The patch seems correct - see
http://www.progonos.com/furuti/MapProj/Dither/CartHow/HowAiHaW3/howAiHaW3.html
or
The Glossary of Mapping Sciences 1994, 
http://books.google.com/books?id=jPVxSDzVRP0C&lpg=PA59&ots=n4hOaubxqo&dq=hammer%20projection%20equation&pg=PA241#v=onepage&q&f=true
Best regards,
Tobias
-- 
Tobias Winchen
III. Physikalisches Institut A
RWTH Aachen University
+49 (0)241 80 27326

Hello,
I have tried in vain to rotate a rectangle patch. I tried to use 
AffineTransform2D.rotate_deg(20.) and apply it to the Rectangle with 
set_transform. If I add the patch it doesnt show.
What is the right course of action here.
Cheers
 Wolfgang

Solved it, was thinking backwards again.
From: qw...@ho...
To: mat...@li...
Date: 2010年11月22日 09:00:38 +0000
Subject: Re: [Matplotlib-users] Formatter dates
Thanks for the reply!
I have been looking into it now and thinks i have get the hang of how it works rougly. Do you know any way to know the max-value of X straight away?
I have a list of dates that with listlength Y, the X spans from 0 - something. So if i could figure out what "something" is i could get the ratio and then plot the dates by indexing the ratio times X eg. (Xmax / listlength) * X.
// Tim
> From: jd...@gm...
> Date: 2010年11月17日 05:55:04 -0600
> Subject: Re: [Matplotlib-users] Formatter dates
> To: qw...@ho...
> CC: mat...@li...
> 
> On Wed, Nov 17, 2010 at 2:29 AM, Tim Åberg <qw...@ho...> wrote:
> > Hello!
> >
> > I have now been tampering with a custom formatter and the more i think about
> > it the more i feel there must be a more easy soulution. I have a set of
> > values that are plotted over time (i use date2num, to get the conversion
> > from date to num), i also have a list with dates that corresponds to these
> > values;
> > ['2010-11-05 10:27:45.605000', '2010-11-05 10:27:50.576000', '2010-11-05
> > 10:27:55.913000'], this to not have to do a conversion back, i imagine its
> > the most effective way to do it.
> >
> > Now to the question;
> >
> > The only two values in the datelist that is vital is the first and the last,
> > in between i really just want to have values that corresponds lineary. Thats
> > brings me to resoulution, if the values are far far in between, say weeks or
> > even moth the time (h:s:ms) isnt so important and vice versa.
> >
> > Is there any formatter that do this sort of things? eg. takes in two
> > values(dates) and format it by itself.
> 
> Take a look at this example
> 
> http://matplotlib.sourceforge.net/examples/api/date_index_formatter.html
> 
> It's trying to solve a different but related problem: in financial
> time series you only have data on Monday - Friday, and you don't want
> to plot the gaps on weekends. So you plot the data linearly with an
> index, and use a custom locator and formatter to set and format the
> ticks.
> 
> JDH
 		 	 		 
------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today
http://p.sf.net/sfu/msIE9-sfdev2dev
_______________________________________________
Matplotlib-users mailing list
Mat...@li...
https://lists.sourceforge.net/lists/listinfo/matplotlib-users 		 	 		 

Thanks for the reply!
I have been looking into it now and thinks i have get the hang of how it works rougly. Do you know any way to know the max-value of X straight away?
I have a list of dates that with listlength Y, the X spans from 0 - something. So if i could figure out what "something" is i could get the ratio and then plot the dates by indexing the ratio times X eg. (Xmax / listlength) * X.
// Tim
> From: jd...@gm...
> Date: 2010年11月17日 05:55:04 -0600
> Subject: Re: [Matplotlib-users] Formatter dates
> To: qw...@ho...
> CC: mat...@li...
> 
> On Wed, Nov 17, 2010 at 2:29 AM, Tim Åberg <qw...@ho...> wrote:
> > Hello!
> >
> > I have now been tampering with a custom formatter and the more i think about
> > it the more i feel there must be a more easy soulution. I have a set of
> > values that are plotted over time (i use date2num, to get the conversion
> > from date to num), i also have a list with dates that corresponds to these
> > values;
> > ['2010-11-05 10:27:45.605000', '2010-11-05 10:27:50.576000', '2010-11-05
> > 10:27:55.913000'], this to not have to do a conversion back, i imagine its
> > the most effective way to do it.
> >
> > Now to the question;
> >
> > The only two values in the datelist that is vital is the first and the last,
> > in between i really just want to have values that corresponds lineary. Thats
> > brings me to resoulution, if the values are far far in between, say weeks or
> > even moth the time (h:s:ms) isnt so important and vice versa.
> >
> > Is there any formatter that do this sort of things? eg. takes in two
> > values(dates) and format it by itself.
> 
> Take a look at this example
> 
> http://matplotlib.sourceforge.net/examples/api/date_index_formatter.html
> 
> It's trying to solve a different but related problem: in financial
> time series you only have data on Monday - Friday, and you don't want
> to plot the gaps on weekends. So you plot the data linearly with an
> index, and use a custom locator and formatter to set and format the
> ticks.
> 
> JDH
 		 	 		 

On Fri, Nov 19, 2010 at 2:59 AM, C M <cmp...@gm...> wrote:
> 2) How can I get the lines belonging to different axes to cycle
> through colors such that the same color is not used for any lines
> shown in the plot? (that is, I don't want to hard code a color to any
> line, I want it to auto-cycle).
>
ax1 = subplot(121)
ax2 = subplot(122)
import itertools
clist = ['b', 'g', 'r', 'c', 'm', 'y', 'k']
color_cycle = itertools.cycle(clist)
ax1.set_color_cycle(color_cycle)
ax2.set_color_cycle(color_cycle)
ax1.plot([1,2,3])
ax2.plot([1,3,2])
-JJ

When cax is an axes instance of the colorbar, you may use
cax.tick_params(labelsize=8)
If you want to directly set the FontProperties, you need to iterate
over the ticks (it seems that tick_params does not support this).
for tick in cax.yaxis.majorTicks:
 tick.label2.set_fontproperties(fp)
Note that the details depends on the current colorbar setup
(orientation etc.). The bottom line is that cax can be treated as a
normal axes.
-JJ
On Fri, Nov 19, 2010 at 9:30 AM, John <was...@gm...> wrote:
> Hello,
>
> I see that for a legend you can do the following:
>
> ax = plt.scatter(x,y,label='test data')
> p_leg = mpl.font_manager.FontProperties(size='8')
> ax.legend(prop=p_leg)
>
> But, how do you do set font properties for the colorbar tick labels?
>
> Thanks!
>

On Thu, Nov 18, 2010 at 11:10 PM, John <was...@gm...> wrote:
> 1) I only need one colorbar, how would I create a single colorbar on
> the right that spanned across all axes? (ie. same height as the stack)
There are a few options you can try. I guess the easiest way is
setting up the axes manually.
dy = 0.8/3.
ax1 = axes([0.1, 0.1, 0.8, dy])
ax2 = axes([0.1, 0.1+dy, 0.8, dy])
ax3 = axes([0.1, 0.1+2*dy, 0.8, dy])
sc = ax1.scatter([1,2],[3,4], c=[0,1], cmap="jet")
cax = axes([0.9, 0.1, 0.03, 0.8])
plt.colorbar(sc, cax=cax)
Also, you make take a look at the axes_grid1 toolkit.
(see e.g, http://matplotlib.sourceforge.net/examples/axes_grid/demo_axes_grid.html)
> 2) is there a way to place the colorbar in the bottom middle of my
> panels (if I end up with more than one)?
Use loc=8.
> 3) How can I customize the tick labels of the colorbar?
You may use "set_ticks" and "set_ticklabels" method of the colorbar object.
 cb = plt.colorbar(a.collections[0],cax=ia,orientation='horizontal')
 cb.set_ticks([-100,0,100])
 cb.set_ticklabels(["-100", "0", "+100"])
Also, locators and formatter can be specified during colorbar creation
(take a look at the doc).
-JJ

I was trying to implement my own nav toolbar, and everything works great
except an exception that gets thrown intermittently when in zoom
rubberbanding operation, from this bit of code in CallbackRegistry
(cbook.py):
 def process(self, s, *args, **kwargs):
 """
 process signal *s*. All of the functions registered to receive
 callbacks on *s* will be called with *\*args* and *\*\*kwargs*
 """
 self._check_signal(s)
 for cid, proxy in self.callbacks[s].items():
 # Clean out dead references
 if proxy.inst is not None and proxy.inst() is None:
 del self.callbacks[s][cid] #<----------- the cid key
doesn't exist.
 else:
 proxy(*args, **kwargs)
is it OK to eat and ignore this exception? Everything seems to work just
fine.
I know that this is probably one of those weak-reference callbacks getting
dropped, but I don't make any connections to 'button_release_event'
 anywhere in my code, and that's what 's' is in the above code when the
dictionary key is missing. In other areas of my code where I am making
connections, I did make sure that I'm saving a reference to them myself.
-- 
Daniel Hyams
dh...@gm...