matplotlib-users Mailing List for matplotlib

On Thu, Jul 17, 2008 at 2:35 PM, Ben Axelrod <bax...@co...> wrote:
> It seems that axes.plot() handles 'None' values in the input arrays
> gracefully by just not plotting that point. But axes.scatter() bugs out.
> Can this be fixed?
We try to support np.nan and np masked arrays to handle missing data.
The fact that None works with plot is a fortuitous consequence of the
fact that numpy converts None->NaN on a coercion to float, but it is
not something we plan on trying to support explicitly. You can easily
write your own none_to_nan function
def none_to_nan(seq):
 return np.asarray(seq, float)
and if nans are failing we'll treat it as a bug in mpl
JDH

It seems that axes.plot() handles 'None' values in the input arrays gracefully by just not plotting that point. But axes.scatter() bugs out. Can this be fixed?
Thanks,
-Ben

On Thu, Jul 17, 2008 at 1:53 PM, Ben Axelrod <bax...@co...> wrote:
> It seems like this should be possible:
>
>
>
> ax.scatter(x, y, c=None)
Just use plot(x, y, 'o') or whatever marker you want, and set the
markersize. scatter is meant for plots where either the marker size
or marker color vary. plot handles homogeneous markers.
JDH

It seems like this should be possible:
ax.scatter(x, y, c=None)
but axes chokes on the c=None parameter.

On Thu, Jul 17, 2008 at 12:42 PM, Michael Droettboom <md...@st...> wrote:
> Anyone have any thoughts on this? It seems like it's serious enough to try
> to resolve before the next bugfix release.
I think what we are seeing here is the known GUI figure canvas leak
(Michael, I think our offlist conversation about mainquit was a red
herring since removing that call doesn't help). We have found in the
past that creation of gtk canvases and tk canvases leak and this is
outside mpl. The reason that commenting out del Gcf.figs[num] "fixes"
the leak because it causes pyplot to simply reuse the figure rather
than re-call new_figure_manager. The chain of logic
_pylab_helpers.Gcf:
 def get_fig_manager(num):
 figManager = Gcf.figs.get(num, None)
 if figManager is not None: Gcf.set_active(figManager)
 return figManager
pyplot.figure
 figManager = _pylab_helpers.Gcf.get_fig_manager(num)
 if figManager is None:
 if get_backend().lower() == 'ps': dpi = 72
 figManager = new_figure_manager(num, figsize=figsize,
 dpi=dpi,
 facecolor=facecolor,
 edgecolor=edgecolor,
 frameon=frameon,
 FigureClass=FigureClass,
 **kwargs)
so when you do not del the figure number, the manager still lives in
the dictionary in Gcf and is returned by
_pylab_helpers.Gcf.get_fig_manager(num), and so subsequent calls to
new_figure_manager are not triggered (so the figure is not really
closed...)
So there is a bug here, but I am not sure it is in mpl -- I think it
is more likely to be in the GUI toolkits themselves, as we do not see
them in any of the mpl image backends. I don't think we need to hold
a release on this, since it is a known and existing problem with no
obvious mpl solution, though getting a reproducible test case that
just used the GUI code for submission to pygtk, tkinter, etc... would
be useful.
JDH
Michael and I just talked through this offlist, and it appears that
what is happening is that form any of the interactive backends, the
trigger to stop the GUI mainloop is when all the figures have been
closed. The typical use case in a script is to raise several GUI
figures and the program exits when all of them have been closed.
pylab otherwise doesn't know when to quit and return the shell prompt.
 The backend first checks to see if you are in interactive mode, and
does not call main quit if you are, so this doesn't affect folks using
mpl in an ipython shell or other interactive sessions.
The only use case where it should arise is like the one in looptest,
where a script creates a GUI figure and then closes it in
non-interactive mode. Although there is a use case where this makes
sense (eg if we had a blocking show) where one would create a figure,
raise it, block, close it, rinse and repeat, this mode is not
currently supported in pylab (show would need to be smarter, though
with our new blocking input functions we might be able to attempt
this). This also does not affect applications since the close/destroy
handling is a pyplot construct.
Michael pointed out that the twinx problem was separate (and fixed) so
is unrelated to the close bug and can be removed from the looptest
test script.
There is a workaround for those who really need this functionality,
although it is unsupported currently, and that is simply to wrap the
close call in ion/ioff function calls to turn interaction on. The
interactive backends won't attempt to call mainquit if interactive
mode is on, so
 PL.ion()
 PL.close(1)
 PL.ioff()
blocks this leak

Anyone have any thoughts on this? It seems like it's serious enough to 
try to resolve before the next bugfix release.
Cheers,
Mike
Michael Droettboom wrote:
> Yes, it should be. I'm further puzzled that removing "del 
> Gcf.figs[num]" prevents the memory leak. There is some side effect that 
> happens when all of the figures have been closed (I think it shuts down 
> the GUI mainloop), that keeping at least one figure around at all times 
> avoids. But I haven't been able to get to the bottom of that, just a 
> half-supported theory at this point.
>
> Cheers,
> Mike
>
> laurent oget wrote:
> 
>> I am puzzled. Wasn't the whole point of close() to avoid memory leaks?
>>
>> Laurent
>>
>> 2008年7月15日 Michael Droettboom <md...@st... <mailto:md...@st...>>:
>>
>> Yes, as of r5747 twinx (well, shared axes specifically) no longer
>> leaks.
>>
>> Manuel has discovered a seemingly generic leak that occurs when
>> pyplot.close() is called and running a GUI backend. I can confirm his
>> results with the script he last sent.
>>
>> Cheers,
>> Mike
>>
>> Manuel Metz wrote:
>> > John Hunter wrote:
>> >> On Mon, Jul 14, 2008 at 3:05 PM, Michael Droettboom
>> <md...@st... <mailto:md...@st...>>
>> >> wrote:
>> >>> I can confirm this.
>> >>>
>> >>> Commenting out "del Gcf.figs[num]" in Gcf.destroy (in
>> >>> _pylab_helpers.py)
>> >>> also seems to resolve the leak. But I have no idea why, so I
>> won't
>> >>> commit it just yet. I don't have much time to look deeper
>> now. Does
>> >>> anyone (who probably understands figure management better than
>> me) have
>> >>> an idea what might cause this?
>> >>
>> >> Can you post the script you are using to test -- I am a little
>> >> confused from reading this thread by whether or not twinx is
>> >> implicated. Also, I saw that you committed some changes
>> vis-a-vis the
>> >> twinx leak
>> >>
>> >> r5747 | mdboom | 2008年07月11日 13:21:53 -0500 (2008年7月11日) | 2
>> >> lines
>> >>
>> >> Fix memory leak when using shared axes.
>> >>
>> >> so I thought that part was resolved already...
>> >>
>> >> JDH
>> >
>> > I use a modified version of the script Laurent Oget posted (see
>> > attachment). Here is the output if I don't comment out PL.close(1).
>> >
>> > ~/python/test$ python looptest.py -dGTK
>> > 0 GC 69354 69354 0 13854
>> > 100 GC 84354 150 0 15163
>> > 200 GC 99354 150 0 16306
>> > 300 GC 114354 150 0 17364
>> > 400 GC 129354 150 0 18576
>> > ~/python/test$ python looptest.py -dTK
>> > 0 GC 69521 69521 0 14065
>> > 100 GC 84521 150 0 15444
>> > 200 GC 99521 150 0 16581
>> > 300 GC 114521 150 0 17719
>> > 400 GC 129521 150 0 18715
>> > ~/python/test$ python looptest.py -dPS
>> > 0 GC 59307 59307 0 7705
>> > 100 GC 59307 0 0 8037
>> > 200 GC 59307 0 0 8038
>> > 300 GC 59307 0 0 8038
>> > 400 GC 59307 0 0 8038
>> >
>> > (so for the window-less backend PS no objects are left)
>> >
>> > And now I commented out the line PL.close(1):
>> >
>> > ~/python/test$ python looptest.py -dGTK
>> > 0 GC 69379 69379 0 13855
>> > 100 GC 69379 0 0 14253
>> > 200 GC 69379 0 0 14253
>> > 300 GC 69379 0 0 14253
>> > 400 GC 69379 0 0 14252
>> >
>> > Manuel
>>
>> --
>> Michael Droettboom
>> Science Software Branch
>> Operations and Engineering Division
>> Space Telescope Science Institute
>> Operated by AURA for NASA
>>
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>> challenge
>> Build the coolest Linux based applications with Moblin SDK & win
>> great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in
>> the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> <http://moblin-contest.org/redirect.php?banner_id=100&url=/>
>> _______________________________________________
>> Matplotlib-users mailing list
>> Mat...@li...
>> <mailto:Mat...@li...>
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>>
>> 
>
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Wed, Jul 16, 2008 at 6:59 PM, Alan G Isaac <ai...@am...> wrote:
> I assume you know of Robert Kern's code?
> <URL:http://cours-info.iut-bm.univ-fcomte.fr/docs/python/scipy/scipy.sandbox.delaunay-module.html>
I was aware of the project but always assumed he was relying on some
GPL/LGPL code since it wasn't in scipy proper, but I now see this is
not the case. I just talked with Jeff Whitaker who wrote the griddata
functionality on top of NCAR's natgrid (GPL), and he expressed a
willingness to port his code to use Robert's code, which we could
include in mpl. Robert, what is the reason this hasn't gone into
scipy proper, and do you see any problems with us folding it into mpl
for use by griddata?
Thanks,
JDH

On Wednesday 16 July 2008 07:20:59 am Ian Harry wrote:
> [spxiwh@sugar 07:14 AM matplotlib]$ diff texmanager.py
> /usr/lib64/python2.4/site-packages/matplotlib/texmanager.py
> 248c248
> < fh = file(outfile,'a')
> ---
>
> > fh = file(outfile)
>
> 252,254c252
> < else:
> < try: verbose.report(fh.read(), 'debug')
> < except: pass
> ---
>
> > else: verbose.report(fh.read(), 'debug')
>
> 259,261c257,258
> < else:
> < try: os.remove(fname)
> < except: pass
> ---
>
> > else: os.remove(fname)
>
> 280c277
> < fh = file(outfile,'a')
> ---
>
> > fh = file(outfile)
>
> 285,287c282
> < else:
> < try: verbose.report(fh.read(), 'debug')
> < except: pass
> ---
>
> > else: verbose.report(fh.read(), 'debug')
>
> 289,290c284
> < try: os.remove(outfile)
> < except: pass
> ---
>
> > os.remove(outfile)
>
> 314c308
> < # else: verbose.report(fh.read(), 'debug')
> ---
>
> > else: verbose.report(fh.read(), 'debug')
>
> --snip--
I took a different approach:
Index: lib/matplotlib/texmanager.py
===================================================================
--- lib/matplotlib/texmanager.py (revision 5771)
+++ lib/matplotlib/texmanager.py (working copy)
@@ -273,16 +273,22 @@
 %(os.path.split(texfile)[-1], outfile))
 mpl.verbose.report(command, 'debug')
 exit_status = os.system(command)
- fh = file(outfile)
+ try:
+ fh = file(outfile)
+ report = fh.read()
+ fh.close()
+ except IOError:
+ report = 'No latex error report available.'
 if exit_status:
 raise RuntimeError(('LaTeX was not able to process the 
following \
-string:\n%s\nHere is the full report generated by LaTeX: \n\n'% repr(tex)) + 
fh.read())
- else: mpl.verbose.report(fh.read(), 'debug')
- fh.close()
+string:\n%s\nHere is the full report generated by LaTeX: \n\n'% repr(tex)) + 
report)
+ else: mpl.verbose.report(report, 'debug')
 for fname in glob.glob(basefile+'*'):
 if fname.endswith('dvi'): pass
 elif fname.endswith('tex'): pass
- else: os.remove(fname)
+ else:
+ try: os.remove(fname)
+ except OSError: pass
 return dvifile
@@ -305,14 +311,19 @@
 os.path.split(dvifile)[-1], outfile))
 mpl.verbose.report(command, 'debug')
 exit_status = os.system(command)
- fh = file(outfile)
+ try:
+ fh = file(outfile)
+ report = fh.read()
+ fh.close()
+ except IOError:
+ report = 'No dvipng error report available.'
 if exit_status:
 raise RuntimeError('dvipng was not able to \
 process the flowing file:\n%s\nHere is the full report generated by dvipng: \
-\n\n'% dvifile + fh.read())
- else: mpl.verbose.report(fh.read(), 'debug')
- fh.close()
- os.remove(outfile)
+\n\n'% dvifile + report)
+ else: mpl.verbose.report(report, 'debug')
+ try: os.remove(outfile)
+ except OSError: pass
 return pngfile
Would you update from svn and see if it works for you?
Thanks,
Darren

On Thu, Jul 17, 2008 at 1:22 AM, Angela Rivera Campos <riv...@in...> wrote:
> Well, I can assure you that I am neither working as root, I might be a
> newbie to matplotlib but not to linux, nor have any problem with the
> display. This problem is happening in both machines, remote (ssh -X, for
> sure, I've tested several times) and local, so no ssh conection. Anyway,
> I'm going to test a few more things and will let you know what happens.
Very strange indeed -- the error clear indicates you cannot connect to
the display. You should see the same in the same nvironment if you
simply
>>> import gtk
as the first line. Do you?
JDH

 > I'd love to see it included to -- I believe the problem is finding a
 > good code that is BSD compatible.
Yes.
Some examples on plotting data using spatial interpolation would be very 
nice.
One with the delauny package:
see below at: http://scipy.org/scipy/scikits/
And one with griddata:
http://code.google.com/p/griddata-python/
We could use the accepted data sets from
http://www.itc.nl/personal/rossiter/teach/lecnotes.html#l6
I have tested the griddata package but didn't reach very far due to 
other task to be accomplished earlier.
Kind regards,
Timmie

Chee Sing Lee wrote:
> Hello,
> 
> I'm pretty new to matplotlib, so please bear with me.
> 
> I am writing an application that includes a hexagonally binned scatter
> plot that updates periodically, up to a couple times a second. My
> current approach is clearing axes.collections, calling hexbin() with
> the updated data, and then calling canvas.draw(). It works, but it is
> entirely too slow. I would like to implement the blitting animation
> technique shown in the matplotlib cookbook, but the collection object
> returned by hexbin() doesn't seem to have any sort of set_data method
> to update the data. Any suggestions?
My guess is that the entire hexbin method is fundamentally slow. Among 
other things, it is looping through all input values in python. In 
addition, the rendering, using a poly collection, is similar to pcolor, 
and that is also slow when the number of polygons is large. Profiling a 
run of hexbin might show whether there are obvious opportunities to 
speed it up.
Eric
> 
> I am using the wxPython GUI tookit, and FigureCanvasWxAgg.
> 
> Thanks,
> Chee Sing Lee
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

>> If you cannot open the display, most likely it is because either you
>> are running as root (bad, bad) or you are running over a remot
>> connection (ssh) and do not have X11 forwarding properly enabled or
>> configures. If you want to use mpl with an interactive window, you
>> will need to have access to the display. The fact that you see this
>> error only on the first time you import pylab is typical of exceptions
>> that are thrown at module import time -- python only tries to import
>> once so you see the exception only once. To solve this, trying
>> running as a normal user rather than root or sudo, and if you are on a
>> remote machine, try using ssh -X to enable X11 forwarding. We will
>> need to know more about exactly what you are doing to help.
> 
> For remote access you also need to allow the remote machine to access 
> the display via "xhost +name" on the local machine.
> 
Well, I can assure you that I am neither working as root, I might be a 
newbie to matplotlib but not to linux, nor have any problem with the 
display. This problem is happening in both machines, remote (ssh -X, for 
sure, I've tested several times) and local, so no ssh conection. Anyway, 
I'm going to test a few more things and will let you know what happens.
AR