matplotlib-devel Mailing List for matplotlib

Stephen Walton wrote:
> John Hunter wrote:
> 
> 
>>So you're saying you get a segfault on CVS with tkagg?
>>
>>Would you be willing to try a few things
>>
>>* rm -rf site-packages/matplotlib and your build subdir and do a
>> clean build. Do you still see the problem?
>> 
>>
> 
> I did both this and rm -rf /usr/share/matplotlib. This fixed the 
> problem. I had done 'rpm -U' after "setup.py bdist_rpm" to replace 0.71 
> with 0.72. I may try to go back and duplicate the problem, but not today.
Don't bother, I doubt you'll be able to. It was most likely caused by a clash 
with old extension code. I'm pounding on the TkAgg backend heavily, with 
multiple plot figures open concurrently with multiple MayaVi windows (which 
have VTK render windows embedded), and I don't have any problems (other than a 
VTK window destruction bug John is fixing at this very moment :)
Best,
f

John Hunter wrote:
>So you're saying you get a segfault on CVS with tkagg?
>
>Would you be willing to try a few things
>
> * rm -rf site-packages/matplotlib and your build subdir and do a
> clean build. Do you still see the problem?
> 
>
I did both this and rm -rf /usr/share/matplotlib. This fixed the 
problem. I had done 'rpm -U' after "setup.py bdist_rpm" to replace 0.71 
with 0.72. I may try to go back and duplicate the problem, but not today.

John Hunter wrote:
>>>>>>"Stephen" == Stephen Walton <ste...@cs...> writes:
> 
> 
> Stephen> I appreciate the effort. It works fine provided I delete
> Stephen> the line "backend: TkAgg" which I'd had in my
> Stephen> ~/.matplotlibrc file. Otherwise I get a core dump. I
> Stephen> don't recall why I was using this instead of the default
> Stephen> GTKAgg backend.
> 
> So you're saying you get a segfault on CVS with tkagg?
> 
> Would you be willing to try a few things
> 
> * rm -rf site-packages/matplotlib and your build subdir and do a
> clean build. Do you still see the problem?
Alternatively, (and this allows you to keep a 'safe' matplotlib in-place, or 
switch to it as needed), my stategy is:
1. Rename site-packages/matplotlib to .ori, and symlink to the CVS tree:
root@planck[site-packages]# d -d matplotlib*
/usr/lib/python2.3/site-packages
lrwxrwxrwx 1 root 51 Feb 7 16:57 matplotlib -> 
/usr/local/installers/src/matplotlib/lib/matplotlib/
drwxr-xr-x 4 fperez 4096 Feb 4 10:01 matplotlib.ori/
2. In the CVS tree, manually symlink all the shared libraries to the path of 
the build dir:
root@planck[matplotlib]# dl
/usr/local/installers/src/matplotlib/lib/matplotlib
lrwxrwxrwx 1 fperez 83 Feb 7 16:59 ft2font.so -> 
/usr/local/installers/src/matplotlib/build/lib.linux-i686-2.3/matplotlib/ft2font.so*
lrwxrwxrwx 1 fperez 87 Feb 7 16:58 _nc_contour.so -> 
/usr/local/installers/src/matplotlib/build/lib.linux-i686-2.3/matplotlib/_nc_contour.so*
lrwxrwxrwx 1 fperez 85 Feb 7 16:58 _nc_image.so -> 
/usr/local/installers/src/matplotlib/build/lib.linux-i686-2.3/matplotlib/_nc_image.so*
lrwxrwxrwx 1 fperez 90 Feb 7 16:58 _nc_transforms.so -> 
/usr/local/installers/src/matplotlib/build/lib.linux-i686-2.3/matplotlib/_nc_transforms.so*
root@planck[matplotlib]# dl backends
/usr/local/installers/src/matplotlib/lib/matplotlib
lrwxrwxrwx 1 fperez 92 Feb 7 16:58 _gtkagg.so -> 
/usr/local/installers/src/matplotlib/build/lib.linux-i686-2.3/matplotlib/backends/_gtkagg.so*
lrwxrwxrwx 1 fperez 100 Feb 9 14:55 _nc_backend_agg.so -> 
/usr/local/installers/src/matplotlib/build/lib.linux-i686-2.3/matplotlib/backends/_nc_backend_agg.so*
lrwxrwxrwx 1 fperez 91 Feb 7 16:58 _tkagg.so -> 
/usr/local/installers/src/matplotlib/build/lib.linux-i686-2.3/matplotlib/backends/_tkagg.so*
These symlinks can be established in 10 seconds with md anc C-x-s.
3. When you do a CVS update in the mpl top-level dir, simply do
rm -rf build
python setup.py build
This will rebuild all the needed extensions, and your symlinks will pick them 
up automagically.
If you ever need to go back to the released mpl, simply repoint the matplotlib 
symlink in site-packages to .ori.
This is the simplest setup I've been able to find to handle transparently CVS 
python code with extensions. It only breaks if the CVS code springs a new 
extension module (or renames one), but that is fixed in a second with the 
proper new symlink.
Cheers,
f

>>>>> "Stephen" == Stephen Walton <ste...@cs...> writes:
 Stephen> I appreciate the effort. It works fine provided I delete
 Stephen> the line "backend: TkAgg" which I'd had in my
 Stephen> ~/.matplotlibrc file. Otherwise I get a core dump. I
 Stephen> don't recall why I was using this instead of the default
 Stephen> GTKAgg backend.
So you're saying you get a segfault on CVS with tkagg?
Would you be willing to try a few things
 * rm -rf site-packages/matplotlib and your build subdir and do a
 clean build. Do you still see the problem?
 * If so, repeat above but recompile matplotlib setting verbose = True
 in setup.py and rerun. If you get the segfault, post the last
 score or so lines of output
 * send a script that generates the fault.
Thanks!
JDH

Fernando Perez wrote:
> Here's a tarball of current CVS I just made for you (I have ssh dev 
> access, which is up):
I appreciate the effort. It works fine provided I delete the line 
"backend: TkAgg" which I'd had in my ~/.matplotlibrc file. Otherwise I 
get a core dump. I don't recall why I was using this instead of the 
default GTKAgg backend.
> I would tweak the scipy profile to do a 'import scipy as S',
Done. I've been doing this manually in any event.
> John and I (with many others) are working together on an NIH grant 
> proposal which, if it goes through, will give us some funds to work on 
> ipython/pylab/scipy issues for a while.
That would be Way Cool if you can get the money.

Everything Fernando said it exactly right, but your problem is a
common on in pre-CVS matlab. FYI, Here is what is happening. It
appears you have previously plotted some nonpositve data. pre-CVS
matplotlib maintained the data limits as min/max for y and y which it
uses in the autoscaler. CVS matplotlib keeps track of min, max and
min-positive so the log autoscaler can do the right thing.
It also implements lots of other log fixes and optimizations. Once
this last bug is hammered out, we hope that you won't be able to break
matplotlib (in agg, other backends haven't ported the new changes yet)
with log plots, with the possible exception of plotting *all*
nonpositive data. But if do that, you deserve what you get :-)
JDH

Stephen Walton wrote:
> I can't seem to get loglog plots to work. Attached is a dump of ipython 
> showing what I tried and what happened. In case you don't want to wade 
> through this, just try the following:
> 
> hold(False)
> x=arange(25)+1
> loglog(x,x**2) # throws error
> close(1)
> plot(log(x),log(x**2)) # works
> close(1)
> plot(x,x**2)
> plot(log(x),log(x**2)) # throws error
> loglog(x,x**2) # also throws error
This works fine, but only with current CVS. I've been giving John a ton of 
nasty tests for log plotting, and he's fixed most problems. I do still have a 
dataset which causes an abort in the C++ extension code, but he's looking into 
it already. So in short, if you update to CVS, most of your log problems will 
be fixed. And John should have the last remaining nasty killed soon.
Additionally, CVS has dramatic performance improvements in several areas, 
noticeable mostly for plots with many elements.
Best,
f

I can't seem to get loglog plots to work. Attached is a dump of ipython 
showing what I tried and what happened. In case you don't want to wade 
through this, just try the following:
hold(False)
x=arange(25)+1
loglog(x,x**2) # throws error
close(1)
plot(log(x),log(x**2)) # works
close(1)
plot(x,x**2)
plot(log(x),log(x**2)) # throws error
loglog(x,x**2) # also throws error
Here's a result of doing something similar to the above manually, with 
traceback:
freyer:~> ipython -pylab
Python 2.3.4 (#1, Feb 2 2005, 12:11:53)
Type "copyright", "credits" or "license" for more information.
IPython 0.6.7 -- An enhanced Interactive Python.
? -> Introduction to IPython's features.
%magic -> Information about IPython's 'magic' % functions.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.
 Welcome to pylab, a matplotlib-based Python environment
 help(matplotlib) -> generic matplotlib information
 help(pylab) -> matlab-compatible commands from matplotlib
 help(plotting) -> plotting commands
In [1]: x=arange(10)+1
In [2]: plot(x,x**2)
Out[2]: [<matplotlib.lines.Line2D instance at 0x410f132c>]
In [3]: loglog(x,x**2)
---------------------------------------------------------------------------
exceptions.ValueError Traceback (most 
recent call last)
/home/swalton/<console>
/usr/lib/python2.3/site-packages/matplotlib/pylab.py in loglog(*args, 
**kwargs)
 1869 hold(b)
 1870 else:
-> 1871 draw_if_interactive()
 1872
 1873 hold(b)
/usr/lib/python2.3/site-packages/IPython/genutils.py in wrapper(*args, **kw)
 686 def wrapper(*args,**kw):
 687 wrapper.called = False
--> 688 out = func(*args,**kw)
 689 wrapper.called = True
 690 return out
/usr/lib/python2.3/site-packages/matplotlib/backends/__init__.py in 
draw_if_interactive()
 39 def draw_if_interactive():
 40 draw_if_interactive._called = True
---> 41 __draw_int()
 42 # Flag to store state, so external callers (like ipython) 
can keep track
 43 # of draw calls.
/usr/lib/python2.3/site-packages/matplotlib/backends/backend_tkagg.py in 
draw_if_interactive()
 56 figManager = Gcf.get_active()
 57 if figManager is not None:
---> 58 figManager.show()
 59
 60
/usr/lib/python2.3/site-packages/matplotlib/backends/backend_tkagg.py in 
show(self)
 275 # anim.py requires this
 276 if sys.platform=='win32' : self.window.update()
--> 277 else: self.canvas.draw()
 278 self._shown = True
 279
/usr/lib/python2.3/site-packages/matplotlib/backends/backend_tkagg.py in 
draw(self)
 140
 141 def draw(self):
--> 142 FigureCanvasAgg.draw(self)
 143 tkagg.blit(self._tkphoto, self.renderer._renderer, 2)
 144 self._master.update_idletasks()
/usr/lib/python2.3/site-packages/matplotlib/backends/backend_agg.py in 
draw(self)
 306 self.renderer = RendererAgg(w, h, self.figure.dpi)
 307 self._lastKey = key
--> 308 self.figure.draw(self.renderer)
 309
 310 def tostring_rgb(self):
/usr/lib/python2.3/site-packages/matplotlib/figure.py in draw(self, 
renderer)
 332
 333 # render the axes
--> 334 for a in self.axes: a.draw(renderer)
 335
 336 # render the figure text
/usr/lib/python2.3/site-packages/matplotlib/axes.py in draw(self, renderer)
 1167 if not self.get_visible(): return
 1168 renderer.open_group('axes')
-> 1169 self.transData.freeze() # eval the lazy objects
 1170 self.transAxes.freeze() # eval the lazy objects
 1171 if self.axison:
ValueError: Cannot take log of nonpositive value
In [4]: plot(log(x),log(x**2))
---------------------------------------------------------------------------
exceptions.ValueError Traceback (most 
recent call last)
/home/swalton/<console>
/usr/lib/python2.3/site-packages/matplotlib/pylab.py in plot(*args, 
**kwargs)
 1957 hold(b)
 1958 else:
-> 1959 draw_if_interactive()
 1960
 1961 hold(b)
/usr/lib/python2.3/site-packages/IPython/genutils.py in wrapper(*args, **kw)
 686 def wrapper(*args,**kw):
 687 wrapper.called = False
--> 688 out = func(*args,**kw)
 689 wrapper.called = True
 690 return out
/usr/lib/python2.3/site-packages/matplotlib/backends/__init__.py in 
draw_if_interactive()
 39 def draw_if_interactive():
 40 draw_if_interactive._called = True
---> 41 __draw_int()
 42 # Flag to store state, so external callers (like ipython) 
can keep track
 43 # of draw calls.
/usr/lib/python2.3/site-packages/matplotlib/backends/backend_tkagg.py in 
draw_if_interactive()
 56 figManager = Gcf.get_active()
 57 if figManager is not None:
---> 58 figManager.show()
 59
 60
/usr/lib/python2.3/site-packages/matplotlib/backends/backend_tkagg.py in 
show(self)
 275 # anim.py requires this
 276 if sys.platform=='win32' : self.window.update()
--> 277 else: self.canvas.draw()
 278 self._shown = True
 279
/usr/lib/python2.3/site-packages/matplotlib/backends/backend_tkagg.py in 
draw(self)
 140
 141 def draw(self):
--> 142 FigureCanvasAgg.draw(self)
 143 tkagg.blit(self._tkphoto, self.renderer._renderer, 2)
 144 self._master.update_idletasks()
/usr/lib/python2.3/site-packages/matplotlib/backends/backend_agg.py in 
draw(self)
 306 self.renderer = RendererAgg(w, h, self.figure.dpi)
 307 self._lastKey = key
--> 308 self.figure.draw(self.renderer)
 309
 310 def tostring_rgb(self):
/usr/lib/python2.3/site-packages/matplotlib/figure.py in draw(self, 
renderer)
 332
 333 # render the axes
--> 334 for a in self.axes: a.draw(renderer)
 335
 336 # render the figure text
/usr/lib/python2.3/site-packages/matplotlib/axes.py in draw(self, renderer)
 1167 if not self.get_visible(): return
 1168 renderer.open_group('axes')
-> 1169 self.transData.freeze() # eval the lazy objects
 1170 self.transAxes.freeze() # eval the lazy objects
 1171 if self.axison:
ValueError: Cannot take log of nonpositive value
In [5]: close(1)
In [6]: plot(log(x),log(x**2))
Out[6]: [<matplotlib.lines.Line2D instance at 0x4154094c>]
In [7]: close(1)
In [8]: loglog(x,x**2)
Out[8]: [<matplotlib.lines.Line2D instance at 0x4168570c>]
In [9]: Exception in Tkinter callback
Traceback (most recent call last):
 File "/usr/lib/python2.3/lib-tk/Tkinter.py", line 1345, in __call__
 return self.func(*args)
 File 
"/usr/lib/python2.3/site-packages/matplotlib/backends/backend_tkagg.py", 
line 139, in resize
 self.show()
 File 
"/usr/lib/python2.3/site-packages/matplotlib/backends/backend_tkagg.py", 
line 142, in draw
 FigureCanvasAgg.draw(self)
 File 
"/usr/lib/python2.3/site-packages/matplotlib/backends/backend_agg.py", 
line 308, in draw
 self.figure.draw(self.renderer)
 File "/usr/lib/python2.3/site-packages/matplotlib/figure.py", line 
334, in draw
 for a in self.axes: a.draw(renderer)
 File "/usr/lib/python2.3/site-packages/matplotlib/axes.py", line 1169, 
in draw self.transData.freeze() # eval the lazy objects
ValueError: Cannot take log of nonpositive value

On Thu, 2005年02月10日 at 10:03, Fer...@co... wrote:
> Quoting Perry Greenfield <pe...@st...>:
> 
> > I'm only going from memory but my recollection is that the code to convert
> > arrays to strings is largely written in Python (we took it from Numeric).
> > Perhaps that has changed (for Numeric or numarray). My recollection is
> > that the Python code made use of many array manipulations so it should
> > be faster than element-by-element string conversions, but I also remember
> > thinking that this was an area that could potentially be speeded up by
> > conversion to C. In short, don't assume that the current array2str code
> > is in C (Todd may be able to correct me).
Your impression is correct; array2str was and still is mostly Python.
Todd

Quoting Perry Greenfield <pe...@st...>:
> I'm only going from memory but my recollection is that the code to convert
> arrays to strings is largely written in Python (we took it from Numeric).
> Perhaps that has changed (for Numeric or numarray). My recollection is
> that the Python code made use of many array manipulations so it should
> be faster than element-by-element string conversions, but I also remember
> thinking that this was an area that could potentially be speeded up by
> conversion to C. In short, don't assume that the current array2str code
> is in C (Todd may be able to correct me).
This is actually something which may be worth improving upon for future
numeric(3)/numarray versions. A fast, reliable, _and_ flexible string
formatter for arrays can be extremely useful in many contexts. Something where
you can control the format with precision, the line and item separators, and a
few other things, but with good speed, would be great. In current Numeric, you
can control item but not line (dimension) separators, for example (each line is
bracketed in []).
With such a formatter, this particular problem for PS formatting (and perhaps
others in mpl) could be delegated to the arrays themselves.
Best,
f

Fernando Perez wrote:
> However, I'm afraid to rewrite this in a low-level way, because of the
> numeric/numarray difference. The right approach for this would
> be to generate
> a string representation of the array via numeric/numarray, which
> can do it in
> C. And then, that can be modified to add the m/l end of line
> markers on the
> first/rest lines via a (fast) python string operation.
>
> But since I don't know whether numeric/numarray provide fully consistent
> array2str functions (I only have numeric to test with), I'm a bit
> afraid of
> touching this part. It's also possible that John's backend architectural
> changes end up modifying this, so perhaps such changes are best
> thought about
> after John finishes his reorganization.
>
I'm only going from memory but my recollection is that the code to convert
arrays to strings is largely written in Python (we took it from Numeric).
Perhaps that has changed (for Numeric or numarray). My recollection is
that the Python code made use of many array manipulations so it should
be faster than element-by-element string conversions, but I also remember
thinking that this was an area that could potentially be speeded up by
conversion to C. In short, don't assume that the current array2str code
is in C (Todd may be able to correct me).
Perry

The first matplotlib toolkit has been released.
Matplotlib toolkits are collections of application-specific functions 
that extend matplotlib.
This is hopefully the first of many - additional contributions are 
encouraged!
The basemap toolkit allows matplotlib to plot regularly-space 
latitiude/longitude grids on map projections, including drawing 
coastlines, political boundaries, parallels and meridians. It 
currently supports six map projections (cylindrical equidistant, 
mercator, lambert conformal conic, lambert azimuthal equal area, albers 
equal area and stereographic).
It uses routines from the proj.4 library (http://proj.maptools.org) to 
perform cartographic transformations.
Documentation - http://matplotlib.sourceforge.net/toolkits.html.
Screenshot - http://matplotlib.sourceforge.net/screenshots/plotmap_large.png
Example Code - http://matplotlib.sourceforge.net/screenshots/plotmap.py
Examples for each map projection are included in the source release 
(available at the sf download page 
(http://sourceforge.net/project/showfiles.php?group_id=80706&package_id=142792).
Windows binaries are provided (thanks John!) - but they do not include 
the examples.
Comments/bug reports/suggestions are welcome.
-Jeff
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
NOAA/OAR/CDC R/CDC1 FAX : (303)497-6449
325 Broadway Web : http://www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80305-3328 Office: Skaggs Research Cntr 1D-124

Hi John,
> Hmm, I thought the point of doing the Skip call
> 
> def _onKeyDown(self, evt):
> """Capture key press."""
> key = self._get_key(evt)
> evt.Skip()
> FigureCanvasBase.key_press_event(self, key)
> 
> would be to prevent the event from being swallowed. So you should be
> able to get those events if you've made a second connection, right?
> 
> If so, what you are arguing for is the ability to get the GUI event
> within the mpl event because this is more convenient than making two
> connections. Is that right?
Yes, it's a matter of convenience. What isn't? If the mpl
event include the GUI event, you can define one MyKeyDown
method, bind it once with
 canvas.mpl_connect('key_press_event',MyKeyDown) 
and have it handle the whole event. To me, this seems cleaner
and easier than having two separate bindings and event handlers
for the mpl and GUI events. I've been doing this, and it's sort 
of ugly to have that many event handlers.
I might be the only one who cares, but I'm binding right-click
over the canvas to show a pop-up menu. I'd like to have similar
functionality in both WX and Tk (WX for new code, but Tk for
legacy apps that currently use Pmw/Blt.graph), and am hoping for
as much overlapping code as possible.
> You can always make the change with guiEvent=None in the
> __init__function so other backends can add it when/if they want it.
> It looks like a good idea to me to add it across backends, though.
Using guiEvent=None in the __init__() is my intention. In fact,
I believe I have this implemented for backend_bases.py,
backend_wx.py, backend_tkagg.py, and backend_gtk.py. I could
not quite figure out how to do this in backend_fltkagg.py or
backend_qtagg.py (any pointers??). I haven't had time to
adequately test this with all event types on wx/tk, and will
probably not get a chance to do so until early next week. 
Again, it's all small changes, basically adding
',guiEvent=guiEvent/None/event/evt' to many argument lists.
--Matt
PS: FWIW, I see a noticeable speed increase with WXAgg on Mac
OSX from matplotlib-0.71 to the cvs version. Excellent!!

 Perry> It might be worth generalizing this to take some optional
 Perry> array/scalar arguments to apply marker by marker scaling
 Perry> (or even independent x/y scaling) as well as overrides to
 Perry> some gc items (e.g. color). This doesn't need to be done
 Perry> right away, but may provide some very useful capability
 Perry> (e.g., error bars could be done as markers if y or x size
 Perry> could be scaled for every x,y location.
What you are describing is basically the draw_*collection methods.
But my experience with them has led me to believe these methods are
not ideal -- for example the i%N indexing of collections is
syntactically convenient but practically rarely useful. In real life,
for a given property, you either want a shared property (marker color)
or you want that property to be different at every pointt (scatter
color with a cmap). And in cases where you have a few properties,
each with lots of segments (read contour) it's fine to create a few
objects rather than have one monolithic object (as Nadia did).
One nice thing about the current draw_marker implementation is that it
is easy (it uses a gc, like all the other drawing methods). Once we
permit scalex and scaley, we'll also want to add rgba (with scalex,
scaley, and rgba as possible len(x) sequences, it could handle
scatters). And then once you allow all these to vary, you might start
thinking about letting every property vary, which is what collections
already do, and which apparently are sufficiently complex that
everyone but agg falls back on the base class implementation, which
while a good bit faster than the bad old days, is still slower than it
should be.
On the subject of speed, the new draw_markers *is* fast. Close to as
fast as you can get, I'd venture, for anti-aliased drawing with alpha
support. I spent a lot of time in the profiler and there isn't much
fat left to be trimmed -- most of the cost is now pure agg, and it's
agg that Maxim helped me optimize by using cached rasters. None of
this precludes your idea, though, because agg is so damned cool that
you can scale and colorize cached rasters pretty cheaply. FYI, here
are my latest benchmark numbers on draw_markers in agg
 0.71 CVS speedup 
 -----------------------------------
 N = 1000 | 0.24s | 0.13s | 1.85x
 N = 5000 | 0.68s | 0.19s | 3.57x
 N = 10000 | 1.17s | 0.28s | 4.19x
 N = 50000 | 5.30s | 0.60s | 8.89x
 N = 100000 | 10.02s | 0.70s | 14.31x
 N = 500000 | 48.81s | 2.32s | 21.03x
As an aside, errorbars, which you mentioned, present a slightly more
complicated case, because the errorbar caplines scale independently of
the x and y error scale.
After discussions with Fernando, who never tires of pointing out the
joys of gnuplot to me :-), I've come to agree that matplotlib ought to
extend the Artists to include "plot items" classees; eg a plot command
should create a LineItem, a scatter command should create a
ScatterItem, an errorbar plot should create and ErrorbarItem. And we
should have, like gnuplot, FuncItems and a few more. This would
satisfy something about the design that has always bothered me --
where should the fundamental plot type functions go? Currently they
are axes methods but this has always stuck me as wrong -- it just
doesn't have the right feel. I think that having high level artists
like ErrorbarItem, which contain the primitives necessary to draw
them, might be the way to go. One could then mix and match various
plot items in different axes and figures.
This might also make the pure OO users happier, who sometimes grumble
that the OO interface is too hard to use. 
JDH