matplotlib-devel Mailing List for matplotlib (Page 7)

Hi Jeff!
I've been using your basemap package as a way to skip the thinking step
when plotting stuff on spheres, which I often do in relation to my
studies of flies' eyes... Anyhow, given that basemap comes with 82 MB
of Earth-specific data which I don't need all the time, I think it would
be really great if it would be possible to make the inclusion of such
data optional. I attach a patch which suffices for my extremely limited
needs (I simply call the __call__ method of the Basemap.Basemap class),
but doubtless causes breakage elsewhere. So, I haven't checked it in,
hoping that you might take the idea and make it a little less brittle.
I suspect there are lots of people who would like to plot things on
spheres, but wouldn't necessarily want the inclusion of coastlines,
rivers, and such. ;) So, I think this is a general purpose improvement
and ask you to consider completing the job.
Unrelated note 1: I committed a patch to setup.py and setupegg.py to
make the thing respect setuptools even if setupegg.py wasn't run.
Unrelated note 2: The setuptools/.egg approach to data is to include it
with the .py files. Initially I thought about modifying the basemap data
path scheme to support this, but in the case of 82M, though, I'd say
that's stretching it a bit, especially with the tendancy for people
(well, me, at least), to have the last 5-100 versions of the .eggs I
built lying in site-packages.
Cheers!
Andrew

Darren,
> I suppose it could be an issue with my gs, a few weeks ago I upgraded to 
> gcc-4.1 and have since recompiled gs. Would you run this script:
> 
> from pylab import *
> x=rand(1000)
> y=rand(1000)
> plot(x,y)
> savefig('apitest.eps')
> 
> and then open the file in ggv? If it opens quickly, maybe you could also try 
> rand(10000).
About 25 seconds on a 1.6G Pentium M, ESP Ghostscript 8.15.1 (2005年09月22日)
Eric

Hi John,
On Monday 03 April 2006 10:16 pm, you wrote:
> >>>>> "Darren" == Darren Dale <dd...@co...> writes:
>
> Darren> I think there are two issues here. The postscript
> Darren> interpreter doesnt like to stroke long paths. If I break
> Darren> the path into 50 point chunks instead of 1000 point
> Darren> chunks, I see an improvement.
>
> That's odd, because we never saw an issue stroking long pats before.
> Or at least noone has ever reported it.
I suppose it could be an issue with my gs, a few weeks ago I upgraded to 
gcc-4.1 and have since recompiled gs. Would you run this script:
from pylab import *
x=rand(1000)
y=rand(1000)
plot(x,y)
savefig('apitest.eps')
and then open the file in ggv? If it opens quickly, maybe you could also try 
rand(10000).
> Darren> Also, if I do the transform in backend_ps, instead of
> Darren> passing it to the postscript interpreter, I see a big
> Darren> speedup at render time. Right now I am doing this
> Darren> transform by hand:
>
> Darren> a,b,c,d,tx,ty=vec6 xo = a*x+c*y +tx yo = b*x+d*y + ty x,y
> Darren> = xo,yo
>
> Darren> Is there a better way to do this? I thought I could simply
> Darren> call numerix_x_y, but that function is not compatible with
> Darren> nonlinear transforms (returns a domain error if one of the
> Darren> axes is log-scaled).
>
> You can transform the xy values point by point. Instead of separating
> them into their nonlinear and affine components as we are currently
> doing in backend_ps, you can call trans.xy_tup which will do both. If
> successful, it will return the transformed xy, if it fails, it will
> raise a ValueError, and you can set the moveto/lineto state
> accordingly.
I see. But can that be done in a list comprehension? I think it will require a 
regular for loop. I think doing the transforms as I described above is a 
better solution. It fixed the render times on my machine and my 
backend_driver tests looked fine. I'll wait for feedback, and test a little 
more tomorrow before committing.
> But I am still confused by your previous post: you said that when you
> tried to load the simple plot example PS file in gs, the line rendered
> quickly, and then there was the interminable pause. This suggests
> that it is not the transformation in gs that is the bottleneck, but
> something that happens after it.
It was axes_demo.py. I discovered the reason: a 10,000 pt line is written to 
the postscript file, but only 1000 points are drawn because of the x-axis 
limits. So it was churning away, processing points that dont get drawn. I 
confirmed this was the case: by commenting out the other 9000 points in 
axes_demo_PS.ps, the render time improved.
Darren

>>>>> "Darren" == Darren Dale <dd...@co...> writes:
 Darren> I think there are two issues here. The postscript
 Darren> interpreter doesnt like to stroke long paths. If I break
 Darren> the path into 50 point chunks instead of 1000 point
 Darren> chunks, I see an improvement.
That's odd, because we never saw an issue stroking long pats before.
Or at least noone has ever reported it.
 Darren> Also, if I do the transform in backend_ps, instead of
 Darren> passing it to the postscript interpreter, I see a big
 Darren> speedup at render time. Right now I am doing this
 Darren> transform by hand:
 Darren> a,b,c,d,tx,ty=vec6 xo = a*x+c*y +tx yo = b*x+d*y + ty x,y
 Darren> = xo,yo
 Darren> Is there a better way to do this? I thought I could simply
 Darren> call numerix_x_y, but that function is not compatible with
 Darren> nonlinear transforms (returns a domain error if one of the
 Darren> axes is log-scaled).
You can transform the xy values point by point. Instead of separating
them into their nonlinear and affine components as we are currently
doing in backend_ps, you can call trans.xy_tup which will do both. If
successful, it will return the transformed xy, if it fails, it will
raise a ValueError, and you can set the moveto/lineto state
accordingly.
But I am still confused by your previous post: you said that when you
tried to load the simple plot example PS file in gs, the line rendered
quickly, and then there was the interminable pause. This suggests
that it is not the transformation in gs that is the bottleneck, but
something that happens after it.
In any case, here is an example script creating a semilogx transform.
The first transformation succeeds, the second raises a ValueError
from matplotlib.transforms import SeparableTransformation, \
 Point, Value, Bbox, LOG10, IDENTITY, Func
import matplotlib.numerix as nx
# make some random bbox transforms 
def rand_point():
 x,y = nx.mlab.rand(2)
 return Point( Value(x), Value(y) )
def rand_bbox():
 ll = rand_point()
 ur = rand_point()
 return Bbox(ll, ur)
b1 = rand_bbox()
b2 = rand_bbox()
funcx = Func(LOG10)
funcy = Func(IDENTITY)
# semilogx
trans = SeparableTransformation(b1, b2, funcx, funcy) 
print trans.xy_tup((1,2)) #ok
print trans.xy_tup((-1,2)) #raises

On Monday 03 April 2006 11:16 am, John Hunter wrote:
> >>>>> "Darren" == Darren Dale <dd...@co...> writes:
>
> Darren> I'm having second thoughts about the wisdom of having
> Darren> postscript handle the transforms. With the new API, I run
> Darren> backend_driver.py and get a file called
> Darren> axes_demo_PS.ps. On my machine, it takes about 10 seconds
> Darren> to open this file if it was created with the new API. If I
> Darren> mask draw_markers and recreate the postscript file, it
> Darren> loads instantly.
>
> So you think the performance hit is caused by gs (or whatever viewer
> you are using) by the postscript engine doing the transformations? It
> surprises me that this would be so inefficient.
>
> We have the option of doing the transformations in the ps backend...
I think there are two issues here. The postscript interpreter doesnt like to 
stroke long paths. If I break the path into 50 point chunks instead of 1000 
point chunks, I see an improvement.
Also, if I do the transform in backend_ps, instead of passing it to the 
postscript interpreter, I see a big speedup at render time. Right now I am 
doing this transform by hand:
a,b,c,d,tx,ty=vec6
xo = a*x+c*y +tx
yo = b*x+d*y + ty
x,y = xo,yo
Is there a better way to do this? I thought I could simply call numerix_x_y, 
but that function is not compatible with nonlinear transforms (returns a 
domain error if one of the axes is log-scaled).
Thanks,
Darren

On 4/3/06, John Hunter <jdh...@ac...> wrote:
> >>>>> "Charlie" =3D=3D Charlie Moad <cw...@gm...> writes:
>
> Charlie> I don't know if you saw my latest post before writing
> Charlie> this. Please try two_scales with the latest and I
> Charlie> believe it is satisfactory now.
>
>
> No, I sent my post before reading your last, but the two_scales
> example is acting weird. When I zoom-to-rect, the zoom does not go to
> the rectangle I choose.
From what I am seeing the x axis is zooming too much, but the y axis
is fine. I am guessing this has something to do with the shared axis
being acted on twice. Indeed the release_zoom method of NavTB2 is
being called twice. Any ideas on how to handle this? The limits need
to be adjusted on both plots still, but somehow we need to know to
only adjust the shared axis once.
- Charlie

On Monday 03 April 2006 11:00, Darren Dale wrote:
> I'm having second thoughts about the wisdom of having postscript handle the
> transforms. With the new API, I run backend_driver.py and get a file called
> axes_demo_PS.ps. On my machine, it takes about 10 seconds to open this file
> if it was created with the new API. If I mask draw_markers and recreate the
> postscript file, it loads instantly.
>
> Any thoughts?
Let me make a correction. Viewing the file should not take so long, but there 
is something strange going on that I dont understand. axes_demo_PS.ps 
actually takes almost a full minute to load on my computer if the new API is 
used (excuse my first incorrect report). What I see is that the blue line is 
immediately rendered, and then ghostscript seems to stall for 45 seconds, and 
then renders the rest of the file. If I comment out the blue line in the 
postscript file itself:
% gsave
% 446.4 345.6 72 43.2 clipbox
% [446.4 0 0 6920.64 72 159.326] concat
% 0 0.001725 m
[...]
% 9.999 -4.733e-05 l
% gsave 0.00224014 0.000144495 scale stroke grestore
% grestore
then the file will load immediately, with no pause. This is strange, I cant 
identify the source of the hangup.
Darren

>>>>> "Charlie" == Charlie Moad <cw...@gm...> writes:
 Charlie> I don't know if you saw my latest post before writing
 Charlie> this. Please try two_scales with the latest and I
 Charlie> believe it is satisfactory now.
No, I sent my post before reading your last, but the two_scales
example is acting weird. When I zoom-to-rect, the zoom does not go to
the rectangle I choose.
JDH

>>>>> "Darren" == Darren Dale <dd...@co...> writes:
 Darren> I'm having second thoughts about the wisdom of having
 Darren> postscript handle the transforms. With the new API, I run
 Darren> backend_driver.py and get a file called
 Darren> axes_demo_PS.ps. On my machine, it takes about 10 seconds
 Darren> to open this file if it was created with the new API. If I
 Darren> mask draw_markers and recreate the postscript file, it
 Darren> loads instantly.
So you think the performance hit is caused by gs (or whatever viewer
you are using) by the postscript engine doing the transformations? It
surprises me that this would be so inefficient.
We have the option of doing the transformations in the ps backend...
JDH

On 4/3/06, John Hunter <jdh...@ac...> wrote:
> >>>>> "Charlie" =3D=3D Charlie Moad <cw...@gm...> writes:
>
> Charlie> In this case I think the correct approach would be to set
> Charlie> the zorder higher on one axes, and in that axes' callback
> Charlie> programmatically invoke the same event for the other
> Charlie> axes. I will look at the example and implement/comment
> Charlie> this approach.
>
> I don't feel strongly about this, but since the patch I applied does
> set a navigation toggle flag, is it not preferable to simply fix the
> logic error and allow the user to set this flag rather than have to
> set a callback?
I don't know if you saw my latest post before writing this. Please
try two_scales with the latest and I believe it is satisfactory now.

On 4/3/06, Charlie Moad <cw...@gm...> wrote:
> > It's a little more than zorder, though, since in the case of
> > examples/two_scales.py you have to axes that entirely overlap one
> > another, and you may want both to respond to navigation. In general I
> > like the idea of using zorder to determine which axes responds to
> > navigation commands, but do you think it can handle this case as well.
I wasn't aware of the pylab twinx function which is what broke. The
nav-toolbar events still work for me. The subplots adjust
functionality broke though. I fixed this by accounting for the case
that _sharex or _sharey is an instance of Subplot. I didn't modify
two_scales.py at all and everything seems to be working correctly.
In the case you want custom key or mouse event interaction in
two_scales, one might have to do the zorder trick I mentioned before.=20
Committed.
The broken subplots toolbar might justify a minor release here soon,
unless you think it can wait until 88. Since no one has complained
(except me), it might be able to wait.
- Charlie

I'm having second thoughts about the wisdom of having postscript handle the 
transforms. With the new API, I run backend_driver.py and get a file called 
axes_demo_PS.ps. On my machine, it takes about 10 seconds to open this file 
if it was created with the new API. If I mask draw_markers and recreate the 
postscript file, it loads instantly.
Any thoughts?

>>>>> "Charlie" == Charlie Moad <cw...@gm...> writes:
 Charlie> In this case I think the correct approach would be to set
 Charlie> the zorder higher on one axes, and in that axes' callback
 Charlie> programmatically invoke the same event for the other
 Charlie> axes. I will look at the example and implement/comment
 Charlie> this approach.
I don't feel strongly about this, but since the patch I applied does
set a navigation toggle flag, is it not preferable to simply fix the
logic error and allow the user to set this flag rather than have to
set a callback?
JDH

> It's a little more than zorder, though, since in the case of
> examples/two_scales.py you have to axes that entirely overlap one
> another, and you may want both to respond to navigation. In general I
> like the idea of using zorder to determine which axes responds to
> navigation commands, but do you think it can handle this case as well.
In this case I think the correct approach would be to set the zorder
higher on one axes, and in that axes' callback programmatically invoke
the same event for the other axes. I will look at the example and
implement/comment this approach.
- Charlie

>>>>> "Charlie" == Charlie Moad <cw...@gm...> writes:
 Charlie> John, you added this logic with comment, "added support
 Charlie> for multiple overlapping axes with pan/zoom". I think
This was a user contributed patch, but I did check it in.
 Charlie> for a user to address this problem correctly they should
 Charlie> use the zorder property. I added a check a while back
 Charlie> that sets the inaxes property of a LocationEvent to the
 Charlie> highest zorder if multiple axes are found.
It's a little more than zorder, though, since in the case of
examples/two_scales.py you have to axes that entirely overlap one
another, and you may want both to respond to navigation. In general I
like the idea of using zorder to determine which axes responds to
navigation commands, but do you think it can handle this case as well.
 Charlie> Fixed in svn.
Thanks for looking into this one.
JDH

On 3/31/06, John Hunter <jdh...@ac...> wrote:
> >>>>> "Charlie" =3D=3D Charlie Moad <cw...@gm...> writes:
>
> Charlie> I shoudl have tried this in the first place. The
> Charlie> widgets/radio_buttons.py example doesn't work at all for
> Charlie> me. In fact quite a few are broken.
>
> I noticed today that the subplots_adjust toolbar widget is broken
> too... So this is a fairly high priority bug.
I dug this one up and it is a one line fix.
Line 670 backend_bases.py: LocationEvent.__init__
< if self.x is not None and self.y is not None and a.in_axes(self.x,
self.y) and a.get_navigate():
> if self.x is not None and self.y is not None and a.in_axes(self.x, self.y=
):
From what I can gather, the _nagivate property means an axes should
not respond to adjusting events from the nav toolbar. Therefore, it
seems incorrect to assume that an axes that does not want to navigate
cannot have mouse/key interactions. The widgets are an example of
this case.
John, you added this logic with comment, "added support for multiple
overlapping axes with pan/zoom". I think for a user to address this
problem correctly they should use the zorder property. I added a
check a while back that sets the inaxes property of a LocationEvent to
the highest zorder if multiple axes are found.
Fixed in svn.
- Charlie

John,
John Hunter wrote:
>>>>>>"Eric" == Eric Firing <ef...@ha...> writes:
> 
> 
> Eric> command. If there is any sentiment that it would be
> Eric> desirable, I could make legend understand it as well, for
> Eric> the sake of consistency.
> 
> I'm +1 on this. It only inconveniences power users. 
OK, I will put this on my mental list, but not at the top.
> 
> Eric> I factored the plot box resizing and repositioning code out
> Eric> into a PBox class ("Position Box", subclassed from list),
> Eric> presently in its own pbox.py file, because I think it will
> Eric> be more generally useful; the colorbar command is one such
> Eric> intended use. It may be helpful to make Axes._position a
> Eric> PBox instance instead of an ordinary list.
> 
> PBox appears to overlap matplotlib._transforms.BBox. Granted, the
> mostly universal consensus is that mpl transforms are SNAFU-d (but
> mostly work!), but is this increasing confusion by duplicating
> functionality? In any case, you might consider simply extending BBox
> (even if in python rather than extension code) or adding PBox to
> transforms, cbook, or mlab, and describing what it does and how it is
> different from BBox. It probably does not merit a separate module.
I agree, and have moved it to transforms and added documentation. I 
originally considered putting it in transforms, but I decided to leave 
it separate initially, since it is not integrated with transforms. I 
also thought a little bit about integrating it, but expediency won 
out--I wanted to make something work first, and figure out the finer 
points, implications, and integrations later.
The topic of how to reduce overlap while increasing readability and/or 
power is open.
> 
> Eric> I think that aspect-ratio handling is now reasonably solid
> Eric> and complete; I have tested it with pan and zoom, with and
As usual, I have found some more things that needed to be fixed. I have 
made those changes, and as you requested in another message I have moved 
the axis command functionality to Axes.axis, so pylab.axis is a thin 
wrapper.
> 
> For the record, this is a major and non-trivial step. I certainly
> threw up my hands at attempting to fix it more than a year ago after
> in-depth discussions with Perry about what was involved in making this
> work in the presence of figure resizing, axes data limits, image
> aspect ratios, etc. Thanks Eric and Mark for months of hard work.
> This, plus prototype 3D, almost made me want to push for 1.0 :-).
> Until, of course, I realized how much work still needs to be done on
> the basics (Axis line handling and easy installation come to mind)
> 
> Eric> without ganged axes. I had to put in a couple of hacks in
> Eric> backend_bases to prevent exceptions. (More careful analysis
> 
> Could you summarize these hacks here so we can look over them?
> 
The basic problem is that if axes.set_xlim or axes.set_ylim get 
identical values, so that the data span of an axis is zero, it can cause 
trouble in several places. Furthermore, it seems that ValueError can be 
raised in the transform module when the data span is very small, not 
just when it is zero. I first tried to solve the problem by adding a 
little bit of code to set_xlim and set_ylim to prevent such singular and 
near-singular values. For some reason, it did not solve all the 
problems; I never figured out why. (In retrospect, perhaps I just did 
not use a large enough threshold.) The problems all seemed to be 
triggered by the button3 drag in zoom/pan, with axis('image'), so I put 
most of the block into a try/except structure, and I added 
singularity-checking to the arguments passed to set_xlim and set_ylim. 
The relevant extract from backend_bases is below, with pointers added:
 elif self._button_pressed==3:
 try: # <-------
 dx=(lastx-event.x)/float(a.bbox.width())
 dy=(lasty-event.y)/float(a.bbox.height())
 dx,dy=format_deltas(event,dx,dy)
 alphax = pow(10.0,dx)
 alphay = pow(10.0,dy)#use logscaling, avoid 
singularities and smother scaling...
 lastx, lasty = trans.inverse_xy_tup( (lastx, lasty) )
 if a.get_xscale()=='log':
 xmin = lastx*(xmin/lastx)**alphax
 xmax = lastx*(xmax/lastx)**alphax
 else:
 xmin = lastx+alphax*(xmin-lastx)
 xmax = lastx+alphax*(xmax-lastx)
 if a.get_yscale()=='log':
 ymin = lasty*(ymin/lasty)**alphay
 ymax = lasty*(ymax/lasty)**alphay
 else:
 ymin = lasty+alphay*(ymin-lasty)
 ymax = lasty+alphay*(ymax-lasty)
 except OverflowError: # <---------
 warnings.warn('Overflow while panning')
 return
 a.set_xlim(self.nonsingular(xmin, xmax)) # <--
 a.set_ylim(self.nonsingular(ymin, ymax)) # <--
 self.dynamic_update()
 def nonsingular(self, x0, x1): # <=================
 '''Desperate hack to prevent crashes when button-3 panning with
 axis('image') in effect.
 '''
 d = x1 - x0
 # much smaller thresholds seem to cause Value Error
 # later in Transformation::freeze in axes.draw()
 if abs(d) < 1e-10:
 warnings.warn('Axis data limit is too small for panning')
 x1 += 1e-10
 x0 -= 1e-10
 return (x0, x1)
What I don't understand at all is why this threshold (e.g., 1e-15 is too 
small) is needed to prevent problems in the freeze method.
I think the try/except is reasonable, and some sort of 
singularity-prevention is also reasonable, preferably inside set_xlim 
and set_ylim; but I am not comfortable with this high an arbitrary 
threshold.
Eric

>>>>> "Eric" == Eric Firing <ef...@ha...> writes:
 Eric> command. If there is any sentiment that it would be
 Eric> desirable, I could make legend understand it as well, for
 Eric> the sake of consistency.
I'm +1 on this. It only inconveniences power users. 
 Eric> I factored the plot box resizing and repositioning code out
 Eric> into a PBox class ("Position Box", subclassed from list),
 Eric> presently in its own pbox.py file, because I think it will
 Eric> be more generally useful; the colorbar command is one such
 Eric> intended use. It may be helpful to make Axes._position a
 Eric> PBox instance instead of an ordinary list.
PBox appears to overlap matplotlib._transforms.BBox. Granted, the
mostly universal consensus is that mpl transforms are SNAFU-d (but
mostly work!), but is this increasing confusion by duplicating
functionality? In any case, you might consider simply extending BBox
(even if in python rather than extension code) or adding PBox to
transforms, cbook, or mlab, and describing what it does and how it is
different from BBox. It probably does not merit a separate module.
 Eric> I think that aspect-ratio handling is now reasonably solid
 Eric> and complete; I have tested it with pan and zoom, with and
For the record, this is a major and non-trivial step. I certainly
threw up my hands at attempting to fix it more than a year ago after
in-depth discussions with Perry about what was involved in making this
work in the presence of figure resizing, axes data limits, image
aspect ratios, etc. Thanks Eric and Mark for months of hard work.
This, plus prototype 3D, almost made me want to push for 1.0 :-).
Until, of course, I realized how much work still needs to be done on
the basics (Axis line handling and easy installation come to mind)
 Eric> without ganged axes. I had to put in a couple of hacks in
 Eric> backend_bases to prevent exceptions. (More careful analysis
Could you summarize these hacks here so we can look over them?
 Eric> Thank you, Mark, for the original work that made the changes
 Eric> possible.
Ditto.
JDH