matplotlib-devel Mailing List for matplotlib (Page 3)

This matplotlib release will be included in enthought's next release
of enthought python, which is widely used by windows users for
scientific computing. I'd like to get an stable-as-possible release
in, since enthought python is released very infrequently. Joe Cooper,
who is handling the release, says we can get fixes in until sometime
on Monday, so I'd be much obliged if you all could stress test this
release in case I need to get a bug-fix in. There have been some
potentially script breaking substantial changes to the numerix module
described below, so these tests are doubly important.
What's new in matplotlib 0.71
numerix refactor
 The organization of the numerix module was refactored to be mindful
 of namespaces. See http://matplotlib.sf.net/API_CHANGES. pylab no
 longer overrides the built-ins min, max, and sum, and provides amin,
 amax and asum as the numerix/mlab versions of these. pylab defines
 __all__ to prevent surprises when doing from pylab import *. To see
 the complete list of symbols provided
 >>> import matplotlib.pylab
 >>> matplotlib.pylab.__all__
contour zigzag bug fixed
 Thanks Nadia for the blood, sweat and tears, and Dominique for the
 report.
contour colormaps
 Contour now uses the current colormap if colors is not provided, and
 works with colorbars. See examples/contour_demo2.py
colorbar enhancements
 Horizontal colorbars supported with keyword arg
 orientation='horizontal' and colorbars can be placed in an arbitrary
 axes with keyword arg cax.
accents in mathtext
 Added accents to mathtext: \hat, reve, \grave, ar, cute, ilde, ec,
 \dot, \ddot. All of them have the same syntax, eg to make an overbar
 you do ar{o} or to make an o umlaut you do \ddot{o}. The shortcuts
 are also provided, eg: "o 'e \`e \~n \.x \^y . See
 examples/accent_demo.py
fixed super/subscript parsing in mathtext
 Widowed superscripts now work, eg r'$^12 m{CO}$'
little bugs and enhancements
 Plugged some memory leaks in wx and image module, fixed x,y args in
 contour, added latex symbol kappa, fixed a yticklabel problem under
 change in clim, fixed colorbar number of color bug, fixed
 set_clip_on bug, reverted pythoninspect in tkagg, fixed event
 handling bugs, fixed matlab-compatible load function, exposed vbox
 attr in FigureManagerGTK.
I did not get a chance to get the aspect=preserve imshow bugs fixed on
this iteration. Something for next time!
 http://matplotlib.sourceforge.net
JDH

Following the extended discussion of namespace clashes on the dev and
users list, and some discussions with Todd offlist, I took a new
approach to fix these problems. The central sticking points were
 1) pylab should not override built-ins
 2) numerix should be faithful to the underlying packages while providing
 a consistent interface to them. Hence numerix would not change
 the names of the functions in mlab
 3) pylab imports the numerix symbols, but didn't want to have a
 different policy than numerix.
The resolution to this dilemma was to refactor numerix to preserve the
namespaces. numerix now attempts to mirrors the numarray directory
structure for namespaces (though this is currently incomplete as I
just added enough to satisfy the needs of matplotlib). So you now do
 from matplotlib.numerix.mlab import min, max, mean
 from matplotlib.numerix import arange, array
 from matplotlib.fft import fft
and you'll get the proper symbols from Numeric or numarray depending
on your numerix setting. For convenience I added amin, amax to mlab
and asum to numerix, though the original python incompatible symbols
min, max, sum still reside in their correct numerix locations
unchanged. At some point Todd will pull numerix out as a freestanding
module, I think, which will make for less typing since the
matplotlib prefix can be removed.
The decision for numerix package structure to mirror numarray's seems
like a good one to me, since everyone is still hoping to see Numeric
disappear, and this provides the most natural path. When the great
merging occurs, one can simply globally replace numerix with numarray
and everything will follow.
pylab still imports (almost) all the symbols from numerix,
numerix.mlab, etc, except the ones that clash with python names. In
their stead, pylab imports amin, amax and asym. pylab also tries to
be a good citizen and defines __all__, to avoid problems like the time
module problem described by danny on the user's list.
This is explained in a little more detail in API_CHANGES.
I've tested this a good bit and everything seems to be working. I am
trying to get a release out the door for Joe Cooper to fold into his
enthought build which is going out Monday. I would like to put a
release up this afternoon on the user's list.
Would you kind folks here mind taking this for a test drive and submit
comments, feedback, problems, back to me as soon as possible?
 http://jdh.uchicago.edu/share/matplotlib-0.71rc1.tar.gz
It would also be helpful if you would inspect the __all__ var of pylab
to see if you think its correct. Currently:
 date2num, num2date, drange, epoch2num, num2epoch, mx2num,
 DateFormatter, IndexDateFormatter, DateLocator, RRuleLocator,
 YearLocator, MonthLocator, WeekdayLocator, DayLocator, HourLocator,
 MinuteLocator, SecondLocator, rrule, MO, TU, WE, TH, FR, SA, SU,
 YEARLY, MONTHLY, WEEKLY, DAILY, HOURLY, MINUTELY, SECONDLY,
 relativedelta, array, zeros, shape, rank, size, fromstring, take,
 put, putmask, reshape, repeat, choose, searchsorted, asum, cumsum,
 product, cumproduct, alltrue, sometrue, allclose, arrayrange,
 arange, asarray, convolve, swapaxes, concatenate, transpose, sort,
 argsort, argmax, argmin, innerproduct, dot, outerproduct, resize,
 indices, fromfunction, diagonal, trace, ravel, nonzero, shape,
 where, compress, clip, zeros, ones, identity, add, logical_or, exp,
 subtract, logical_xor, log, multiply, logical_not, log10, divide,
 maximum, sin, minimum, sinh, conjugate, bitwise_and, sqrt, power,
 bitwise_or, tan, absolute, bitwise_xor, tanh, negative, ceil,
 greater, fabs, greater_equal, floor, less, arccos, arctan2,
 less_equal, arcsin, fmod, equal, arctan, hypot, not_equal, cos,
 around, logical_and, cosh, arccosh, arcsinh, arctanh,
 cross_correlate, pi, ArrayType, matrixmultiply, rand, randn, eye,
 tri, diag, fliplr, flipud, rot90, tril, triu, ptp, mean, msort,
 median, std, cumsum, prod, cumprod, trapz, diff, cov, corrcoef,
 squeeze, kaiser, blackman, bartlett, hanning, hamming, sinc, eig,
 svd, angle, roots, amin, amax, linspace, window_hanning,
 window_none, conv, detrend, detrend_mean, detrend_none,
 detrend_linear, corrcoef, polyfit, polyval, vander, entropy,
 normpdf, levypdf, find, trapz, prepca, fix, rem, norm, orth, rank,
 sqrtm, prctile, center_matrix, meshgrid, rk4, exp_safe, amap,
 sum_flat, mean_flat, rms_flat, l1norm, l2norm, norm, frange,
 diagonal_matrix, base_repr, binary_repr, log2, ispower2,
 bivariate_normal, inverse, eigenvectors, fft, Int8, UInt8, Int16,
 UInt16, Int32, UInt32, Float32, Float64, Complex32, Complex64,
 Float, Int, Complex, TickHelper, Formatter, FixedFormatter,
 NullFormatter, FuncFormatter, FormatStrFormatter, ScalarFormatter,
 LogFormatter, LogFormatterExponent, LogFormatterMathtext, Locator,
 IndexLocator, FixedLocator, NullLocator, LinearLocator, LogLocator,
 AutoLocator, MultipleLocator, plotting, colormaps,
 get_current_fig_manager, connect, disconnect, get_plot_commands,
 raise_msg_to_str, axis, axes, delaxes, clim, close, clf, colorbar,
 draw, figtext, figimage, figlegend, figure, gca, gcf, gci, get,
 hold, ishold, isinteractive, imread, load, rc, rcdefaults, save,
 savefig, set, subplot, title, xlabel, ylabel, xlim, ylim, xticks,
 rgrids, thetagrids, yticks, polar, over, ioff, ion, axhline,
 axhspan, axvline, axvspan, bar, barh, cohere, contour, csd,
 errorbar, fill, hist, hlines, imshow, loglog, pcolor,
 pcolor_classic, pie, plot, plot_date, psd, scatter, scatter_classic,
 semilogx, semilogy, specgram, spy, spy2, stem, vlines, cla, grid,
 legend, table, text, autumn, bone, cool, copper, flag, gray, hot,
 hsv, jet, pink, prism, spring, summer, winter, Line2D, Polygon,
 Rectangle, Circle, Text, cm, get_cmap, normalize, show
Thanks!
JDH 

Thanks John,
Sheesh, 95 degrees, I can't imagine how I'd take that. Some Christmases my 
family would go visit relatives in Chicago. It seemed just a little bit more 
warmer (but not that much) when we returned to Oklahoma. Never really gone 
anywhere to warm for Christmas.
More below...
On Tuesday 18 January 2005 20:25, John Hunter wrote:
> >>>>> "Joe" == Joe <gar...@us...> writes:
>
> Joe> Hello devels, I chose to use matplotlib for a restaurant
> Joe> simulation. It supplied all the features I needed and I was
> Joe> pleasantly surprised. However the people I was doing this
> Joe> project for mentioned that the simulation slowed down to a
> Joe> crawl 10 or so minutes into the simulation. It sounded like
> Joe> a memory leak to me so I investigated and found a 100 KB/sec
> Joe> leak in Windows (via the interpreter as well as frozen with
> Joe> py2exe) as well as in Linux. I narrowed it down to a part of
> Joe> the code where I updated xlim and ylim via Axes::set_xlim,
> Joe> Axes::set_ylim. Commenting that part out yielded no leak,
>
> So you're doing animation right? Are you using images, eg imshow? I
> just fixed a memory leak in image resizing that amounts to about
> 200K/resize -- the changes are in CVS. Of course on windows it's hard
> to test because the changes are in extension code, and require
> building on win32. The instructions for building on win32 are in the
> file setupext.py.
I wish so, but no, I'm not using any images.
> If you are not using images, another place to look is in text caching.
> text layout is expensive (to support arbitrary rotations, etc). So I
> cache the layout information in a dictionary "cached" in the text
> module. When you change the xlim/ylim, that would trigger a new
> layout and an addition to the cache. We need to add some
> auto-clearing mechanisms to prevent the cache from growing w/o bound
> but its not done yet. I find it hard to believe, though, that this
> would leak 100kb/sec. Assuming a 20Hz frame rate, the leak is about
> 5kb/sec, and I don't think I'm caching that much info.
I am using 5 GTKAgg FIgure widgets with 2-3 subplots a piece (a maximum of 3 
subplots get drawn at a time). The simulation runs at 1 Hz. Therefore 6.67 
kB/sec at max. Which sounds about right. I am updating each Axes point set 
when I do the update. 
> You can estimate the memory effect of the cached simply by commenting
> out the line
>
> self.cached[key] = ret
>
> in the matplotlib.text module.
I'll definitely check that out. With the above math that just might prove to 
be the culprit.
> What backend are you using? Are you refreshing the same figure data,
> or creating new figures? More information would certainly help.....
>
> Joe> however I lost some critical functionality in my charts
> Joe> (which behave like the Windows Task Manager resource monitor
> Joe> graphs). So I really need this fixed. I will send shortly a
> Joe> test case which replicates the problem.
>
> That would be great.
>
> Joe> Since the maintainer is out on vacation, I was also wondering
> Joe> if anyone knew enough about the internals of matplotlib that
> Joe> they could give me an idea of which modules (Python or
> Joe> otherwise) are affected by Axes::set_xlim and Axes::set_ylim.
>
> I'm back. I was in Rio for a couple of weeks and have suffered a 95
> degree temperature drop returning to Chicago (sigh)...
>
> JDH
I'll get the test case up on the web ASAP, I just wanted to reply now to let 
everyone know that I'm still working on the problem.
Joe

Hi there,
I just submitted a patch changing the errorhandling of Line2D on non-existant 
linestyle/marker strings. Now I realize that it obviously clashes with the 
current policy for reporting errors.
What happened before was that
 plot(...,linestyle='garbage',...)
would verbose.report_error and set linestyle to a default, while
 set_linestyle(...,linestyle='garbage')
would be silent and still set linestyle to a default.
Clearly, this is inconsistent, so there will be little argument that the 
behavior should be unified. Question is, to what?
What I would expect is, that set_linestyle raises an exception if it is called 
with the wrong arguments. This is, what I expect of any python library, no 
matter whether it offers a GUI or not. When I work in ipython at the 
commandline, I expect the errors to be displayed right there, and not be 
written to a file or displayed in some window.
Obviously, this clashes with the very existance of the 'Verbose' class, used 
to report errors which just writes errors to some file and goes on with work 
afterwards.
The verbose.report_error approach certainly is useful within the code that is 
run in the GUI-thread. Those routines that are called from outside, though, 
should rather raise exception.
Furthermore, the outside functions should do as much checking on the input as 
reasonably possible, so that errors are simpler to debug. Once some erraneous 
input has reached the internals of matplotlib, it really is hard for the user 
to guess what actually caused the error.
Ciao,
Norbert
-- 
_________________________________________Norbert Nemec
 Bernhardstr. 2 ... D-93053 Regensburg
 Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
 eMail: <No...@Ne...>

>>>>> "Joe" == Joe <gar...@us...> writes:
 Joe> Hello devels, I chose to use matplotlib for a restaurant
 Joe> simulation. It supplied all the features I needed and I was
 Joe> pleasantly surprised. However the people I was doing this
 Joe> project for mentioned that the simulation slowed down to a
 Joe> crawl 10 or so minutes into the simulation. It sounded like
 Joe> a memory leak to me so I investigated and found a 100 KB/sec
 Joe> leak in Windows (via the interpreter as well as frozen with
 Joe> py2exe) as well as in Linux. I narrowed it down to a part of
 Joe> the code where I updated xlim and ylim via Axes::set_xlim,
 Joe> Axes::set_ylim. Commenting that part out yielded no leak,
So you're doing animation right? Are you using images, eg imshow? I
just fixed a memory leak in image resizing that amounts to about
200K/resize -- the changes are in CVS. Of course on windows it's hard
to test because the changes are in extension code, and require
building on win32. The instructions for building on win32 are in the
file setupext.py.
If you are not using images, another place to look is in text caching.
text layout is expensive (to support arbitrary rotations, etc). So I
cache the layout information in a dictionary "cached" in the text
module. When you change the xlim/ylim, that would trigger a new
layout and an addition to the cache. We need to add some
auto-clearing mechanisms to prevent the cache from growing w/o bound
but its not done yet. I find it hard to believe, though, that this
would leak 100kb/sec. Assuming a 20Hz frame rate, the leak is about
5kb/sec, and I don't think I'm caching that much info.
You can estimate the memory effect of the cached simply by commenting
out the line
 self.cached[key] = ret
in the matplotlib.text module.
What backend are you using? Are you refreshing the same figure data,
or creating new figures? More information would certainly help.....
 Joe> however I lost some critical functionality in my charts
 Joe> (which behave like the Windows Task Manager resource monitor
 Joe> graphs). So I really need this fixed. I will send shortly a
 Joe> test case which replicates the problem.
That would be great.
 Joe> Since the maintainer is out on vacation, I was also wondering
 Joe> if anyone knew enough about the internals of matplotlib that
 Joe> they could give me an idea of which modules (Python or
 Joe> otherwise) are affected by Axes::set_xlim and Axes::set_ylim.
I'm back. I was in Rio for a couple of weeks and have suffered a 95
degree temperature drop returning to Chicago (sigh)...
JDH

Hello devels,
I chose to use matplotlib for a restaurant simulation. It supplied all the 
features I needed and I was pleasantly surprised. However the people I was 
doing this project for mentioned that the simulation slowed down to a crawl 
10 or so minutes into the simulation. It sounded like a memory leak to me so 
I investigated and found a 100 KB/sec leak in Windows (via the interpreter as 
well as frozen with py2exe) as well as in Linux. I narrowed it down to a 
part of the code where I updated xlim and ylim via Axes::set_xlim, 
Axes::set_ylim. Commenting that part out yielded no leak, however I lost 
some critical functionality in my charts (which behave like the Windows Task 
Manager resource monitor graphs). So I really need this fixed. I will send 
shortly a test case which replicates the problem. 
Since the maintainer is out on vacation, I was also wondering if anyone knew 
enough about the internals of matplotlib that they could give me an idea of 
which modules (Python or otherwise) are affected by Axes::set_xlim and 
Axes::set_ylim.
Joe

Thanks!
 --- Todd Miller <jm...@st...> wrote: 
> The setup scripts copy _transforms.cpp to
> _na_transforms.cpp and
> _nc_transforms.cpp at build time. The good news is
> there's only one
> source file for both: src/_transforms.cpp.
> 
> Regards,
> Todd
> 
> On Wed, 2005年01月12日 at 17:30, Brendan Simons wrote:
> > Hi. I'm attempting to extend the polar_axes class
> to
> > better handle scrolling and zooming, and to allow
> > setting a lower limit on the r-axis. There
> isn't
> > much in the way of documentation 'round these
> parts,
> > so I'm kind of stretching. I have a hunch though
> I
> > may have to add a transform to the extension
> module
> > listed in the subject line. Can someone point me
> to
> > the source?
> > 
> > Brendan
> > 
> >
>
______________________________________________________________________
> 
> > Post your free ad now! http://personals.yahoo.ca
> > 
> > 
> >
>
-------------------------------------------------------
> > The SF.Net email is sponsored by: Beat the
> post-holiday blues
> > Get a FREE limited edition SourceForge.net t-shirt
> from ThinkGeek.
> > It's fun and FREE -- well,
> almost....http://www.thinkgeek.com/sfshirt
> > _______________________________________________
> > Matplotlib-devel mailing list
> > Mat...@li...
> >
>
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> -- 
> 
> 
______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca

The setup scripts copy _transforms.cpp to _na_transforms.cpp and
_nc_transforms.cpp at build time. The good news is there's only one
source file for both: src/_transforms.cpp.
Regards,
Todd
On Wed, 2005年01月12日 at 17:30, Brendan Simons wrote:
> Hi. I'm attempting to extend the polar_axes class to
> better handle scrolling and zooming, and to allow
> setting a lower limit on the r-axis. There isn't
> much in the way of documentation 'round these parts,
> so I'm kind of stretching. I have a hunch though I
> may have to add a transform to the extension module
> listed in the subject line. Can someone point me to
> the source?
> 
> Brendan
> 
> ______________________________________________________________________ 
> Post your free ad now! http://personals.yahoo.ca
> 
> 
> -------------------------------------------------------
> The SF.Net email is sponsored by: Beat the post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
> It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
-- 

Hi. I'm attempting to extend the polar_axes class to
better handle scrolling and zooming, and to allow
setting a lower limit on the r-axis. There isn't
much in the way of documentation 'round these parts,
so I'm kind of stretching. I have a hunch though I
may have to add a transform to the extension module
listed in the subject line. Can someone point me to
the source?
Brendan
______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca

OK, I understand that the overhead added by my routine is a bit much. Maybe, 
we could just go half-way? Overriding min and max for arrays, but leaving the 
axis-argument for more specialized amin and amax routines?
Am Montag, 10. Januar 2005 22:38 schrieb Andrew Straw:
> Also, we must not forget about round, sum, and abs (and any others I have
> missed). For example, abs() caught me because I use the cgkit quaternion
> type, which overrides the __abs__ method and thus fails to work properly
> with the mlab.py implementation of abs().
Look at these one by one:
* abs() already calls an __abs__() method. The clean way to extend it, would 
therefore be to give arrays such a method. This should solve the problem 
completely.
* round() does not seem to be extendable in Python. Maybe we should propose to 
change Python itself to introduce a __round__ method? That would only be 
straightforward.
* min, max and sum are all based on iterating over a sequence. Maybe, one 
should again have __min__, __max__ and __sum__ which should then be checked 
by the builtin before falling back to iterating over the sequence? I could 
imagine many kinds of containers that could optimize these operations. So 
this would again be a detail that should be changed in Python itself. If the 
builtin min() function would then also pass on keyword arguments, that would 
solve our problem completely and thoroughly.
Does anybody have experience with discussions in the Python forum to estimate 
how realistic such a PEP would be?
Ciao,
Norbert
-- 
_________________________________________Norbert Nemec
 Bernhardstr. 2 ... D-93053 Regensburg
 Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
 eMail: <No...@Ne...>

John Hunter wrote:
>>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:
>>>>>> 
>>>>>
>
> Perry> My 2 cents is that I think Fernando is right on this
> Perry> issue. I'd rather go with a solution that causes temporary
> Perry> pain for matlab users rather than one that causes
> Perry> lingering, long-term irritations.
>
> OK, looks like a consensus to me :-)
> 
>
I guess my opinion on this is already clear :).
> I'm happy with Fernando's proposed names amin, amax, around, etc. If
> everyone else is too, I propose Andrew implement his patch, provide
> the compatibility names, and update the relevant docs to advertise
> this prominently: API_CHANGES, CHANGELOG, tutorial and users guide.
> Particularly in the latter two, I think we should warn people about
> the potential performance hit of using the builtin min, max and
> friends on large arrays.
> 
>
I'm happy to implement whatever the consensus is, but I'm quite busy 
this week and away this weekend, so it'll be next week until I can do 
anything. If someone else wants to jump in and do it, I certainly won't 
mind.
Norbert Nemec wrote:
> There might be a solution that avoids the performance hit: there 
> should not be any problem with pylab offering an optimized set of min, 
> max, etc. as long as their signature is identical to the builtins and 
> the behavior only extends them. Something along the line of:
>
> def min(*args, **kwargs):
> if args == ():
> raise TypeError, "min() takes at least 1 argument (0 given)"
> if len(args) == 1 and type(args[0]) is ArrayType:
> axis=kwargs.pop('axis',0)
> res = minimum.reduce(args[0],axis)
> else:
> res = __builtin__.min(*args)
> if len(kwargs)>0:
> raise TypeError, (
> "min() got an unexpected keyword argument '%s'"
> %kwargs.keys()[0]
> )
> return res
What do people think about Norbert's "best of both worlds" approach? 
Although it seems great in theory, I'm disinclined to use it simply 
because it does override the builtin. Although he's doubtlessly 
constructed this with the greatest of care to perform exactly as the 
builtin, I wonder about obscure corner cases which won't behave exactly 
the same and may result in even more obscure bugs. Maybe my misgivings 
are undue paranoia on my part, and his 3rd way really is best. I 
suppose I'd want to throw lots of tests at it before I pronounce my 
final judgement on it, which I don't have time to do at the moment. 
(Next week, if need be...)
Just a thought for consideration: perhaps Norbert's code could actually 
be used by the underlying mlab.py modules? I guess some code in the 
wild uses the axis argument not as a keyword, so there would be a 
backwards incompatible change in that regard... Other than that, 
though, this kind of behavior from the mlab.py modules would probably 
have resulted in a less serious conundrum than what we now face.
Also, we must not forget about round, sum, and abs (and any others I 
have missed). For example, abs() caught me because I use the cgkit 
quaternion type, which overrides the __abs__ method and thus fails to 
work properly with the mlab.py implementation of abs().
Cheers!
Andrew

John Hunter wrote:
>>>>>>"Perry" == Perry Greenfield <pe...@st...> writes:
>>>>>> 
>>>>>>
>
> Perry> My 2 cents is that I think Fernando is right on this
> Perry> issue. I'd rather go with a solution that causes temporary
> Perry> pain for matlab users rather than one that causes
> Perry> lingering, long-term irritations.
>
>OK, looks like a consensus to me :-)
> 
>
I guess my opinion on this is already clear :).
>I'm happy with Fernando's proposed names amin, amax, around, etc. If
>everyone else is too, I propose Andrew implement his patch, provide
>the compatibility names, and update the relevant docs to advertise
>this prominently: API_CHANGES, CHANGELOG, tutorial and users guide.
>Particularly in the latter two, I think we should warn people about
>the potential performance hit of using the builtin min, max and
>friends on large arrays.
> 
>
I'm happy to implement whatever the consensus is, but I'm quite busy 
this week and away this weekend, so it'll be next week until I can do 
anything. If someone else wants to jump in and do it, I certainly won't 
mind.
Norbert Nemec wrote:
> There might be a solution that avoids the performance hit: there should not be 
> any problem with pylab offering an optimized set of min, max, etc. as long as 
> their signature is identical to the builtins and the behavior only extends 
> them. Something along the line of:
> 
> def min(*args, **kwargs):
> if args == ():
> raise TypeError, "min() takes at least 1 argument (0 given)"
> if len(args) == 1 and type(args[0]) is ArrayType:
> axis=kwargs.pop('axis',0)
> res = minimum.reduce(args[0],axis)
> else:
> res = __builtin__.min(*args)
> if len(kwargs)>0:
> raise TypeError, (
> "min() got an unexpected keyword argument '%s'"
> %kwargs.keys()[0]
> )
> return res
What do people think about Norbert's "best of both worlds" approach? Although it seems great in theory, I'm disinclined to use it simply because it does override the builtin. Although he's doubtlessly constructed this with the greatest of care to perform exactly as the builtin, I wonder about obscure corner cases which won't behave exactly the same and may result in even more obscure bugs. Maybe my misgivings are undue paranoia on my part, and his 3rd way really is best. I suppose I'd want to throw lots of tests at it before I pronounce my final judgement on it, which I don't have time to do at the moment. (Next week, if need be...)
Just a thought for consideration: perhaps Norbert's code could actually be used by the underlying mlab.py modules? I guess some code in the wild uses the axis argument not as a keyword, so there would be a backwards incompatible change in that regard... Other than that, though, this kind of behavior from the mlab.py modules would probably have resulted in a less serious conundrum than what we now face.
Also, we must not forget about round, sum, and abs (and any others I have missed). For example, abs() caught me because I use the cgkit quaternion type, which overrides the __abs__ method and thus fails to work properly with the mlab.py implementation of abs().
Cheers!
Andrew

Norbert Nemec wrote:
> Am Montag, 10. Januar 2005 11:46 schrieb John Hunter:
> 
>>I'm happy with Fernando's proposed names amin, amax, around, etc. If
>>everyone else is too, I propose Andrew implement his patch, provide
>>the compatibility names, and update the relevant docs to advertise
>>this prominently: API_CHANGES, CHANGELOG, tutorial and users guide.
>>Particularly in the latter two, I think we should warn people about
>>the potential performance hit of using the builtin min, max and
>>friends on large arrays.
> 
> 
> There might be a solution that avoids the performance hit: there should not be 
> any problem with pylab offering an optimized set of min, max, etc. as long as 
> their signature is identical to the builtins and the behavior only extends 
> them. Something along the line of:
Hmm. Those extra checks in your code don't come for free... I'd rather leave 
the builtins alone (many of them are C-coded, hence quite fast), and just 
provide array versions where needed.
Just my 1e-2
Best,
f

On Mon, 2005年01月10日 at 06:23, John Hunter wrote:
> >>>>> "John" == John Hunter <jdh...@ac...> writes:
> 
> John> I'm happy with Fernando's proposed names amin, amax, around,
> John> etc. If everyone else is too, I propose Andrew implement
> John> his patch, provide the compatibility names, and update the
> John> relevant docs to advertise this prominently: API_CHANGES,
> John> CHANGELOG, tutorial and users guide. Particularly in the
> John> latter two, I think we should warn people about the
> John> potential performance hit of using the builtin min, max and
> John> friends on large arrays.
> 
> Hmm, another thought.
> 
> If we are going to make this change for the pylab namespace, then it
> seems we might as well make the change in the numerix namespace as
> well, since all the same arguments apply. I think doing different
> things in numerix and pylab *is* a recipe for confusion. What do you
> think Todd, does this seem sensible?
> 
> Basically, the theme would be we use the underlying names from
> Numeric/numarray except when they clash with python builtins, in which
> case we'd use something like the a* names Fernando proposed.
> 
> Of course, changing the numerix names would mean the matplotlib code
> and examples will have to be updated as well. The former should be
> relatively easy since I believe there are no uses of unqualified min
> and max from numerix in the base code. Eg, we do in axes.py
> 
> from numerix import max as nxmax
> from numerix import min as nxmin
> 
> and in mlab.py we do MLab.max. But we'll have to be careful to make
> sure all the names Andrew identified are handled thoughout the coude
> and examples...
> 
> JDH
This all sounds fine to me. My understanding is that anywhere we "get
this wrong" the impact will be degraded performance. As an aside, I
avoid "from *" like the plague myself but understand that it's important
to optimize interactive use.
Unleash Andrew and I'll try to propagate the changes forward to Scipy.
Todd

On Jan 10, 2005, at 6:23 AM, John Hunter wrote:
>>>>>> "John" == John Hunter <jdh...@ac...> writes:
>
> John> I'm happy with Fernando's proposed names amin, amax, around,
> John> etc. If everyone else is too, I propose Andrew implement
> John> his patch, provide the compatibility names, and update the
> John> relevant docs to advertise this prominently: API_CHANGES,
> John> CHANGELOG, tutorial and users guide. Particularly in the
> John> latter two, I think we should warn people about the
> John> potential performance hit of using the builtin min, max and
> John> friends on large arrays.
>
> Hmm, another thought.
>
> If we are going to make this change for the pylab namespace, then it
> seems we might as well make the change in the numerix namespace as
> well, since all the same arguments apply. I think doing different
> things in numerix and pylab *is* a recipe for confusion. What do you
> think Todd, does this seem sensible?
>
I can't argue with that (though Todd may have some comments of his 
own). In fact it would seem silly if numerix wasn't consistent.

Am Montag, 10. Januar 2005 11:46 schrieb John Hunter:
> I'm happy with Fernando's proposed names amin, amax, around, etc. If
> everyone else is too, I propose Andrew implement his patch, provide
> the compatibility names, and update the relevant docs to advertise
> this prominently: API_CHANGES, CHANGELOG, tutorial and users guide.
> Particularly in the latter two, I think we should warn people about
> the potential performance hit of using the builtin min, max and
> friends on large arrays.
There might be a solution that avoids the performance hit: there should not be 
any problem with pylab offering an optimized set of min, max, etc. as long as 
their signature is identical to the builtins and the behavior only extends 
them. Something along the line of:
def min(*args, **kwargs):
 if args == ():
 raise TypeError, "min() takes at least 1 argument (0 given)"
 if len(args) == 1 and type(args[0]) is ArrayType:
 axis=kwargs.pop('axis',0)
 res = minimum.reduce(args[0],axis)
 else:
 res = __builtin__.min(*args)
 if len(kwargs)>0:
 raise TypeError, (
 "min() got an unexpected keyword argument '%s'"
 %kwargs.keys()[0]
 )
 return res
Probably, one could even avoid separate amin, amax, etc. functions. The user 
just has to be aware that the axis can only be given as keyword argument.
-- 
_________________________________________Norbert Nemec
 Bernhardstr. 2 ... D-93053 Regensburg
 Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
 eMail: <No...@Ne...>

>>>>> "John" == John Hunter <jdh...@ac...> writes:
 John> I'm happy with Fernando's proposed names amin, amax, around,
 John> etc. If everyone else is too, I propose Andrew implement
 John> his patch, provide the compatibility names, and update the
 John> relevant docs to advertise this prominently: API_CHANGES,
 John> CHANGELOG, tutorial and users guide. Particularly in the
 John> latter two, I think we should warn people about the
 John> potential performance hit of using the builtin min, max and
 John> friends on large arrays.
Hmm, another thought.
If we are going to make this change for the pylab namespace, then it
seems we might as well make the change in the numerix namespace as
well, since all the same arguments apply. I think doing different
things in numerix and pylab *is* a recipe for confusion. What do you
think Todd, does this seem sensible?
Basically, the theme would be we use the underlying names from
Numeric/numarray except when they clash with python builtins, in which
case we'd use something like the a* names Fernando proposed.
Of course, changing the numerix names would mean the matplotlib code
and examples will have to be updated as well. The former should be
relatively easy since I believe there are no uses of unqualified min
and max from numerix in the base code. Eg, we do in axes.py
from numerix import max as nxmax
from numerix import min as nxmin
and in mlab.py we do MLab.max. But we'll have to be careful to make
sure all the names Andrew identified are handled thoughout the coude
and examples...
JDH

>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:
 Perry> My 2 cents is that I think Fernando is right on this
 Perry> issue. I'd rather go with a solution that causes temporary
 Perry> pain for matlab users rather than one that causes
 Perry> lingering, long-term irritations.
OK, looks like a consensus to me :-)
I'm happy with Fernando's proposed names amin, amax, around, etc. If
everyone else is too, I propose Andrew implement his patch, provide
the compatibility names, and update the relevant docs to advertise
this prominently: API_CHANGES, CHANGELOG, tutorial and users guide.
Particularly in the latter two, I think we should warn people about
the potential performance hit of using the builtin min, max and
friends on large arrays.
I'll put this on the new "News Flash" section of the web site with the
next release, which at least should get people's attention.
A-foolish-consistency-is-the-hobgobblin-of-a-small-mindly-yours,
JDH

Fernando Perez wrote:
> While pylab's mission is indeed is matlab compatibility, you 
> already point out 
> that this is not an 'at all costs' objective. This is one case 
> where I really 
> think that breaking compatibility with the base python language 
> is a too high 
> price to pay. I'm having a hard time justifying my position in a clear 
> manner, but I have a strong 'gut feeling' about it. I'll try to 
> provide some 
> rational points, though:
> 
[...]
My 2 cents is that I think Fernando is right on this issue. I'd rather
go with a solution that causes temporary pain for matlab users rather
than one that causes lingering, long-term irritations.
Perry

John Hunter wrote:
> I'm weakly inclined to leave the situation as it is: it's compatible
> with matlab which is essentially the pylab mission, and it's worked
> for 10 or so years for Numeric's MLab. Cautious users and power users
> have a clear alternative. If we leave it as in, we can easily provide
> pymin, pymax, pyround, etc, for users who want the python version. I
> am open to the proposal, but I think we should frame the argument as
> one of performance versus convenience versus
> least-likely-to-bite-your-ass versus matlab-compatibility rather than
> fixing a bug.
While pylab's mission is indeed is matlab compatibility, you already point out 
that this is not an 'at all costs' objective. This is one case where I really 
think that breaking compatibility with the base python language is a too high 
price to pay. I'm having a hard time justifying my position in a clear 
manner, but I have a strong 'gut feeling' about it. I'll try to provide some 
rational points, though:
One of pylab's, objectives is to help matlab users move over to python. While 
initially they will naturally only use the compatible functions, we hope they 
will grow out into using all the things python offers which matlab lacks (nice 
OO, listcomps, generator expressions, the great generic standard library, 
etc.). This means that ultimately, we hope they will really use the python 
language to its fullest. At that point, if they begin using pyton code from 
'the wild', they are very likely to be bitten by this kind of incompatibility 
(as we all have).
The result: a decision made to ease the initial stages of a transition, ends 
up causing an everlasting headache. And it's not like we can guarantee 100% 
source compatibility, since they are after all different languages. I think 
it's much better to add min, max & friends to the few things matlab users need 
to learn in the transition, rather than have everyone pay for this from now on.
You also need to keep in mind that pylab is likely to be used by _python 
users_ who have no matlab experience (I am such a person). For this group, 
the change of a builtin in this manner is very unexpected, and the source of 
all sorts of problems. As anecdotal evidence, it was precisely this 
particular problem with MLab which convinced me, a few years ago, to _never_ 
use 'from foo import *'. Even though I was not a matlab user, I thought the 
MLab names were nice and short, and for a while imported it wholesale. Until 
I wasted a lot of time tracking the min/max bug one day. When I found it, I 
felt like screaming at the MLab writers, and decided never again to trust a 
third party library with a * import. To this day, I use Numeric and Scipy 
always with qualified imports.
IMHO, MLab simply got this one wrong 10 years ago, and pylab should not repeat 
their mistake. In my own code, I have often written simple a* routines: amap, 
amin, amax, around, short for arraymap, arraymin, etc. I think it's short and 
clear, and provides a nice distinction of their functionality versus the 
builtins (it's quicker to type amin than nxmin, esp. on a qwerty keyboard 
where nx is an off-home-row chord).
Anyway, this is as much as I'll say on the topic. It's ultimately your 
choice. But if I had my way, pylab would just provide a set of a*foo routines 
as array-based counterparts to the builtins, and it would document such a 
feature very prominently.
Cheers,
f

>>>>> "Norbert" == Norbert Nemec <Nor...@gm...> writes:
 Norbert> I fear that this "fix" will rather add to the general
 Norbert> confusion than lighten it. First doing "from ... import
 Norbert> *" and then reverting it partly really seems like a bad
 Norbert> idea. The cleaner solution would be to fix
 Norbert> Numeric/numarray in such a way that the replacement
 Norbert> min/max/round/... retain the complete original
 Norbert> functionality of the builtins only extending them to new
 Norbert> types. Independent of pylab, there obviously is a bug in
 Norbert> the underlying libraries which should not be obscured but
 Norbert> solved.
This is not a bug in either matplotlib, Numeric or numarray. The min
and max in question were originally defined in Numeric's MLab module.
If you read that module's docstring
 Matlab(tm) compatibility functions.
 This will hopefully become a complete set of the basic functions available in
 matlab. The syntax is kept as close to the matlab syntax as possible. 
So the functions in question were explicitly designed to be compatible
with matlab, not python. The pylab interface of matplotlib was
designed to "finish the job" that MLab started, and extend the matlab
compatibility into the plotting arena, which is does quite faithfully.
Hence the pylab module imports the MLab symbols /
numarray.linear_algebra.mlab.
Because python handles namespaces well, there is no problem with MLab
defining a min with a different call syntax than the built-in. The
informed user can choose between them by avoiding the 'from MLab
import *' or 'from pylab import *' and using namespaces instead, eg
MLab.min or pylab.min.
In an effort to provide a matlab compatible environment, I have
encouraged the use of 'from pylab import *', which like matlab gives
you everything at your fingertips. Mainly this is for convenience to
the newbie and interactive user, who doesn't want to have to sort out
where all the symbols are. If you take a quick look at na_imports,
which imports the numarray symbols for matploltib.numerix, you'll see
this is nontrivial, with functions coming from a handful of different
places.
So I feel the current situation is reasonably coherent: pylab provides
matlab compatible functions. That said, I agree this is a problem,
and part of the migration from matplotlib.matlab to matplotlib.pylab
is made with the sense that this is ultimately a *python* library that
strives for matlab compatibility but not at all costs. I myself have
been bitten by the min/max confusion on more than one occasion, as
have other mpl developers.
So I see the merits of Andrew's proposal. If adopted, we should
provide nxmin, nxmax, nxround, etc in the pylab namespace and
advertise the changes widely in API_CHANGES, the CHANGELOG, the
tutorial and user's guide. We should be mindful that changing the
current behavior probably will break some mpl scripts and can result
in a *substantial* performance hit for users who use min and max in
place of nxmin and nxmax for numerix arrays because those functions
rely on the python sequence protocol. This was the source of a
considerable performance hit in the colormapping that was recently
fixed.
So newbies and naive users are going to get screwed one way or the
other. If we retain MLab.min, people who expect the python min will
sometimes get a signature error. If we instead provide nxmin, they'll
inadvertently take a performance hit if they use python min naively.
I'm weakly inclined to leave the situation as it is: it's compatible
with matlab which is essentially the pylab mission, and it's worked
for 10 or so years for Numeric's MLab. Cautious users and power users
have a clear alternative. If we leave it as in, we can easily provide
pymin, pymax, pyround, etc, for users who want the python version. I
am open to the proposal, but I think we should frame the argument as
one of performance versus convenience versus
least-likely-to-bite-your-ass versus matlab-compatibility rather than
fixing a bug.
JDH

I fear that this "fix" will rather add to the general confusion than lighten 
it. First doing "from ... import *" and then reverting it partly really seems 
like a bad idea. The cleaner solution would be to fix Numeric/numarray in 
such a way that the replacement min/max/round/... retain the complete 
original functionality of the builtins only extending them to new types. 
Independent of pylab, there obviously is a bug in the underlying libraries 
which should not be obscured but solved.
I already hate the current situation as it is: Both, SciPy and matplotlib, are 
based upon Numeric/numarray, but then they change and extend its behaviour in 
subtle ways so that - in the end - the user is completely confused. If there 
is a bug or a missing feature in the array library, it should be fixed there. 
Having arrays behave similar but slightly different in SciPy, matplotlib and 
numarray/Numeric should be avoided by all means.
-- 
_________________________________________Norbert Nemec
 Bernhardstr. 2 ... D-93053 Regensburg
 Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
 eMail: <No...@Ne...>
-- 
_________________________________________Norbert Nemec
 Bernhardstr. 2 ... D-93053 Regensburg
 Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
 eMail: <No...@Ne...>

Andrew Straw wrote:
> So, I offer the following patch [2] to pylab.py which fixes this. I 
> hereby ask for your feedback indicating if you are happy with the 
> current behavior, or if you prefer that pylab does not override 
> builtins. (Also, if you have a better way, that would be appreciated, too.)
+1 on your patch.
I'd also much rather have the builtin namespace be left alone.
Cheers,
f

A few weeks ago (on Dec 18), I send an email to the matplotlib-devel 
list indicating that "from pylab import *" overrides some builtin 
functions, such as min() and max(). [1]
This results from pylab enlarging its namespace greatly with statements 
like "from numerix import *" and "from mlab import *". In general this 
is a good thing, but it seems numarray.linear_algebra.mlab (amongst 
possibly others) overrides some builtin names. Although I don't see the 
benefit in this design for numarray.linear_algebra.mlab, I can live with 
it, since I never do "from numarray.linear_algebra.mlab import *". 
However, I (and many others here, I suspect) would like to frequently 
use "from pylab import *", and it really pisses me off when I discover 
that various builtins are overridden, causing mysterious errors that may 
be hard to track down.
So, I offer the following patch [2] to pylab.py which fixes this. I 
hereby ask for your feedback indicating if you are happy with the 
current behavior, or if you prefer that pylab does not override 
builtins. (Also, if you have a better way, that would be appreciated, too.)
You may check your own systems using a little script I wrote while 
testing this. [3]
Cheers!
Andrew
[1] 
http://sourceforge.net/mailarchive/forum.php?thread_id=6190717&forum_id=36187
[2] Patch to pylab.py, to be inserted anywhere after the last "from blah 
import *", e.g. line 216.
# restore builtin functions which may have been overridden
min = getattr(sys.modules['__builtin__'],'min')
max = getattr(sys.modules['__builtin__'],'max')
sum = getattr(sys.modules['__builtin__'],'sum')
round = getattr(sys.modules['__builtin__'],'round')
abs = getattr(sys.modules['__builtin__'],'abs')
[3] Script to test for overriding of builtin names:
import sys
def check_globals():
 for key in globals().keys():
 if key in dir(sys.modules['__builtin__']):
 if globals()[key] != getattr(sys.modules['__builtin__'],key):
 print "'%s' was overridden in globals()."%key
print 'before pylab import'
check_globals()
print
from pylab import *
print 'after pylab import'
check_globals()
print

Just kicking the tires of matplotlib when I discovered
this in the polar_demo.py example:
# you could change the radial bounding box (zoom out)
by setting the
# ylim (radial coordinate is the second argument to
the plot command,
# as in matlab(TM), though this is not advised
currently because it is not
# clear to me how the axes should behave in the change
of view limits.
# Please advise me if you have opinions. Likewise,
the pan/zoom
# controls probably do not do what you think they do
and are better
# left alone on polar axes. Perhaps I will disable
them for polar
# axes unless we come up with a meaningful, useful and
functional
# implementation for them.
I respectfully (and codelessly) suggest that scrolling
and zooming of polar plots be handled as follows:
scrolling: 
Scrolling should progress in two stages:
first the chart center should move to the furthest
edge of the bounding box from the scroll direction.
(The radial axis labels should swing around so that
they are still visible in the bounding box.) The outer
circle should remain tangent to the opposite edges so
that, once the center point reaches a corner, the
chart forms a quarter-circle, with the radial labels
along an edge, and the angular labels along the curve.
once the quadrant is formed, the gridlines can move to
accomodate the scrolling. 
zooming:
would also proceed in two parts. First as before, and
then changing the spacing between gridlines to
accomodate the zooming. (perhaps even drawing new ones
at finer intervals).
I'd have to think some before I could best suggest how
to implement this idea. I wanted to see what people
thought first.
Sincerely,
 Brendan 
--
Brendan Simons
Stern Laboratories
Hamilton, ON, CA
______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca