matplotlib-devel Mailing List for matplotlib

ds...@us... wrote:
> Revision: 4325
> http://matplotlib.svn.sourceforge.net/matplotlib/?rev=4325&view=rev
> Author: dsdale
> Date: 2007年11月15日 13:23:27 -0800 (2007年11月15日)
> 
> Log Message:
> -----------
> added npy.seterr(invalid='ignore') to beginning of axes.py, to silence 
> repeated warnings created by finding extrema of arrays containing nans
> (discovered during calls to errorbar)
Darren,
Is this hiding a problem that will pop up farther down the line? I 
think the strategy so far has been that inputs to plotting functions 
should use masked arrays, not nans, and correspondingly, the plotting 
functions should handle masked arrays gracefully. Although nans are 
used at some internal stages, I don't think they are handled correctly 
from end to end. We could add nan checks at the early argument 
processing stage, but it would slow things down a bit.
Eric

I've created a new basemap branch
https://matplotlib.svn.sourceforge.net/svnroot/matplotlib/trunk/toolkits/basemap-testing
with lots of under-the-hood optimizations and code refactoring. The most significant change is that I now use the GEOS library (http://geos.refractions.net) to determine whether a given coastline or political boundary feature is within the map projection region or not. If it is, the library clips it to the map region, and only the clipped features are processed. This results in big speedups (more than a factor of 10) for small map regions with high-resolution coastlines and boundaries. These changes were motivated by the oceanographers (Eric and Rob Hetland) who often deal with quite small domains. 
There are a couple of downsides:
1) an external dependency on the GEOS lib (which is LGPL). I've included a copy of the source in svn, but there's still an extra ./configure; make; make install required. Eric and I searched high and low for a lighter weight, well-tested, BSD licensed solution, without much luck. General, robust polygon clipping is trickier than I thought.
2) more code paths through Pyrex C-extension code, increasing the probability of segfaults and bus errors. 
All in all, I think the speed gains and increased code readability are worth it. However, before I make a new release I'd appreciate any feedback. Are there build issues? (esp. on Windows, which I've not tested) Does it feel faster? Is the dependence on an LGPL'ed lib a problem?
-Jeff
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jef...@no...
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg

Michael Droettboom wrote:
> John Hunter wrote:
>>> log_demo.py 1.769 2.011 0.242 113%
>>
>> Here is another area where there is an important difference. Panning
>> and zooming interactively with log scaling is much slower on the
>> branch, presumably because you have to redo the non-affine part every
>> time. 
> 
> The non-affine part is not computed on every pan and zoom -- that was 
> one of the main design goals of the branch. (You can put a print 
> statement in Log10Transform.transform to see when it gets called.) I 
> can't feel a speed difference between the two, but...
> 
>> Also, the old grid line bug on log plots seems to be back, as
>> evinced when you zoom from the "home" view.
> 
> ...I should fix this bug first to have a fair comparison.
Fixing that bug actually had a net positive effect on the benchmarks 
overall... (More correct *and* faster? Never happens.)
I created a benchmark using the middle plot of log_demo.py and moving 
the bounds (just like simple_plot_fps.py) and I get the following:
Trunk: 23.68 fps
Branch: 16.83 fps
So there's definitely a slow down there. The profiler shows that a huge 
chunk of the time is spent in numpy/core/ma.py, suggesting that masked 
arrays are the culprit. I think further quarantining of masked arrays 
will help -- for instance, a masked array is created whether or not 
there are any values <= 0.0.
Any thoughts on this are welcome.
Cheers,
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Thursday 15 November 2007 03:27:28 pm Darren Dale wrote:
> I have a script that is generating many plots, and would like to create
> some more informative window titles than "Figure 1" through "Figure 50". Is
> there an easier way to do it than this, which is specific to backend_qt4?
>
> fig = figure()
> fig.canvas.window().setWindowTitle('Hi')
Please disregard. I hit the send button too quickly:
fig.canvas.set_window_title("D'oh!")

I have a script that is generating many plots, and would like to create some 
more informative window titles than "Figure 1" through "Figure 50". Is there 
an easier way to do it than this, which is specific to backend_qt4?
fig = figure()
fig.canvas.window().setWindowTitle('Hi')
Thanks,
Darren

John Hunter wrote:
> On Nov 15, 2007 1:51 PM, Michael Droettboom <md...@st...> wrote:
> 
>> Very odd. I've been running my own very similar benchmark as I've been
>> going, and the two code bases perform quite similarly. The branch
>> continues to cache the markers in more or less the same way as on the
>> trunk. Here are my results with your benchmark:
> 
> Hmm, good guess. I did a clean reinstall of both and the timing
> numbers are very close. Thanks for catching this.
Phew! I was worried we'd have a real mystery on our hands... ;)
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Nov 15, 2007 1:51 PM, Michael Droettboom <md...@st...> wrote:
> Very odd. I've been running my own very similar benchmark as I've been
> going, and the two code bases perform quite similarly. The branch
> continues to cache the markers in more or less the same way as on the
> trunk. Here are my results with your benchmark:
Hmm, good guess. I did a clean reinstall of both and the timing
numbers are very close. Thanks for catching this.
JDH

John Hunter wrote:
> On Nov 15, 2007 12:53 PM, Michael Droettboom <md...@st...> wrote:
> 
>> Thought some of you may be interested to know that the speed on the
>> branch is getting much better. Whereas earlier the branch was about 2x
>> slower than the trunk, now most things are close to equal with the trunk
>> speed-wise (with a few outliers for some things such as auto legends,
>> quivers and the pcolor stuff that Eric and I have been working on).
> 
> Hey Michael, this is very encouraging I just wanted to let know about
> another important use case which I think you are aware of because
> you've referred to optimized marker drawing in the past, but this is
> something I put a lot of effort into (the agg cached marker rasters in
> extension code) because it is an important use case. The script below
> is a useful test, with performance numbers below
> 
> 
> import time
> import numpy as n
> import matplotlib
> matplotlib.use('Agg')
> from pylab import figure
> 
> fig = figure()
> ax = fig.add_subplot(111)
> for i in range(1,7):
> N = 10**i
> x, y = n.random.rand(2,N)
> ax.cla()
> tstart = time.time()
> ax.plot(x, y, 'o')
> fig.canvas.draw()
> print 'N=%d; elapsed=%1.3f'%(N, time.time()-tstart)
> 
> 
> Trunk:
> N=10; elapsed=0.139
> N=100; elapsed=0.092
> N=1000; elapsed=0.082
> N=10000; elapsed=0.133
> N=100000; elapsed=0.594
> N=1000000; elapsed=5.193
> 
> 
> Branch:
> N=10; elapsed=0.207
> N=100; elapsed=0.118
> N=1000; elapsed=0.138
> N=10000; elapsed=0.280
> N=100000; elapsed=1.671
> N=1000000; elapsed=15.877
Very odd. I've been running my own very similar benchmark as I've been 
going, and the two code bases perform quite similarly. The branch 
continues to cache the markers in more or less the same way as on the 
trunk. Here are my results with your benchmark:
Trunk:
N=10; elapsed=0.056
N=100; elapsed=0.039
N=1000; elapsed=0.042
N=10000; elapsed=0.067
N=100000; elapsed=0.326
N=1000000; elapsed=2.913
Branch:
N=10; elapsed=0.033
N=100; elapsed=0.028
N=1000; elapsed=0.030
N=10000; elapsed=0.055
N=100000; elapsed=0.310
N=1000000; elapsed=2.858
I wonder what environmental and/or hardware difference could cause this? 
 Perhaps a fresh rebuild would make a difference? (Due to distutils' 
lack of dependency tracking...?)
>> log_demo.py 1.769 2.011 0.242 113%
> 
> 
> Here is another area where there is an important difference. Panning
> and zooming interactively with log scaling is much slower on the
> branch, presumably because you have to redo the non-affine part every
> time. 
The non-affine part is not computed on every pan and zoom -- that was 
one of the main design goals of the branch. (You can put a print 
statement in Log10Transform.transform to see when it gets called.) I 
can't feel a speed difference between the two, but...
> Also, the old grid line bug on log plots seems to be back, as
> evinced when you zoom from the "home" view.
...I should fix this bug first to have a fair comparison.
Cheers,
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Nov 15, 2007 12:53 PM, Michael Droettboom <md...@st...> wrote:
> Thought some of you may be interested to know that the speed on the
> branch is getting much better. Whereas earlier the branch was about 2x
> slower than the trunk, now most things are close to equal with the trunk
> speed-wise (with a few outliers for some things such as auto legends,
> quivers and the pcolor stuff that Eric and I have been working on).
Hey Michael, this is very encouraging I just wanted to let know about
another important use case which I think you are aware of because
you've referred to optimized marker drawing in the past, but this is
something I put a lot of effort into (the agg cached marker rasters in
extension code) because it is an important use case. The script below
is a useful test, with performance numbers below
import time
import numpy as n
import matplotlib
matplotlib.use('Agg')
from pylab import figure
fig = figure()
ax = fig.add_subplot(111)
for i in range(1,7):
 N = 10**i
 x, y = n.random.rand(2,N)
 ax.cla()
 tstart = time.time()
 ax.plot(x, y, 'o')
 fig.canvas.draw()
 print 'N=%d; elapsed=%1.3f'%(N, time.time()-tstart)
Trunk:
N=10; elapsed=0.139
N=100; elapsed=0.092
N=1000; elapsed=0.082
N=10000; elapsed=0.133
N=100000; elapsed=0.594
N=1000000; elapsed=5.193
Branch:
N=10; elapsed=0.207
N=100; elapsed=0.118
N=1000; elapsed=0.138
N=10000; elapsed=0.280
N=100000; elapsed=1.671
N=1000000; elapsed=15.877
> log_demo.py 1.769 2.011 0.242 113%
Here is another area where there is an important difference. Panning
and zooming interactively with log scaling is much slower on the
branch, presumably because you have to redo the non-affine part every
time. Also, the old grid line bug on log plots seems to be back, as
evinced when you zoom from the "home" view.
Anyway, with a few exceptional cases, your new timing results are
starting to look very promising.
Thanks,
JDH

Thought some of you may be interested to know that the speed on the 
branch is getting much better. Whereas earlier the branch was about 2x 
slower than the trunk, now most things are close to equal with the trunk 
speed-wise (with a few outliers for some things such as auto legends, 
quivers and the pcolor stuff that Eric and I have been working on).
Here are the results for the "simple_plot_fps.py" benchmark, which is 
meant to measure the interactive performance of panning and zooming:
 trunk: 21.63 fps
 branch: 23.25 fps
Attached are the time differences for everything in backend_driver.py. 
(Sorted by the percentage difference in speed.) Note that, unlike the 
above, this measures only one drawing of the plot. It would be 
interesting to measure the difference in interactive performance for 
some of these -- I suspect the branch may do better.
Cheers,
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Nov 14, 2007 3:00 PM, Boyd Waters <bw...@nr...> wrote:
>
> On Nov 14, 2007, at 11:11 AM, Jeremy Conlin wrote:
>
> > Mr. Waters,
> >
> > I read on the matplotlib mailing lists that you have compiled
> > matplotlib on Leopard using libpng, freetype, etc. that are included
> > with Leopard. I am trying to do the same, but have so far been
> > unsuccessful. I could install those libraries from source, but would
> > rather not. Would you mind sharing what you did to compile matplotlib
> > with the included libraries?
>
>
> I did this with a MacPorts build system. It checks out the matplotlib
> from subversion, applies some patches, then builds.
>
> Ah, I built with Apple's pre-release GCC 4.2 for Leopard, which is
> available for ADC members... I have also built this with a "stock" GCC
> 4.2.1, so you might be able to just get that compiler. And almost
> certainly it would work with Apple's GCC 4.0, but you'll have to
> remove the CFLAGS settings (see below).
>
> Oh yes... I also didn't like the flags that Apple used to build
> Python; distutils has no way of overriding these and it breaks GCC
> 4.2.x. So I used a 'fake' gcc that strips off the offending flags and
> then calls the 'real' compiler:
>
> I used MacPorts to do all this. But here is a way that might work with
> a bash shell -- I'm just writing this down as an example, I haven't
> tested it, but this is sort of what I have in mind:
>
> ### ;;;;;;;;;;;;;;;;;;;;;;;;;;; cut here
>
> mkdir mpl-devel
> cd mpl-devel
>
> cat > gccflt-4.2 <<EOF
> #!/usr/bin/env python
>
> # test with this at the command line
> # ./gccflt-4.2 -v -x c -E -mtune=native /dev/null -o /dev/null 2>&1
>
> import sys
> import os
>
> realprog = "/usr/bin/gcc-4.2"
> options_to_remove = set(['-no-cpp-precomp', '-Wno-long-double',
> '-mno-fused-madd', '-faltivec',
> ])
>
> args = [a for a in sys.argv[1:] if a not in options_to_remove]
>
> args.insert(0, '-fno-strict-aliasing')
> print args
> os.execvp(realprog, [realprog] + args)
> EOF
>
>
> sed -e 's,gcc-4.2,g++-4.2' < gccflt > g++flt
>
> chmod +x gccflt
> chmod +x g++flt
>
> svn co http://matplotlib.svn.sourceforge.net/svnroot/matplotlib/trunk/matplotlib
> -r '{2007年10月25日}'
> ;
> export CPPFLAGS=$(/usr/X11R6/bin/freetype-config --cflags)
> export LDFLAGS=$(/usr/X11R6/bin/freetype-config --libs )
> export ARCHFLAGS='-arch i386'
>
> cd matplotlib
>
> patch -p0 < ~/Desktop/patch-setupext.py
> patch -p0 < ~/Desktop/patch-stdc++
>
> python setup.py build
> python setup.pu install
>
>
> #;;;;;;;;;;;;;;;;;;;
> #================================= end shell script
>
>
>
> One of my targets is to make a "developer" build available - which
> means that my boss will be gently reminding me to provide this.
>
> I was granted commit permissions on MacPorts, but have so far failed
> to commit my things back upstream to them; before Leopard I had really
> torn up their Python modules so that everything is deployed in a
> single /opt/local... tree. (I didn't put things in /Library).
>
> With Leopard, if you want to use the system (Apple) Python, then I put
> things in /Library/Python/2.5/site-packages. I'm not sure that's the
> best thing to do. Initially it seems like no problem but we *always*
> run into conflicts, or want to have multiple versions available
> simultaneously...
> Hope this helps!
I'm sorry, but I couldn't get that shell script to work. I tried
copying everything between the "perforations" into a MPL.sh file.
Then I executed the file as
bash MPL.sh
That didn't work. I also tried making a Python file from the portion
that looks like Python code, but I couldn't get that to do anything
useful either.
Please forgive my ignorance :-!
>
> Oh, shoot... I didn't tell you how to build IPython with readline
> support... this email is too long already, but search the IPython dev
> list or I can send along a patch.
I would like a patch for this too, if you don't mind! IPython was my
next step after matplotlib.
Thanks a bunch,
Jeremy
>
> - boyd
>
> Boyd Waters
> http://www.aoc.nrao.edu/~bwaters
>
>
>
>
>
>
>