I have just submitted a patch against CVS to sourceforge which adds support for animation to the WXAgg backend. It also includes an optional C++ extension module intended to speed up the conversion from an Agg image buffer to a wxPython bitmap or image. I would really appreciate any feedback you all have to offer, especially on the follow points. Thanks in advance! 1. How does the _wxagg.cpp module look? I've never dealt with Agg before, nor have I make a Python extension using CXX. 2. Are the changes to setup.py and setupext.py correct and/or sensible? I've tried to keep things tidy while dealing with being unable to find the `wx-config' program. Since animation works without the presence of the WXAgg accelerator, I don't think it makes sense to abort the build process when BUILD_WXAGG is set to "auto". 3. What kind of performance increase are you seeing, for both blit-based animation and regular drawing? I'm not expecting that you see much of an improvement, due to limitations in the way wxPython deals with raster image data, but it would be useful to know. Ken
On Aug 29, 2005, at 12:03 PM, Arnd Baecker wrote: > Also I think according to xvsdiff it > should not be __init__.py in matplotlib but the one > matplotlib/lib/matplotlib/ which should be patched. > So I must be doing something wrong here. I doubt you're doing anything wrong here. I probably don't know how to generate correct CVS diffs for patching, and didn't think to validate my patch by applying it myself. Any advice on submitting diffs against CVS, mailing list? Ken
I may be coming in late to this, but I would suggest: 1. Make sure you are up to date with cvs 2. cvs diff -c -3 > my-mpl-changes.diff I did the tkagg animation and I would be willing to test your wx changes and commit barring any problems. - Charlie Ken McIvor wrote: > On Aug 29, 2005, at 12:03 PM, Arnd Baecker wrote: > >> Also I think according to xvsdiff it >> should not be __init__.py in matplotlib but the one >> matplotlib/lib/matplotlib/ which should be patched. >> So I must be doing something wrong here. > > > I doubt you're doing anything wrong here. I probably don't know how to > generate correct CVS diffs for patching, and didn't think to validate my > patch by applying it myself. > > Any advice on submitting diffs against CVS, mailing list? > > Ken > > > > ------------------------------------------------------- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA > Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Hi Ken, On 2005年8月29日, Ken McIvor wrote: [...] > I will resubmit the wx-anim.tar.gz patch to sourceforge with this > correction. Many thanks for the update of the patch. It turned out that I could not apply it to this mornings CVS checkout because John's time machine was already active and he has put your patch into CVS ;-) During installation I encountered two glitches: a) debian problem: Very interestingly debian stable (sarge) does not have wxPython.h As a work-around I downloaded the corresponding source from debian wget http://ftp.debian.org/debian/pool/main/w/wxwindows2.4/wxwindows2.4_2.4.3.1.tar.gz and copied all header files wxwindows2.4-2.4.3.1/wxPython/src/*.h to a (newly created) /usr/include/wx/wxPython. Then the installation went fine. b) CVS problem: Calling python animation_blit_wx.py from the examples directory gives the error lib/python2.3/site-packages/matplotlib/__init__.py:788: UserWarning: could not find rc file; returning defaults warnings.warn(message) Traceback (most recent call last): File "animation_blit_wx.py", line 10, in ? import pylab as p File "/scratch/abaecker/SOFT//lib/python2.3/site-packages/pylab.py", line 1, in ? from matplotlib.pylab import * File "/scratch/abaecker/SOFT//lib/python2.3/site-packages/matplotlib/pylab.py", line 194, in ? import cm File "/scratch/abaecker/SOFT//lib/python2.3/site-packages/matplotlib/cm.py", line 372, in ? atad['Blues']=_Blues_data NameError: name '_Blues_data' is not defined Commenting out the corresponding block in matplotlib/cm.py everything works fine again (John: I think this is a leftover from rev. 1.19 for cm.py "removed extra cmaps until license is resolved") OK, now to the new WXAgg (and animation in general): Running: python animation_blit.py python animation_blit_tk.py python animation_blit_wx.py python animation_blit_wx.py --no-accel gives: GTKAgg FPS: 131.897223221 TkAgg FPS: 65.5153012595 WXAgg FPS: 100.55 (200 frames: 1.99 seconds, 200 blits: 0.95 seconds) (FPS: 100.55 BPS: 211.00) WXAgg --no-accel FPS: 53.21 (200 frames: 3.76 seconds, 200 blits: 2.55 seconds (FPS: 53.21 BPS: 78.40) I am running debian sarge, on a PIV, 2.8 GHz and an nVdia Quadro4 NVS graphics card (@1600x1200). So WXAgg is almost on par with GTGAgg - great!! Ken, in _wxagg.cpp you mention that you planned to write a agg_to_wx_bitmap() which draws directly to a bitmap, but that this might not speed up things much because of: AGG->wx.Image->wx.Bitmap before you can blit using a MemoryDC. Maybe you can contact Chris Barker if he has an idea on this (or are there any solutions available in the context of wxart2d or chaco? - quick googling did not reveal anything ...) Anyway, the speed provided by the animate option is a fantastic improvement - many thanks John and Ken!! Just for comparison: for backend in `echo "Tk TkAgg GTK GTKAgg WX WXAgg"` do echo $backend python anim.py -d${backend} done Tk FPS: 35.5331434162 TkAgg FPS: 20.3487964649 GTK FPS: 35.528699289 GTKAgg FPS: 26.9074415174 WX FPS: 13.7550465745 WXAgg FPS: 26.0163513384 Many thanks, Arnd
On Aug 30, 2005, at 9:31 AM, Arnd Baecker wrote: > Many thanks for the update of the patch. > It turned out that I could not apply it to this mornings CVS checkout > because John's time machine was already active and he has put your > patch into CVS ;-) Really? Yay for me! ;-) > During installation I encountered two glitches: > > a) debian problem: > Very interestingly debian stable (sarge) > does not have wxPython.h It does have wxPython.h, it's just not in the libwxgtk2.4 package. You'll need to install the libwxgtk2.4-dev package as well to get the headers. > GTKAgg FPS: 131.897223221 > TkAgg FPS: 65.5153012595 > WXAgg FPS: 100.55 > (200 frames: 1.99 seconds, 200 blits: 0.95 seconds) > (FPS: 100.55 BPS: 211.00) > WXAgg --no-accel FPS: 53.21 > (200 frames: 3.76 seconds, 200 blits: 2.55 seconds > (FPS: 53.21 BPS: 78.40) > > > I am running debian sarge, on a PIV, 2.8 GHz and an nVdia Quadro4 NVS > graphics card (@1600x1200). > So WXAgg is almost on par with GTGAgg - great!! That's very exciting news. I hadn't had a chance to test and profile the accelerator under Debian, nor had I bothered profiling it against GtkAgg and TkAgg under OSX. I should probably get those backends up and running (darn macs) and do so. > Ken, in _wxagg.cpp you mention that you planned to write a > agg_to_wx_bitmap() > which draws directly to a bitmap, but that this might not speed up > things > much because of: AGG->wx.Image->wx.Bitmap before you can blit using a > MemoryDC. Yes, I am concerned that re-implementing that part of the blit into C++ won't speed things up enough. I can understand doing the whole Image/Bitmap thing for portability (especially when wxWindows got started in 1992), but it's rather obnoxious when all it does is make extra copies of RGB data. However, I think there are a couple pieces of lower-handing fruit, like not making an extra source copy when blitting (_wxagg.cpp:159-190), that will offer speedups. Unfortunately, they'd be portable across Agg backends, so GtkAgg would get faster too. ;-) > Maybe you can contact Chris Barker if he has an idea on this (or are > there any > solutions available in the context of wxart2d or chaco? - quick > googling did not > reveal anything ...) I'll do so, if he doesn't see this part of the message than chime in. I have come across some old wxWindows mailing list exchanges indicating that there's no way to avoid the Image->Bitmap conversion as a requirement for drawing RGB images, so I'm not planning to loose too much sleep over it. I hadn't heard about wxart2d before, but will look into it to see how their agg canvas is implemented. I'll see if I can find the chaco repository and do the same there. Thanks for sharing your anim.py profiling results. It's heartening to see WXAgg's full-redraw speed being limited by the rest of matplotlib, rather than the backend. Ken
Hi, On 2005年8月30日, Ken McIvor wrote: > On Aug 30, 2005, at 9:31 AM, Arnd Baecker wrote: [...] > > a) debian problem: > > Very interestingly debian stable (sarge) > > does not have wxPython.h > > It does have wxPython.h, it's just not in the libwxgtk2.4 package. > You'll need to install the libwxgtk2.4-dev package as well to get the > headers. I would have thought so as well. However I get: dpkg -L libwxgtk2.4-dev /. /usr /usr/bin /usr/bin/wxgtk-2.4-config /usr/lib /usr/lib/wx /usr/lib/wx/include /usr/lib/wx/include/univ /usr/lib/wx/include/gtk-2.4 /usr/lib/wx/include/gtk-2.4/wx /usr/lib/wx/include/gtk-2.4/wx/setup.h /usr/lib/libwx_gtk-2.4.a /usr/lib/libwx_gtk_gl-2.4.a /usr/share /usr/share/doc /usr/share/doc/libwxgtk2.4-dev /usr/share/doc/libwxgtk2.4-dev/copyright /usr/share/doc/libwxgtk2.4-dev/changelog.gz /usr/share/man /usr/share/man/man1 /usr/lib/libwx_gtk-2.4.so /usr/lib/libwx_gtk_gl-2.4.so /usr/share/man/man1/wxgtk-2.4-config.1.gz A search for wxPython.h on http://www.debian.org/distrib/packages#search_packages gives no hits. > > GTKAgg FPS: 131.897223221 > > TkAgg FPS: 65.5153012595 > > WXAgg FPS: 100.55 > > (200 frames: 1.99 seconds, 200 blits: 0.95 seconds) > > (FPS: 100.55 BPS: 211.00) > > WXAgg --no-accel FPS: 53.21 > > (200 frames: 3.76 seconds, 200 blits: 2.55 seconds > > (FPS: 53.21 BPS: 78.40) > > > > > > I am running debian sarge, on a PIV, 2.8 GHz and an nVdia Quadro4 NVS > > graphics card (@1600x1200). > > So WXAgg is almost on par with GTGAgg - great!! > > That's very exciting news. I hadn't had a chance to test and profile > the accelerator under Debian, nor had I bothered profiling it against > GtkAgg and TkAgg under OSX. I should probably get those backends up > and running (darn macs) and do so. > > > Ken, in _wxagg.cpp you mention that you planned to write a > > agg_to_wx_bitmap() > > which draws directly to a bitmap, but that this might not speed up > > things > > much because of: AGG->wx.Image->wx.Bitmap before you can blit using a > > MemoryDC. > > Yes, I am concerned that re-implementing that part of the blit into C++ > won't speed things up enough. I can understand doing the whole > Image/Bitmap thing for portability (especially when wxWindows got > started in 1992), but it's rather obnoxious when all it does is make > extra copies of RGB data. I agree. I wonder if anything improved with wx2.6? (for our PlottingCanvas we even dared to keep drawing DCs around, and it works without problems...) > However, I think there are a couple pieces > of lower-handing fruit, like not making an extra source copy when > blitting (_wxagg.cpp:159-190), that will offer speedups. That would be cool (I have to admit, that I am not too worried when FPS>100 on my machine. However, we are currently investigating to use matplotlib for a computational physics course (which will be next summer) and many of the students have *much* slower machines. So we need maximum speed but with a minimum of coding hassle (around 30% of the students have never programmed before ...). > Unfortunately, they'd be portable across Agg backends, so GtkAgg would > get faster too. ;-) I would not mind too much ;-). > > Maybe you can contact Chris Barker if he has an idea on this (or are > > there any > > solutions available in the context of wxart2d or chaco? - quick > > googling did not > > reveal anything ...) > > I'll do so, if he doesn't see this part of the message than chime in. > I have come across some old wxWindows mailing list exchanges indicating > that there's no way to avoid the Image->Bitmap conversion as a > requirement for drawing RGB images, so I'm not planning to loose too > much sleep over it. I hadn't heard about wxart2d before, but will look > into it to see how their agg canvas is implemented. I'll see if I can > find the chaco repository and do the same there. > > Thanks for sharing your anim.py profiling results. It's heartening to > see WXAgg's full-redraw speed being limited by the rest of matplotlib, > rather than the backend. Indeed this is good news!! Many thanks, Arnd
>>>>> "Arnd" == Arnd Baecker <arn...@we...> writes: >> It does have wxPython.h, it's just not in the libwxgtk2.4 >> package. You'll need to install the libwxgtk2.4-dev package as >> well to get the headers. Arnd> I would have thought so as well. However I get: .... Arnd> A search for wxPython.h on Arnd> http://www.debian.org/distrib/packages#search_packages gives Arnd> no hits. I found this too -- it may be rightly considered a bug in the dev package. I checked out the src package and then copied over the headers. The following worked for me apt-get source libwxgtk2.5.3 sudo cp -a wxwidgets2.5-2.5.3.2ubuntu4/include/wx-2.5/wx/wxPython /usr/include/wx-2.5/wx/ JDH
Ken, all, I'd tried this patch out on Mac OS X, with standard Mac-based python and wxPython 2.6. I had to explicitly set the environmental variable WX_CONFIG, but after that it built fine and runs fine. For my scripts on a slow G4 Powerbook, I see a small improvement in speed compared with mpl 0.81. I'll be eager to try this on Windows. Thanks Ken, --Matt Newville <newville at cars.uchicago.edu>
On Aug 30, 2005, at 10:45 AM, Arnd Baecker wrote: > I would have thought so as well. However I get: <snip> > A search for wxPython.h on > http://www.debian.org/distrib/packages#search_packages > gives no hits. Sorry, due to a lack of coffee this morning I misread "wxPython.h" as "wxWidgets.h" or something daft, yielding the kneejerk "install the -dev package" response. I wasn't aware of this situation, but it has the potential to be a big problem for me at work, where we primarily run Debian. I have emailed Ron Lee, the wxgtk2.4 package maintainer, about the situation. We'll see what he has to say on the matter. > I agree. I wonder if anything improved with wx2.6? (for our > PlottingCanvas > we even dared to keep drawing DCs around, and it works without > problems...) What PlottingCanvas is this? I'd be interested in seeing what optimizations you guys performed, if the source is available. >> However, I think there are a couple pieces of lower-handing fruit, >> like not >> making an extra source copy when blitting (_wxagg.cpp:159-190), that >> will >> offer speedups. > > That would be cool (I have to admit, that I am not too worried when > FPS>100 > on my machine. However, we are currently investigating to use > matplotlib for > a computational physics course (which will be next summer) and many of > the > students have *much* slower machines. So we need maximum speed but > with a > minimum of coding hassle (around 30% of the students have never > programmed > before ...). Just leveling the playing field between WXAgg and GtkAgg is exciting for me, because that means that future efforts at general optimization will net a bigger speed improvement for WXAgg. I'd imagine the plotting speed will be good enough for something along the lines of interactive plotting with iPython or visualizing results with pylab. Speaking as a recent survivor of a computational physics class, I expect you to see a huge benefit from using Python as the language and matplotlib as the visualization, especially if you have students who have never programmed before. Ken
Hi, On 2005年8月30日, Ken McIvor wrote: > On Aug 30, 2005, at 10:45 AM, Arnd Baecker wrote: > > I would have thought so as well. However I get: > <snip> > > A search for wxPython.h on > > http://www.debian.org/distrib/packages#search_packages > > gives no hits. > > Sorry, due to a lack of coffee this morning I misread "wxPython.h" as > "wxWidgets.h" or something daft, yielding the kneejerk "install the > -dev package" response. > > I wasn't aware of this situation, but it has the potential to be a big > problem for me at work, where we primarily run Debian. I have emailed > Ron Lee, the wxgtk2.4 package maintainer, about the situation. We'll > see what he has to say on the matter. Excellent - I was thinking about doing the same. > > I agree. I wonder if anything improved with wx2.6? (for our > > PlottingCanvas > > we even dared to keep drawing DCs around, and it works without > > problems...) > > What PlottingCanvas is this? I'd be interested in seeing what > optimizations you guys performed, if the source is available. http://www.physik.tu-dresden.de/~baecker/python/plot.html One of our main goals was to plot many points quickly, in such a way that one appears after another to get a "dynamic" appearance. See http://www.physik.tu-dresden.de/~baecker/python/StandardMap.py as an example. (note that it still uses the old wx style...) [...] > > However, we are currently investigating to use > > matplotlib for > > a computational physics course (which will be next summer) and many of > > the > > students have *much* slower machines. So we need maximum speed but > > with a > > minimum of coding hassle (around 30% of the students have never > > programmed > > before ...). > > Just leveling the playing field between WXAgg and GtkAgg is exciting > for me, because that means that future efforts at general optimization > will net a bigger speed improvement for WXAgg. > > I'd imagine the plotting speed will be good enough for something along > the lines of interactive plotting with iPython or visualizing results > with pylab. We have just finished the conversion of all exercices of our course from scipy.xplt (aka pygist) to matplotlib. Unfortunately, quite a few are prohibitively slow, even on our fast machines. But we have some ideas on possible improvements (both on the side of our code and on the side of matplotlib) - this is going to be separate thread though ;-) > Speaking as a recent survivor of a computational physics > class, I expect you to see a huge benefit from using Python as the > language and matplotlib as the visualization, especially if you have > students who have never programmed before. I absolutely agree - we have been running our course now for the third year, so far with scipy and scipy.xplt as plotting programm. We had very positive feedback (of course, those who never programmed before, had to work harder ;-). Just in case: http://www.comp-phys.tu-dresden.de/cp2005/, however the material (apart from the FAQ) is in German. Best, Arnd