matplotlib-devel Mailing List for matplotlib

On Nov 8, 2007, at 10:23 AM, Michael Droettboom wrote:
>
> So, we need to look at the pros/cons of continuing to support these
> legacy APIs going forward.
I've got some more benchmarks for the WX and WXAgg backends in 
trunk. It looks like using ssh with compression or the NX remote 
desktop both substantially improve performance. I don't know why 
WXAgg consistently outperforms WX for me, but my guess is that the 
difference involves Apple's X server.
X11 Remote
	WX: 0.96 fps
	WXAgg: 1.93 fps
X11 Remote w/compression
	WX: 3.93 fps
	WXAgg: 6.43 fps
NX
	WX: 7.39 fps
	WXAgg: 10.83 fps
Local
	WX: 11.91 fps
	WXAgg: 19.16 fps
Ken

Christopher Barker wrote:
> Michael Droettboom wrote:
>> Wx supports polycurves in its new wxGraphicsContext API (but not the 
>> wxDC API that mpl uses now). This means a fairly complete rewrite of 
>> the wx backend,
> 
> not necessarily. You can create a GraphicsContext from a wx.DC, you may 
> be able to just add that step for drawing polycurves.
>
>> since you 
>> can no longer draw to an in-memory buffer, but only a true wxClientDC). 
> 
> you can create a GraphicsContext from a wxMemeoryDC, so yes, you can 
> draw to a buffer.
I'm having trouble just getting a wxPython to compile with 
wxGraphicsContext support (perhaps another argument against using it...)
I was basing this determination on the wx documentation which clearly 
states one can only create a wxGraphicsContext from a wxWindowDC. 
wxMemoryDC inherits from wxDC, not wxWindowDC. I'd be certainly happy to 
find out that the documentation is incorrect on this point, as it would 
certainly make the implementation easier.
> I haev no idea what the impact on performance would be, but in general, 
> GraphicsContext is slower than DC.
Particularly, if it works as it appears to by rendering to a Cairo 
buffer first.
>> So, we need to look at the pros/cons of continuing to support these 
>> legacy APIs going forward.
> 
> yup. I know I never need a non-agg wx, but then I can see why folks 
> running remote X severs would want it.
> 
> I wonder if there is another way to speed that up with Agg -- can you 
> compress the bitmap data to pass it to the Xserver? is that happening 
> already?
You can compress the ssh connection over which X11 tunnels with the -C
commandline flag. But whether that makes things faster or slower 
depends on the compressability of the data itself, and the relative 
speeds of the processor vs. network link, of course.
Cheers,
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Nov 8, 2007 7:10 AM, Darren Dale <dar...@co...> wrote:
> Fernando, do you have a record of the changes you guys made at SciPy-2007?
No, we just went through the codebase by hand and quickly removed
anything that looked like it would do namespace packages. Once that
was out of the way, the traits init was actually *shorter* than the
non-traits one, since traits property-like checks are dispatched in a
tight, highly optimized C core (ctraits.c) instead of via pure python.
Sorry not to have that benchmark code available, at this point I think
I'd have to redo it by hand.
Cheers,
f

On Nov 8, 2007, at 12:17 PM, Christopher Barker wrote:
>
> I wonder if there is another way to speed that up with Agg -- can you
> compress the bitmap data to pass it to the Xserver? is that happening
> already?
I'm not aware of any obvious method for enabling compression in 
remote X11 connections. Using the NX remote desktop system might be 
faster:
	http://www.nomachine.com
Ken

Michael Droettboom wrote:
> Gael Varoquaux wrote:
>> On Thu, Nov 08, 2007 at 11:23:54AM -0500, Michael Droettboom wrote:
>>> So, we need to look at the pros/cons of continuing to support these
>>> legacy APIs going forward.
>> IMHO a gtk and a wx back end are very important for embedding MPL in
>> applications. I am sure you have a replacement proposition, but I fail to
>> see in your list of back-end what it would be.
> 
> I'm talking specifically about the gdk and wx drawing backends, not the 
> GUI backends. Gtk and Wx applications that embed MPL can still use Agg 
> (or Cairo with Gtk) for rendering.
Can Cairo handle the remote-X use case?
Eric

Michael Droettboom wrote:
> Wx supports polycurves in its new wxGraphicsContext API (but not the 
> wxDC API that mpl uses now). This means a fairly complete rewrite of 
> the wx backend,
not necessarily. You can create a GraphicsContext from a wx.DC, you may 
be able to just add that step for drawing polycurves.
> since you 
> can no longer draw to an in-memory buffer, but only a true wxClientDC). 
you can create a GraphicsContext from a wxMemeoryDC, so yes, you can 
draw to a buffer.
I haev no idea what the impact on performance would be, but in general, 
GraphicsContext is slower than DC.
> So, we need to look at the pros/cons of continuing to support these 
> legacy APIs going forward.
yup. I know I never need a non-agg wx, but then I can see why folks 
running remote X severs would want it.
I wonder if there is another way to speed that up with Agg -- can you 
compress the bitmap data to pass it to the Xserver? is that happening 
already?
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

Gael Varoquaux wrote:
> On Thu, Nov 08, 2007 at 11:23:54AM -0500, Michael Droettboom wrote:
>> So, we need to look at the pros/cons of continuing to support these
>> legacy APIs going forward.
> 
> IMHO a gtk and a wx back end are very important for embedding MPL in
> applications. I am sure you have a replacement proposition, but I fail to
> see in your list of back-end what it would be.
I'm talking specifically about the gdk and wx drawing backends, not the 
GUI backends. Gtk and Wx applications that embed MPL can still use Agg 
(or Cairo with Gtk) for rendering.
Cheers,
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Thu, Nov 08, 2007 at 11:23:54AM -0500, Michael Droettboom wrote:
> So, we need to look at the pros/cons of continuing to support these
> legacy APIs going forward.
IMHO a gtk and a wx back end are very important for embedding MPL in
applications. I am sure you have a replacement proposition, but I fail to
see in your list of back-end what it would be.
Cheers,
Gaël

The following drawing backends are functional on the transforms branch:
	Agg
	Ps
	Pdf *
	Svg
	Cairo *
* These could still use some optimizations to draw_markers and/or 
draw_path_collection.
Now onto the "problem" ones. I had asked on the mailing list a while 
back about the current usefulness of the Gdk and Wx backends, and the 
one advantage cited was that they are faster when running in a remote X 
session (a valid point for some uses). I can confirm that this is the 
case -- for a completely non-controlled experiment, I ran a remote X11 
session over our network here. I get (on simple_plot_fps.py):
Gtk: 38.44 fps
GtkAgg: 1.87 fps
Wx: 19.37 fps
WxAgg: 1.83 fps
Running locally, I get:
Gtk: 49.39 fps
GtkAgg: 27.15 fps
Wx: 19.40 fps
WxAgg: 26.04 fps
Interestingly, WxAgg is faster than Wx when running locally.
What makes updating these backends on the branch difficult? The 
fundamental drawing operation on the branch is compound polycurves (i.e. 
one or more separate polylines with possible bezier curve segments).
Gdk doesn't support polycurves natively. We could convert our 
polycurves to polygons (by approximating the bezier curves) on the fly 
and send that. That shouldn't be a major slow down in the case where 
there aren't in fact any curves, but it would mean writing some 
extension code etc.
Wx supports polycurves in its new wxGraphicsContext API (but not the 
wxDC API that mpl uses now). This means a fairly complete rewrite of 
the wx backend, that would likely affect embedders as well (since you 
can no longer draw to an in-memory buffer, but only a true wxClientDC). 
 wxGraphicsContext is built on top of GDI+, Quartz or Cairo depending 
on platform -- therefore it's not a given that it will perform well over 
remote X-Windows, either.
So, we need to look at the pros/cons of continuing to support these 
legacy APIs going forward.
Lastly, what is the status of the EMF backend? Is anyone relying on it 
and/or willing to look into updating it?
Cheers,
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Thursday 08 November 2007 03:16:14 am Eric Firing wrote:
> Darren Dale wrote:
> > Building does not seem to be a problem, just do the usual python setup.=
py
> > build, etc. I ran backend_driver on my not-so-speedy home computer:
>
> Darren,
>
> Well done! I blew away other versions of traits, updated mpl, built and
> installed, and traits appeared. No pain.
>
> The not quite as good news is that on my Lenovo T60 laptop, the Template
> takes 0.57 minutes with traited config versus 0.45 with old config. Not
> a huge difference, and roughly consistent with your timing, but still a
> penalty. I also see about 20% in simple_plot using Agg. It's probably
> tolerable. (The comparison was made by switching only the NEWCONFIG
> variable in __init__.py.)
Yes, these results are not consistent with what Fernando reported after=20
working with Dave Peterson:
# Using traits
maqroll[mpl-traits-debug]> time ./simple_plot.py
*** Using Traits!!!
1.844u 0.212s 0:02.13 96.2% =A0 =A0 0+0k 0+0io 0pf+0w
maqroll[mpl-traits-debug]> time ./simple_plot.py
*** Using Traits!!!
1.840u 0.216s 0:02.58 79.4% =A0 =A0 0+0k 0+0io 0pf+0w
maqroll[mpl-traits-debug]> time ./simple_plot.py
*** Using Traits!!!
1.836u 0.196s 0:02.12 95.2% =A0 =A0 0+0k 0+0io 0pf+0w
# NOT Using traits
maqroll[mpl-traits-debug]> time ./simple_plot.py
2.200u 0.280s 0:02.67 92.8% =A0 =A0 0+0k 0+0io 0pf+0w
maqroll[mpl-traits-debug]> time ./simple_plot.py
2.248u 0.220s 0:02.74 89.7% =A0 =A0 0+0k 0+0io 0pf+0w
maqroll[mpl-traits-debug]> time ./simple_plot.py
2.216u 0.244s 0:02.72 90.0% =A0 =A0 0+0k 0+0io 0pf+0w
=46ernando, do you have a record of the changes you guys made at SciPy-2007?
> > with traited config:
> >
> > Backend Agg took 2.77 minutes to complete
> > template ratio 1.748, template residual 1.187
> > Backend PS took 2.64 minutes to complete
> > template ratio 1.666, template residual 1.055
> > Backend Template took 1.59 minutes to complete
> > template ratio 1.000, template residual 0.000
> > Backend PDF took 3.35 minutes to complete
> > template ratio 2.112, template residual 1.764
> > Backend SVG took 2.90 minutes to complete
> > template ratio 1.826, template residual 1.309
> >
> >
> > and without:
> >
> > Backend Agg took 2.59 minutes to complete
> > template ratio 1.898, template residual 1.226
> > Backend PS took 2.29 minutes to complete
> > template ratio 1.675, template residual 0.921
> > Backend Template took 1.36 minutes to complete
> > template ratio 1.000, template residual 0.000
> > Backend PDF took 2.92 minutes to complete
> > template ratio 2.139, template residual 1.555
> > Backend SVG took 2.91 minutes to complete
> > template ratio 2.129, template residual 1.541

Darren Dale wrote:
> Building does not seem to be a problem, just do the usual python setup.py 
> build, etc. I ran backend_driver on my not-so-speedy home computer:
Darren,
Well done! I blew away other versions of traits, updated mpl, built and 
installed, and traits appeared. No pain.
The not quite as good news is that on my Lenovo T60 laptop, the Template 
takes 0.57 minutes with traited config versus 0.45 with old config. Not 
a huge difference, and roughly consistent with your timing, but still a 
penalty. I also see about 20% in simple_plot using Agg. It's probably 
tolerable. (The comparison was made by switching only the NEWCONFIG 
variable in __init__.py.)
Eric
> 
> with traited config:
> 
> Backend Agg took 2.77 minutes to complete
> template ratio 1.748, template residual 1.187
> Backend PS took 2.64 minutes to complete
> template ratio 1.666, template residual 1.055
> Backend Template took 1.59 minutes to complete
> template ratio 1.000, template residual 0.000
> Backend PDF took 3.35 minutes to complete
> template ratio 2.112, template residual 1.764
> Backend SVG took 2.90 minutes to complete
> template ratio 1.826, template residual 1.309
> 
> 
> and without:
> 
> Backend Agg took 2.59 minutes to complete
> template ratio 1.898, template residual 1.226
> Backend PS took 2.29 minutes to complete
> template ratio 1.675, template residual 0.921
> Backend Template took 1.36 minutes to complete
> template ratio 1.000, template residual 0.000
> Backend PDF took 2.92 minutes to complete
> template ratio 2.139, template residual 1.555
> Backend SVG took 2.91 minutes to complete
> template ratio 2.129, template residual 1.541
> 
> 
> changes in svn-4156.
> 
> Darren

On Wednesday 07 November 2007 3:54:36 pm Darren Dale wrote:
> On Wednesday 07 November 2007 03:34:18 pm Eric Firing wrote:
> > Darren Dale wrote:
> > > I have been working on updating the trunk to provide enthought.traits
> > > version 2.6b1. backend_driver.py is running without exceptions using
> > > the traited config package with the internal traits package.
[...]
> > 2) Does Gael's version already get around the slow initialization
> > problem? (I presume so--it was pkg-resources that was causing the
> > trouble, wasn't it?) I think this is important. Some mpl applications
> > involve running simple scripts many times, so startup time matters. I
> > would not want to see traits as an external dependency if that brought
> > with it the startup lag--apart from all the other questions associated
> > with making it an external dependency in any of its present forms.
>
> I think it does get us around the initialization problem, although I need
> to run backend_driver once I get the installation worked out to be sure.
I removed the old enthought package from lib/matplotlib, and added the 
setuptools-less 2.6b1 package to lib/. It now installs like pytz or dateutil, 
directly into site-packages and only if it is not already available. The only 
modification I made to the enthought code was in the version files, which 
return '2.6b1-mpl' instead of ''. The sys.path workaround mentioned in the 
original post is no longer necessary and has been removed.
Building does not seem to be a problem, just do the usual python setup.py 
build, etc. I ran backend_driver on my not-so-speedy home computer:
with traited config:
Backend Agg took 2.77 minutes to complete
 template ratio 1.748, template residual 1.187
Backend PS took 2.64 minutes to complete
 template ratio 1.666, template residual 1.055
Backend Template took 1.59 minutes to complete
 template ratio 1.000, template residual 0.000
Backend PDF took 3.35 minutes to complete
 template ratio 2.112, template residual 1.764
Backend SVG took 2.90 minutes to complete
 template ratio 1.826, template residual 1.309
and without:
Backend Agg took 2.59 minutes to complete
 template ratio 1.898, template residual 1.226
Backend PS took 2.29 minutes to complete
 template ratio 1.675, template residual 0.921
Backend Template took 1.36 minutes to complete
 template ratio 1.000, template residual 0.000
Backend PDF took 2.92 minutes to complete
 template ratio 2.139, template residual 1.555
Backend SVG took 2.91 minutes to complete
 template ratio 2.129, template residual 1.541
changes in svn-4156.
Darren