matplotlib-users Mailing List for matplotlib

>>>>> "matthew" == matthew arnison <ma...@ca...> writes:
 matthew> Excuse me, what does AGG stand for? And I'm curious,
 matthew> have you looked into cairo as a possible backend? It's a
 matthew> vector drawing library that's trying to be OS
 matthew> independent. I think there's a python interface. So far
 matthew> it outputs to bitmap images, X11, postscript, and an Open
 matthew> GL port is underway.
Agg is the antigrain geometry image library. It's a vector oriented
C++ graphics library that is very fast and has lots of nice features.
The name antigrain means that it is designed not to have pixel
granularity, aka aliasing, in drawing. It achieves this with1 pixel
blending algorithms from computational geometry.
 http://antigrain.com
It's also the rendering library used by chaco.
 matthew> http://freedesktop.org/Cairo/Home
Many moons ago, a cairo backend was suggested -- also by an
Australian :-). At that time, the consensus was perhaps it made more
sense to do an SVG backend - which you could then feed to cairo.
 matthew> Interested to hear your thoughts. Maybe it's a matter of
 matthew> a job looking for a volunteer.
Are you considering it? Actually, an svg (or cairo) backend would not
be too hard since you could follow the paint backend. paint/libart is
modeled after SVG, so it provides a good template.
JDH

John,
I've looked into this a bit more.
I think I'll have it working shortly, seems I have to do three things to
get landscape:
 * set the paper size
 * add a '90 rotate' incantation to the postscript at a suitable point
 * add a 'x y translate' command to adjust the origin 
Getting last bit right is making my head hurt, but I should have
something working by tomorrow.
John
On Tue, 2004年02月10日 at 15:05, John Hunter wrote:
> >>>>> "John" == John Gill <jn...@eu...> writes:
> 
> John> I have some plots I'd like to print out and they would make
> John> better use of the paper if they were done landscape.
> 
> John> Can the postscript backend do this?
> 
> Hi John,
> 
> I haven't had time to take a close look at this. My initial
> suggestions is to experiment with the paper size
> 
> import matplotlib
> matplotlib.use('PS')
> import matplotlib.backends.backend_ps as backend_ps
> backend_ps.defaultPaperSize = 11,8.5 # default is 8.5, 11
> 
> You may also have to specify a landscape portrait at print time, or
> rotate it.
> 
> I'll take a closer look later.
> 
> JDH
> 
> 
> 

>
>
> Peter> 3) When I do:
>
> plot1 = plot(arange(10000), arange(20000,30000))
> lod, aa = False, False
> set(l, 'lod', lod, 'antialiased', aa)
>
>This code isn't correct. plot returns a list of lines. The set
> 
>
Of course.. changed the 'l' to 'plot1' because I hate how my 'l's look 
like '1's.. But i didn't do the same in the 'set' command.. I meant it 
though.. :)
 
> lines1 = plot(arange(10000), arange(20000,30000))
> set(lines1, 'antialiased', False)
>
> lines2 = plot([1,2,3]) # a small plot
> set(lines2, 'antialiased', True)
>
>Now lines1 is aliased and lines2 is antialiased.
> 
>
Great! This proides awesome flexibility!
Thanks for all the info. I will get a 
usable-skeleton-proof-of-concept-type app going with GD first, and once 
that is working, will get experiment with agg.
Peter

Perry Greenfield wrote:
That was a response to:
>If you want to work off line (ie, draw
>to image only and not display to screen ) and are on a linux box, you
>can do this with GTK and Xvfb
Perhaps it was a misinterpretation on my part. All I meant was that I do not need to look at the images through any of the standard tools (ie, via show()). For my purposes I need to write the image to stdout (not to disk) in its binary form - which is what I do now after modifying some stuff in backend_gd.py. 
>Peter Groszkowski wrote:
> 
>
>>Yes I am only planning to work offline. Want to be able to pipe the 
>>output images to stdout. I am looking for the fastest solution possible.
>>
>> 
>>
>Following up on this, I was curious what exactly you meant by
>this. A stream of byte values in ascii separated by spaces?
>Or the actual binary bytes? If the latter, it wouldn't appear
>to be difficult to write a C extension to return the image
>as a string, but I'm figuring there is more to it than that
>since the representations for the image structure can change
>from one backend to another.
>
>Perry
>
> 
>

>>>>> "Peter" == Peter Groszkowski <pgr...@ge...> writes:
 >> 
 Peter> With antialiasing off, the performance is superb!.. I plot
 Peter> 500,000 points in ~4-5 seconds.. The visual quality of the
 Peter> graphs is (naturally) inferior to the antialiased
 Peter> counterparts, but the software is now feasible for my
 Peter> purposes.
Glad to hear it... The next big performance boost will come from some
frontend refactoring along the lines Perry discussed, but I'm glad to
hear it's usable for you now.
 Peter> 1) Seems like setting 'lod' to true does not improve
 Peter> performance? I would imagine it should, because it limits
 Peter> the amount of points used. What am I missing?
I'll look into this further. In the special case of EEG (128 channels
plotted simultaneously over the same time axis), I do see significant
benefits but I cache the sampling indexes from one line to the next.
It may be that for single lines the time it takes to do the
subsampling balances the time it takes to plot them in a fast backend.
 Peter> 2) Is there any way to make the graphs look "prettier"?
 Peter> They really look quite OK but in some cases having a little
 Peter> more detail would be nice. Is it possible specify just how
 Peter> much antialiasing is needed? Are there any other "visual
 Peter> enchantment options" that can be set, and will not impact
 Peter> performace too much?
Well, fortunately for you, I just finished the agg backend this
morning. This backend draws antialiased lines as fast as GD draws
unaliased lines. I still don't have support for turning off
antialiasing in agg, but it sounds like you want to have it. Also, it
doesn't suffer from a known color allocation and fill bug that GD has.
See install instructions at the end of this email.
 Peter> 3) When I do:
 plot1 = plot(arange(10000), arange(20000,30000))
 lod, aa = False, False
 set(l, 'lod', lod, 'antialiased', aa)
This code isn't correct. plot returns a list of lines. The set
command should operate on that list of lines. It applies only to the
lines returned. So *you can* apply antialising with respect to one
set of lines and not another, in the same axes
 lines1 = plot(arange(10000), arange(20000,30000))
 set(lines1, 'antialiased', False)
 lines2 = plot([1,2,3]) # a small plot
 set(lines2, 'antialiased', True)
Now lines1 is aliased and lines2 is antialiased.
 Peter> I have been playing around with the dpi setting a
 Peter> little. Is it supposed to change the size of the image
 Peter> and/or the resolution??
The figure size in pixels is determined by the figsize parameter and
dpi. 
 width, height = figsize
 width *= dpi
 height *= dpi
Everything scales with DPI, line width, text size, dash spacing,
etc.. So the answer to your question is: both figure size and
resolution increase with dpi. If you want to change figure size w/o
changing resolution, change the figsize argument to figure.
The agg backend
 Warning: you will be the first agg crash test dummy. I just ran a
 suite of examples across all backends and agg was the fastest - it's
 even faster than template, which does no rendering or filesaving!
 And in my opinion it also produced the highest quality output.
 Features that are implemented
 * capstyles and join styles
 * dashes
 * linewidth 
 * lines, rectangles, ellipses, polygone
 * clipping to a rectangle
 * output to RGBA and PNG
 * alpha blending
 * DPI scaling - (dashes, linewidths, fontsizes, etc)
 * freetype1
 TODO:
 * use ttf manager to get font - right now I just use Vera
 INSTALLING 
 Grab the latest matplotlib from
 http://nitace.bsd.uchicago.edu:8080/files/share/matplotlib-0.50l.tar.gz
 REQUIREMENTs
 python2.2+
 Numeric 22+
 agg2 (see below)
 freetype 1
 libpng
 libz ?
 
 Install AGG2 (cut and paste below into xterm should work)
 wget http://www.antigrain.com/agg2.tar.gz
 tar xvfz agg2.tar.gz
 cd agg2
 make
 (Optional) if you want to make the examples:
 cd examples/X11
 make
 Installing backend_agg
 Edit setup.py: change aggsrc to point to the agg2 src tree and
 replace if 0: with if 1: in the backend_agg section
 Then just do the usual thing: python setup.py build
 Please let me know if you encounter build problems, and tell me
 platform, gcc version, etc... Currently the paths in setupext.py
 assume as linux like filesystem (eg X11 include dir, location of
 libttf, etcc) so you may need to tweak these. But if I recall
 correctly, we're both on RHL9 so you shouldn't have a problem.
 Using agg backend
 python somefile.py -dAgg 
 or
 import matplotlib
 matplotlib.use('Agg')
 
Let me know how it works out! Note also that backend agg is the first
backend to support alpha blending; see scatter_demo2.py.
JDH

Peter Groszkowski wrote:
> >
> Yes I am only planning to work offline. Want to be able to pipe the 
> output images to stdout. I am looking for the fastest solution possible.
> 
Following up on this, I was curious what exactly you meant by
this. A stream of byte values in ascii separated by spaces?
Or the actual binary bytes? If the latter, it wouldn't appear
to be difficult to write a C extension to return the image
as a string, but I'm figuring there is more to it than that
since the representations for the image structure can change
from one backend to another.
Perry

John:
Thanks very much for your investigative work.
>antialiased line drawing. Without it, I can draw and save your figure
>(including python startup time, etc, etc) in 0.25s on a 2GHz Pentium
>4. Is this in the ballpark for you, performance wise?
> 
>
yes.. yes..yes..
>My guess is: when you turn off antialiasing you'll be a whole lot
>happier. Let me know.
> 
>
With antialiasing off, the performance is superb!.. I plot 500,000 
points in ~4-5 seconds.. The visual quality of the graphs is (naturally) 
inferior to the antialiased counterparts, but the software is now 
feasible for my purposes.
Just couple more questions:
1) Seems like setting 'lod' to true does not improve performance? I 
would imagine it should, because it limits the amount of points used. 
What am I missing?
2) Is there any way to make the graphs look "prettier"? They really look 
quite OK but in some cases having a little more detail would be nice. Is 
it possible specify just how much antialiasing is needed? Are there any 
other "visual enchantment options" that can be set, and will not impact 
performace too much?
3) When I do:
plot1 = plot(arange(10000), arange(20000,30000)) #dummy data.. 10,000 pairs
lod, aa = False, False
set(l, 'lod', lod, 'antialiased', aa)
Do these options only apply to the current plot (ie. plot1)?
Is it possible to have a plot inside a plot with one being antialiased, 
and the other one not?
Do I have to re-set them after I call savefig() (Will test this.. )
I have been playing around with the dpi setting a little. Is it supposed 
to change the size of the image and/or the resolution??
Thanks again.
-- 
Peter Groszkowski Gemini Observatory
Tel: +1 808 974-2509 670 N. A'ohoku Place
Fax: +1 808 935-9235 Hilo, Hawai'i 96720, USA

John Hunter writes:
> >>>>> "Perry" == Perry Greenfield <pe...@st...> writes:
> Perry> I like the sounds of this approach even more. But I wonder
> Perry> if it can be made somewhat more generic. This approach (if
> Perry> I read it correctly seems to need a backend function for
> Perry> each shape: perhaps only for circle?). What I was thinking
> Perry> was if there was a way to pass it the vectors or path for a
> Perry> symbol (for very often, many points will share the same
> Perry> shape, if not all the same x,y scale). 
> 
> Of course (slaps self on head). matplotlib 0.1 was designed around
> gtk drawing which doesn't support paths. Although I've been mumbling
> about adding paths for sometime (what with paint, ps, and agg), I'm
> still thinking inside the box. A collection of paths is the natural
> solution
> 
Based on your previous description of collection of circles, I think
so (though I wonder about the name, paths may imply many independent
paths whereas I'm implying that sharing of one path for a collection
of points. Since circles are all identical in shape, that confusion
doesn't exist with the plural.
But I can see this approach being used for things like error bars
(one can view them as scalable symbols).
> Perry> I suppose circle and other curved items could be handled
> Perry> with A bezier type call.
> 
> Agg special cases this one with a dedicated ellipse function. 
> 
> ellipse(ex, y, width, height, numsteps)
> 
> It's still a path, but you have a dedicated function to build that
> path up speedily.
> 
> One potential drawback: how do you bring along the other backends that
> don't have path support? In the RectangleCollection approach, we can
> always fall back on draw_rectangle. In the path collection, it's more
> difficult.
> 
Maybe I'm still missing something, but couldn't paths be implemented
using the backend lines primitive? After all, any path is a finite
set of lines (unless you are using bezier curves). And if lines
are available in python, the loop could also be coded in C.
Now it is true that some backends don't have the concept of
defining a path that can be reused with different coordinate 
transforms. But that isn't needed for the functionality of rendering
it, is it. It just makes it a bit more work to keep rendering the
same set of points with different offsets and scales (i.e., you 
must keep giving the transformed path array(s) to the backend to render
within a loop (in python or C). Right?
Perry
> 

>>>>> "Peter" == Peter Groszkowski <pgr...@ge...> writes:
 Peter> Although the data I'm playing with right now is monotonic
 Peter> (in x), I cannot assume that this will always be the case,
 Peter> and need an efficient solutions for all situations.
Agreed. 
 Peter> the 'lod' option in: l = plot(arange(10000),
 Peter> arange(20000,30000)) #dummy data.. 10,000 pairs set(l,
 Peter> 'lod', True) option does not work for me. It's still
 Peter> roughly 1000 points/second
I left out a *critical* detail. The new gd backend code implements
antialiased drawing by default. Very slow. Check out the numbers
below based on the demo script you supplied
 backend = 'GD'
 import matplotlib
 matplotlib.use(backend)
 from matplotlib.matlab import *
 l = plot(arange(10000), arange(20000,30000)) #dummy data.. 10,000 pairs
 lod, aa = False, False
 print 'Backend: %s, LOD %d, AA %d' % (backend, lod, aa)
 set(l, 'lod', lod, 'antialiased', aa)
 savefig('test')
 Backend: GD, LOD 1, AA 1
 23.770u 0.030s 0:23.77 100.1% 0+0k 0+0io 793pf+0w
 Backend: GD, LOD 0, AA 1
 23.500u 0.020s 0:23.52 100.0% 0+0k 0+0io 793pf+0w
 Backend: GD, LOD 1, AA 0
 0.270u 0.000s 0:00.28 96.4% 0+0k 0+0io 794pf+0w
 Backend: GD, LOD 0, AA 0
 0.240u 0.030s 0:00.27 100.0% 0+0k 0+0io 794pf+0w
In other words, if you are using the new GD in it's default
configuration, you are paying a *100 fold performance hit* for
antialiased line drawing. Without it, I can draw and save your figure
(including python startup time, etc, etc) in 0.25s on a 2GHz Pentium
4. Is this in the ballpark for you, performance wise?
While we're on the subject of performance, I took the opportunity to
test the other backends. Note the numbers are not strictly comparable
(discussed below) but are informative.
 Backend: Paint, LOD 0, AA 0
 0.520u 0.000s 0:00.52 100.0% 0+0k 0+0io 726pf+0w
 Backend: PS, LOD 0, AA 0
 1.030u 0.040s 0:01.08 99.0% 0+0k 0+0io 582pf+0w
 Backend: Agg, LOD 0, AA 0
 0.320u 0.010s 0:00.28 117.8% 0+0k 0+0io 681pf+0w
 Backend: GTK, LOD 0, AA 0
 0.650u 0.020s 0:00.66 101.5% 0+0k 0+0io 3031pf+0w
The GTK results are in xvfb so it appears to be a no-go for you even
if we could figure out how to print to stdout. These numbers are
repeatable and consistent.
Worthy of comment:
 * GD with antialiased off wins
 * paint is not as fast as I hoped
 * GTK is not as fast as I thought
 * Agg is an interesting case. It is doing antialiased drawing
 despite the AA 0 flag because I haven't made this conditional in
 the backend. It draws antialised unconditionally currently. But
 it hasn't implemented text yet. So it's not strictly comparable,
 but it is noteworthy that it is 100 times faster than GD at AA
 lines. It remains to be seen what speed we can get with plain
 vanilla aliased rendering.
My guess is: when you turn off antialiasing you'll be a whole lot
happier. Let me know.
The last thing I looked at was how the GD numbers scale with line
size. Below, N is the number of data points (with LOD false the
numbers are very close to these results where LOD is true)
 Backend: GD, LOD 1, AA 0, N 10000
 0.230u 0.040s 0:00.24 112.5% 0+0k 0+0io 794pf+0w
 Backend: GD, LOD 1, AA 0, N 20000
 0.260u 0.060s 0:00.31 103.2% 0+0k 0+0io 794pf+0w
 Backend: GD, LOD 1, AA 0, N 40000
 0.390u 0.030s 0:00.41 102.4% 0+0k 0+0io 794pf+0w
 Backend: GD, LOD 1, AA 0, N 80000
 0.590u 0.060s 0:00.60 108.3% 0+0k 0+0io 815pf+0w
 Backend: GD, LOD 1, AA 0, N 160000
 1.070u 0.090s 0:01.13 102.6% 0+0k 0+0io 818pf+0w
JDH

Thanks for the prompt answers.
>Bad terminology: for x I meant sorted (monotonic) and for y the ideal
>cases is smooth and not varying too rapidly. Try the lod feature and
>see if it works for you.
>
Although the data I'm playing with right now is monotonic (in x), I 
cannot assume that this will always be the case, and need an efficient 
solutions for all situations.
the 'lod' option in:
l = plot(arange(10000), arange(20000,30000)) #dummy data.. 10,000 pairs
set(l, 'lod', True)
option does not work for me. It's still roughly 1000 points/second
> >> econdly, the standard gdmodule will iterate over the x, y
> >> values in a python loop in gd.py. This is slow for lines with
> >> lots of points. I have a patched gdmodule that I can send you
> >> (provide platform info) that moves this step to the extension
> >> module. Potentially a very big win.
>
> Peter> Yes, that would be great! System info:
>
>Here is the link
>
>http://nitace.bsd.uchicago.edu:8080/files/share/gdmodule-0.52b.tar.gz
>
>You must also upgrade gd to 2.0.22 (alas 2.0.21 is obsolete!) since I
>needed the latest version to get this sucker ported to win32.
> 
>
Installed gd 2.0.22, and gdmodule-0.52b (from the link you provided) but 
there is no change in the times. Not sure why.. I should probably notice 
at least a little difference.
>I don't know how to write a GTK pixbuf to stdout. I inquired on the
>pygtk mailing list, so perhaps we'll learn something soon. To use GTK
>in Xvfb, make sure you have Xvfb (X virtual frame buffer) installed
>(/usr/X11R6/bin/Xvfb). There is probably an RPM, but I don't
>remember.
>
[...]
Installed Xvfb, and ran the little script you included. It complained about:
File 
"/usr/lib/python2.2/site-packages/matplotlib/backends/backend_gtk.py", 
line 528, in _quit_after_print_xvfb
 if len(manager.drawingArea._printQued): break
AttributeError: FigureManagerGTK instance has no attribute 'drawingArea'
Didn't inquire further because in my case it is crucial to have stdout 
output.. I have to be able to pipe these plots to cgi scrips.
If you have any other ideas, please let me know. Can anyone else tell me 
what kind of performance they're getting doing these 10k, 50k, 100k plots?
Best,
Peter