matplotlib-devel Mailing List for matplotlib (Page 17)

Michael Droettboom wrote:
> Here's simple_plot.ps generated with the old PS backend.
Crash! Again, with 400% magnification. So now you really are off the hook.
efiring@manini:~/programs/py/mpl/tests$ evince simple_plot_ps.ps
ERROR: /unknownerror in --%op_show_continue--
Operand stack:
Execution stack:
 %interp_exit .runexec2 --nostringval-- --nostringval-- 
--nostringval-- 2 %stopped_push --nostringval-- --nostringval-- 
 --nostringval-- false 1 %stopped_push 1 3 %oparray_pop 
1 3 %oparray_pop 1 3 %oparray_pop 1 3 %oparray_pop 
.runexec2 --nostringval-- --nostringval-- --nostringval-- 2 
%stopped_push --nostringval--
Dictionary stack:
 --dict:1126/1686(ro)(G)-- --dict:0/20(G)-- --dict:103/200(L)-- 
 --dict:6/7(L)--
Current allocation mode is local
ESP Ghostscript 815.04: Unrecoverable error, exit code 1
** (evince:7887): WARNING **: Interpreter failed.
Eric

Eric Firing wrote:
> Michael Droettboom wrote:
>> Even if my Postscript is to spec, it's not terribly useful if it 
>> crashes a very popular tool ;)
>>
>> I'm curious -- does the file fail if you set "ps.fonttype: 42" in 
>> your matplotlibrc? That would at least rule out anything non-font in 
>> the file.
I should also add that the Type 42 output is now using the ttconv code, 
not the old Python code. As a third point of comparison, it might be 
worth reverting my changes (just reverting backend_ps.py should be 
enough) and trying that. (I'll also send you a copy of simple_plot.ps I 
have lying around off-list to save you the trouble).
>>
>> I just built my own ESP Ghostscript 815.04 and am able to make some 
>> pretty high resolution renderings at the commandline. I'm not 
>> exactly sure how "400%" in the GUI relates to dpi, but I can do 1200 
>> dpi without problem. Perhaps there's something else going on in the 
>> evince stack... I'll try later on my Ubuntu box at home, since 
>> building all that on this RHEL4 box could be time consuming.
>
> If you have gv installed on your RHEL4 box, you can use that instead 
> of evince.
I don't have gv installed, but I tried ggv and kghostscript and both can 
zoom into their maximums without crashing. Both are running against 
ghostscript 7.07, however.
>>
>> Does the following work for you?
>>
>> gs -sOUTPUT_DEVICE=pngalpha -sOutputFile=test%03d.png -r1200x1200 
>> simple_plot_ps.ps
> gs -sDEVICE=pngalpha -sOutputFile=testps2.png -dNOPAUSE -r1200 -c 
> showpage -c quit testps.ps
>
> does not yield an error message. The problem seems peculiar to the 
> way gui apps are running gs, and to the mpl way of generating the 
> postscript, since it does not occur with cairo-generated postscript.
It seems that way. Now if only I knew why that was... ;)
Cheers,
Mike

Michael Droettboom wrote:
> Even if my Postscript is to spec, it's not terribly useful if it crashes 
> a very popular tool ;)
> 
> I'm curious -- does the file fail if you set "ps.fonttype: 42" in your 
> matplotlibrc? That would at least rule out anything non-font in the file.
Aha! You may be off the hook. It fails in the same way, at 400%. With 
a ps file generated either way, it starts to render the image at that 
magnification, then crashes.
efiring@manini:~/programs/py/mpl/tests$ evince testps.ps
ERROR: /unknownerror in --%op_show_continue--
Operand stack:
Execution stack:
 %interp_exit .runexec2 --nostringval-- --nostringval-- 
--nostringval-- 2 %stopped_push --nostringval-- --nostringval-- 
 --nostringval-- false 1 %stopped_push 1 3 %oparray_pop 
1 3 %oparray_pop 1 3 %oparray_pop 1 3 %oparray_pop 
.runexec2 --nostringval-- --nostringval-- --nostringval-- 2 
%stopped_push --nostringval--
Dictionary stack:
 --dict:1126/1686(ro)(G)-- --dict:0/20(G)-- --dict:103/200(L)-- 
 --dict:7/7(L)--
Current allocation mode is local
Current file position is 85042
ESP Ghostscript 815.04: Unrecoverable error, exit code 1
** (evince:7480): WARNING **: Interpreter failed.
> 
> I just built my own ESP Ghostscript 815.04 and am able to make some 
> pretty high resolution renderings at the commandline. I'm not exactly 
> sure how "400%" in the GUI relates to dpi, but I can do 1200 dpi without 
> problem. Perhaps there's something else going on in the evince 
> stack... I'll try later on my Ubuntu box at home, since building all 
> that on this RHEL4 box could be time consuming.
If you have gv installed on your RHEL4 box, you can use that instead of 
evince.
> 
> Does the following work for you?
> 
> gs -sOUTPUT_DEVICE=pngalpha -sOutputFile=test%03d.png -r1200x1200 
> simple_plot_ps.ps
 gs -sDEVICE=pngalpha -sOutputFile=testps2.png -dNOPAUSE -r1200 -c 
showpage -c quit testps.ps
does not yield an error message. The problem seems peculiar to the way 
gui apps are running gs, and to the mpl way of generating the 
postscript, since it does not occur with cairo-generated postscript.
Eric

On Mon, July 9, 2007 10:57 am, John Hunter wrote:
> I wonder if you cached some of the interval tests whether you might
> see some additional speedups. Eg, something like
> ....
You could do this, but I don't think you would even notice the difference.
Really, my impression is that the only things that really take time are
drawing a line, and maybe doing excessive amounts of
multiplication/division. It can't hurt to try, though, if there is time.
The full rectangle test would give a boost in some cases, but I think
these cases are going to be extremely rare, so I didn't bother. All x vs
time type data works fine under the simpler test, for example. I'll give
it a try sometime later this week though, just to see.
> There is one case where your culling algorithm will fail, I think
 view window x2
 -----------------------------
 | |
 | |
 | |
 _______________
x1
It doesn't fail, as you can see by trying to plot something like that.
Both points need to be on the 'undrawn' side of one of the lines of the
box. In your case, the points are to either side of the top line, the left
line and the bottom line, and are both to the 'drawn' side of the right
line. They are not both on the undrawn side of any line.
> Do you think it is worth making this a line property so users can
> tweak it? There are a couple of ways to get the line property information
> into the backend -- the typical way is to make a corresponding graphics
> context property and then set it in the line draw method.
I thought of doing this, but I didn't get to it. I originally wrote it
just for myself. If you know how, it seems like a good idea.
> I do think it would be nice to support culling points outside the
> viewport across backends. A recurring criticism is that our postscript
> files with lots of points, in which the zoom is set to just a subset of
> points, are quite large because we use graphics clipping to get rid of
> them but the data are still in the file. As you note, this can be a good
> thing for users who want to do post-hoc editing, and culling is tricky
> because, for example, a point whose center is outside the viewport could
> have a edge vertex inside the viewport. So we would need polygon
> intersections to do it right (and even this would not completely solve the
> case of polygons with thick edge strokes) but having the option to cull
> points whose centers fell outside the viewport, or line segments whose
> vertices fell outside the x and y intervals, would be a big win in most
> common use cases.
Dropping parallel lines in the visible area still seems like a bad idea
for SVG, but I can see that culling points outside it could be OK. There
are two things you want to cull: lines between two points, and polygons
drawn around single points.
I don't know exactly how the polygon parts work in matplotlib, but it
seems reasonable that you could find an upper value to the radius of the
polygon around a point. Add max radius of polygon + stroke width to get a
'bound' value, and then instead of (x < 0 || x > width ...) do (x < -bound
|| x > width + bound ...) to cull. Just make the bound big enough that you
are sure nothing could be in the viewing area, even if you draw a few more
points than you need to.
For lines between two points, if you use just the simple cull test I am
using, you could do the same thing as for polygons where the bound depends
only on the stroke width. Actually, I should really do that in the agg
code. If you use a full box test, you would again need a more complicated
algorithm with extra intersection tests, it could get messy.
Also, I am not sure how and when transforms are applied to the data, but I
can imagine the transform and culling interfering with each other. In the
agg code, the data is transformed to its final pixel units before I do
culling.
Allan

Even if my Postscript is to spec, it's not terribly useful if it crashes 
a very popular tool ;)
I'm curious -- does the file fail if you set "ps.fonttype: 42" in your 
matplotlibrc? That would at least rule out anything non-font in the file.
I just built my own ESP Ghostscript 815.04 and am able to make some 
pretty high resolution renderings at the commandline. I'm not exactly 
sure how "400%" in the GUI relates to dpi, but I can do 1200 dpi without 
problem. Perhaps there's something else going on in the evince 
stack... I'll try later on my Ubuntu box at home, since building all 
that on this RHEL4 box could be time consuming.
Does the following work for you?
 gs -sOUTPUT_DEVICE=pngalpha -sOutputFile=test%03d.png -r1200x1200 
simple_plot_ps.ps
Cheers,
Mike
Eric Firing wrote:
> Michael Droettboom wrote:
>> Oh dear. It works for me with my fairly old version of GNU 
>> ghostscript (7.07) and a couple different Postscript HP laser 
>> printers I tried. The file position where the error occurs at least 
>> seems plausible -- it's where the first text is shown.
>>
>> I'll have to look into this further. More data points are welcome.
>>
>> Cheers,
>> Mike
>
> Mike,
>
> Attached are a minimal script to reproduce the problem, and the ps 
> file I get that shows the problem--but only when I try to display at 
> sufficiently high resolution. Threshold seems to be between 200% and 
> 400% on my machine, but I have no idea why there is a dependence on 
> magnification at all.
>
> With this simpler example, the error output is:
>
> ERROR: /unknownerror in --%op_show_continue--
> Operand stack:
>
> Execution stack:
> %interp_exit .runexec2 --nostringval-- --nostringval-- 
> --nostringval-- 2 %stopped_push --nostringval-- 
> --nostringval-- --nostringval-- false 1 %stopped_push 1 
> 3 %oparray_pop 1 3 %oparray_pop 1 3 %oparray_pop 1 3 
> %oparray_pop .runexec2 --nostringval-- --nostringval-- 
> --nostringval-- 2 %stopped_push --nostringval--
> Dictionary stack:
> --dict:1126/1686(ro)(G)-- --dict:0/20(G)-- --dict:103/200(L)-- 
> --dict:7/7(L)--
> Current allocation mode is local
> Current file position is 5228
> ESP Ghostscript 815.04: Unrecoverable error, exit code 1
>
> Now, if I do
>
> ps2pdf testps.ps
>
> The resulting pdf file works fine; evince displays it at 400%.
> If I then do
>
> pdftops testps.pdf testpsrt.ps
>
> the new ps file resulting from this round trip *fails* in evince at 
> 400% in the same way:
>
> evince testpsrt.ps
> ERROR: /unknownerror in --%op_show_continue--
> Operand stack:
> 80.9999 219.2 0.0 0.0 32 6.93003e-05 0.0 (0.0)
> Execution stack:
> %interp_exit .runexec2 --nostringval-- --nostringval-- 
> --nostringval-- 2 %stopped_push --nostringval-- 
> --nostringval-- --nostringval-- false 1 %stopped_push 1 
> 3 %oparray_pop 1 3 %oparray_pop 1 3 %oparray_pop 1 3 
> %oparray_pop .runexec2 --nostringval-- --nostringval-- 
> --nostringval-- 2 %stopped_push --nostringval-- --nostringval--
> Dictionary stack:
> --dict:1126/1686(ro)(G)-- --dict:0/20(G)-- --dict:103/200(L)-- 
> --dict:63/75(L)-- --dict:18/24(L)-- --dict:0/15(L)-- 
> --dict:0/15(L)-- --dict:8/15(L)-- --dict:2/15(L)--
> Current allocation mode is local
> ESP Ghostscript 815.04: Unrecoverable error, exit code 1
>
> ** (evince:6215): WARNING **: Interpreter failed.
>
>
> On the other hand, running testps.py -dcairo.ps results in a ps file 
> which does *not* fail in evince at 400%.
>
>
> Eric
> ------------------------------------------------------------------------
>
> import pylab
> pylab.plot([1,2,3], '.')
> pylab.savefig('testps.ps')
> 

Oh dear. It works for me with my fairly old version of GNU ghostscript 
(7.07) and a couple different Postscript HP laser printers I tried. The 
file position where the error occurs at least seems plausible -- it's 
where the first text is shown.
I'll have to look into this further. More data points are welcome.
Cheers,
Mike
Eric Firing wrote:
> Michael Droettboom wrote:
> [...]
>> All should be in SVN.
>
> Mike,
>
> Now it compiles and runs all the demos, and displays OK at normal 
> resolution; but when I try to show it with gv or evince at 400%, gs 
> crashes:
> efiring@manini:~/programs/py/mpl/matplotlib_units/examples$ evince 
> simple_plot_ps.ps
> ERROR: /unknownerror in --%op_show_continue--
> Operand stack:
>
> Execution stack:
> %interp_exit .runexec2 --nostringval-- --nostringval-- 
> --nostringval-- 2 %stopped_push --nostringval-- 
> --nostringval-- --nostringval-- false 1 %stopped_push 1 
> 3 %oparray_pop 1 3 %oparray_pop 1 3 %oparray_pop 1 3 
> %oparray_pop .runexec2 --nostringval-- --nostringval-- 
> --nostringval-- 2 %stopped_push --nostringval--
> Dictionary stack:
> --dict:1126/1686(ro)(G)-- --dict:0/20(G)-- --dict:103/200(L)-- 
> --dict:6/7(L)--
> Current allocation mode is local
> Current file position is 16588
> ESP Ghostscript 815.04: Unrecoverable error, exit code 1
>
> ** (evince:5270): WARNING **: Interpreter failed.
>
> Eric
>
>
>>
>> Cheers,
>> Mike
>>
>
>

Michael Droettboom wrote:
[...]
> All should be in SVN.
Mike,
Now it compiles and runs all the demos, and displays OK at normal 
resolution; but when I try to show it with gv or evince at 400%, gs crashes:
efiring@manini:~/programs/py/mpl/matplotlib_units/examples$ evince 
simple_plot_ps.ps
ERROR: /unknownerror in --%op_show_continue--
Operand stack:
Execution stack:
 %interp_exit .runexec2 --nostringval-- --nostringval-- 
--nostringval-- 2 %stopped_push --nostringval-- --nostringval-- 
 --nostringval-- false 1 %stopped_push 1 3 %oparray_pop 
1 3 %oparray_pop 1 3 %oparray_pop 1 3 %oparray_pop 
.runexec2 --nostringval-- --nostringval-- --nostringval-- 2 
%stopped_push --nostringval--
Dictionary stack:
 --dict:1126/1686(ro)(G)-- --dict:0/20(G)-- --dict:103/200(L)-- 
 --dict:6/7(L)--
Current allocation mode is local
Current file position is 16588
ESP Ghostscript 815.04: Unrecoverable error, exit code 1
** (evince:5270): WARNING **: Interpreter failed.
Eric
> 
> Cheers,
> Mike
> 

On 7/9/07, Paul Kienzle <pki...@ni...> wrote:
> This change works for me under wx os OS X.
> I got the gui_paint idea from examples/dynamic_demo_wx.py. Replacing
> gui_paint with draw_idle seems to work okay there as well. I'll
> create a patch. Do you small patches like this on tracker as well?
Sorry if I wasn't clear, I already made the change when I committed
it, so all is well.
As for patches, small patches are fine just sent to the site. If you
are worried about them getting lost, post them to the sf site.
But if you send me a sf id, I'll add you to the developers list and
you can commit directly. You may want to check out the CODING_GUIDE i
the svn repository.
Thanks,
JDH

On Mon, Jul 09, 2007 at 10:37:35AM -0500, John Hunter wrote:
> One minor bug I fixed for non wx backends in the (very cool)
> figure.canvas.mpl_connect("motion_notify_event",figure.canvas.hilite)
> functionality. gui_repaint is a wx only method, and I just commented
> it out and replaced the "draw" with a draw_idle. This should work
> across backends. Note draw_idle by default in backend_bases just
> calls draw, but backends can override it to use their GUIs idle
> handler. This is usually a good idea so you don't get an event pile
> up. In particular, it is not implemented for backend_wx, so one of
> you wx gurus/users may want to consider adding it, because for
> interactive stuff like motion events, it can be a big win.
This change works for me under wx os OS X.
I got the gui_paint idea from examples/dynamic_demo_wx.py. Replacing
gui_paint with draw_idle seems to work okay there as well. I'll
create a patch. Do you small patches like this on tracker as well?
	- Paul

Eric Firing wrote:
> Michael Droettboom wrote:
>> Sorry about that. I didn't notice the SVN login error the first 
>> time. It should be in SVN now.
>> The default is to output Type 3 fonts (i.e. the new way).
>>
>> Darren: you mean rcdefaults.py, not rcsetup.py, right? I can't find 
>> an rcsetup.py.
>
> Mike,
>
> I renamed rcdefaults.py to rcsetup.py a few days ago.
I have moved my new rcdefaults.py stuff into rcsetup.py.
> After svn update right now (to 3490), I am getting a compilation 
> failure with the following error:
Ah. Forgot to explicitly include <assert.h> (which older gcc's don't 
complain about).
All should be in SVN.
Cheers,
Mike

Michael Droettboom wrote:
> Sorry about that. I didn't notice the SVN login error the first time. 
> It should be in SVN now. 
> 
> The default is to output Type 3 fonts (i.e. the new way).
> 
> Darren: you mean rcdefaults.py, not rcsetup.py, right? I can't find an 
> rcsetup.py.
Mike,
I renamed rcdefaults.py to rcsetup.py a few days ago. Try "svn 
update". Maybe that will solve your quiver_demo.py problem also; it has 
been working fine for me.
In general, I find that while working on mpl it is helpful to run svn 
update frequently, and always right before committing, so as to avoid 
conflicts and be sure that changes I make work with the latest changes 
others have made.
After svn update right now (to 3490), I am getting a compilation failure 
with the following error:
building 'matplotlib.ttf2ps' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O2 -Wall 
-Wstrict-prototypes -fPIC -I/usr/local/include -I/usr/include -I. 
-I/usr/include/python2.5 -c ttconv/pprdrv_tt.cpp -o 
build/temp.linux-i686-2.5/ttconv/pprdrv_tt.o
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for 
Ada/C/ObjC but not for C++
ttconv/pprdrv_tt.cpp: In function ‘void 
sfnts_glyf_table(TTStreamWriter&, TTFONT*, unsigned int, unsigned int)’:
ttconv/pprdrv_tt.cpp:579: error: ‘assert’ was not declared in this scope
ttconv/pprdrv_tt.cpp: In function ‘void insert_ttfont(const char*, 
TTStreamWriter&, int, std::vector<int, std::allocator<int> >&)’:
ttconv/pprdrv_tt.cpp:1104: error: ‘assert’ was not declared in this scope
error: command 'gcc' failed with exit status 1
Eric

On 7/9/07, Michael Droettboom <md...@st...> wrote:
> Sorry about that. I didn't notice the SVN login error the first time.
> It should be in SVN now.
>
> The default is to output Type 3 fonts (i.e. the new way).
>
> Darren: you mean rcdefaults.py, not rcsetup.py, right? I can't find an
> rcsetup.py.
Something seems to be missing in the rcsetup
johnh@flag:examples> rm -rf ~/.matplotlib/ttffont.cache
johnh@flag:examples> PYTHONPATH=~/dev/lib/python2.4/site-packages/
python simple_plot.py -dPS
Traceback (most recent call last):
 File "simple_plot.py", line 18, in ?
 savefig('simple_plot')
 File "/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/pylab.py",
line 802, in savefig
 return fig.savefig(*args, **kwargs)
 File "/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/figure.py",
line 769, in savefig
 self.canvas.print_figure(*args, **kwargs)
 File "/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/backends/backend_ps.py",
line 1039, in print_figure
 convert_ttf_to_ps(font_filename, fh, rcParams['ps.fonttype'], glyph_ids)
KeyError: 'ps.fonttype'
johnh@flag:examples>

Sorry about that. I didn't notice the SVN login error the first time. 
It should be in SVN now. 
The default is to output Type 3 fonts (i.e. the new way).
Darren: you mean rcdefaults.py, not rcsetup.py, right? I can't find an 
rcsetup.py.
Cheers,
Mike
John Hunter wrote:
> On 7/9/07, Darren Dale <dd...@co...> wrote:
>
> 
>> Could you be more specific about how to test your changes? I dont see anything
>> in matplotlibrc.template, or in rcsetup.py, concerning the valid settings of
>> ps.fonttype. (rc settings should be validated, have a default value, and have
>> a commented listing in matplotlibrc.template.)
>> 
>
> Also, when I run an older mpl or svn mpl, using simple_demo.py, I get
> a font header that looks like the one below. It looks like I am not
> getting the new stuff either...
>
> %%BeginFont: BitstreamVeraSerif-Roman
> %!PS-TrueType-1.0-2.0
> 8 dict begin
> /FontName /BitstreamVeraSerif-Roman def
> /FontMatrix [1 0 0 1 0 0] def
> /FontType 42 def
> /Encoding StandardEncoding def
> /FontBBox [-375 -483 2636 1901] def
> /PaintType 0 def
> /FontInfo 7 dict dup begin
> /Notice (Copyright (c) 2003 by Bitstream, Inc. All Rights Reserved.) def
> /FamilyName (Bitstream Vera Serif) def
> /FullName (Bitstream Vera Serif) def
> /version (Release 1.10) def
> /isFixedPitch false def
> /UnderlinePosition -213 def
> /UnderlineThickness 133 def
> end readonly def
> /sfnts [
> <0001000000110100000400104F532F32B658F2650000D55C0000005650434C54798E6268
> 0000D5B400000036636D6170A4C3E8A00000B58800000358637674208F33ABF400001F04
> 000001946670676DE780F1C4000025F80000008B67617370000700070000EB340000000C
> 676C796627203BE80000268400008F0268646D78BBC32F430000D5EC0000154868656164
> B5279A060000EB4000000036686865611045086F0000D53800000024686D747831C577F3
> 0000B8E0000004306B65726ECF55CE200000C1BC0000135C6C6F6361E324C0E00000BFA0
> 0000021A6D617870053D063B0000D518000000206E616D65D4B5E8800000011C00001DE5
> 706F7374B45A2FB10000BD100000028E70726570757906F6000020980000055E00000016
> 010E0001000000000000003A000000010000000000010014003A00010000000000020005
> 006100010000000000030014003A00010000000000040014003A0001000000000005000C
> 006600010000000000060018004E0001000000000007003000AF0001000000000008000E
> 086E000100000000000B00180985000100000000000D0913007200030001040900000074
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
> 

On 7/9/07, Darren Dale <dd...@co...> wrote:
> Could you be more specific about how to test your changes? I dont see anything
> in matplotlibrc.template, or in rcsetup.py, concerning the valid settings of
> ps.fonttype. (rc settings should be validated, have a default value, and have
> a commented listing in matplotlibrc.template.)
Also, when I run an older mpl or svn mpl, using simple_demo.py, I get
a font header that looks like the one below. It looks like I am not
getting the new stuff either...
%%BeginFont: BitstreamVeraSerif-Roman
%!PS-TrueType-1.0-2.0
8 dict begin
/FontName /BitstreamVeraSerif-Roman def
/FontMatrix [1 0 0 1 0 0] def
/FontType 42 def
/Encoding StandardEncoding def
/FontBBox [-375 -483 2636 1901] def
/PaintType 0 def
/FontInfo 7 dict dup begin
/Notice (Copyright (c) 2003 by Bitstream, Inc. All Rights Reserved.) def
/FamilyName (Bitstream Vera Serif) def
/FullName (Bitstream Vera Serif) def
/version (Release 1.10) def
/isFixedPitch false def
/UnderlinePosition -213 def
/UnderlineThickness 133 def
end readonly def
/sfnts [
<0001000000110100000400104F532F32B658F2650000D55C0000005650434C54798E6268
0000D5B400000036636D6170A4C3E8A00000B58800000358637674208F33ABF400001F04
000001946670676DE780F1C4000025F80000008B67617370000700070000EB340000000C
676C796627203BE80000268400008F0268646D78BBC32F430000D5EC0000154868656164
B5279A060000EB4000000036686865611045086F0000D53800000024686D747831C577F3
0000B8E0000004306B65726ECF55CE200000C1BC0000135C6C6F6361E324C0E00000BFA0
0000021A6D617870053D063B0000D518000000206E616D65D4B5E8800000011C00001DE5
706F7374B45A2FB10000BD100000028E70726570757906F6000020980000055E00000016
010E0001000000000000003A000000010000000000010014003A00010000000000020005
006100010000000000030014003A00010000000000040014003A0001000000000005000C
006600010000000000060018004E0001000000000007003000AF0001000000000008000E
086E000100000000000B00180985000100000000000D0913007200030001040900000074

On Monday 09 July 2007 11:25:57 am Michael Droettboom wrote:
> An initial revision of TTF font subsetting has been committed to SVN.
Thank you for doing this.
[...]
> ttconv supports outputting a Type 42 font (without subsetting), so
> matplotlib's old Type 42 output has been replaced with ttconv's. An rc
> value (ps.fonttype) can be used to select Type 3 or Type 42 output.
[...]
> Please kick the tires and let me know how it goes,
Could you be more specific about how to test your changes? I dont see anything 
in matplotlibrc.template, or in rcsetup.py, concerning the valid settings of 
ps.fonttype. (rc settings should be validated, have a default value, and have 
a commented listing in matplotlibrc.template.)
Thanks,
Darren

On 7/9/07, Paul Kienzle <pki...@ni...> wrote:
> On Sun, Jul 08, 2007 at 03:04:41AM -0400, Paul Kienzle wrote:
> > I submitted the 'contains' patch to the patch tracker on sourceforge.
>
> I've updated the 'contains' patch to handle figure enter/leave, at
> least on wx backends, and fixed a minor hit test bug in ellipse.
> I've also removed the special status of the pick event notifier,
> using the mpl_connect event processing framework to handle it.
>
Hey Paul, this is a very impressive piece of work. You weren't
kidding when you said you had a lot of work ahead of you in your first
post on the subject. The recent pick framework was a significant
improvement on what we had before it, which was barely usable, but
your stuff is going to make it *much* better. It is impressive that
your code doesn't break the existing API, but don't kill yourself over
maintaining API compatibiltiy. The pick_even was only added in the
0.90 release, and is still very lightly used. I would like to
preserve most of the functionality that is there (eg custom hit
testing functions, as you have) but if we need to tweak the API that
will be fine.
When you finish the architectural changes, I hope you'll provide us
with a nice example or two, and or modify the existing pick_event_demo
and data_browser code to use the cleaner API , eg registering with the
artist directly).
One minor bug I fixed for non wx backends in the (very cool)
figure.canvas.mpl_connect("motion_notify_event",figure.canvas.hilite)
functionality. gui_repaint is a wx only method, and I just commented
it out and replaced the "draw" with a draw_idle. This should work
across backends. Note draw_idle by default in backend_bases just
calls draw, but backends can override it to use their GUIs idle
handler. This is usually a good idea so you don't get an event pile
up. In particular, it is not implemented for backend_wx, so one of
you wx gurus/users may want to consider adding it, because for
interactive stuff like motion events, it can be a big win.
JDH

An initial revision of TTF font subsetting has been committed to SVN. 
It seems to work for all tests run by backend_driver.py (with the 
exception quiver_demo, which didn't seem to work without my changes either).
Some key results:
fonts_demo_kq_ps.py: 633518 -> 45232
mathtext_demo.py: 366268 -> 34932
unicode_demo.py: 92713 -> 27281
Over all the backend_driver demos, the size goes from 44,428,655 to 
35,925,889.
The original ttconv code has been modified to be exception safe, since 
its error handling as a commandline app was to just quit the process. 
The easiest way to do this was to use C++ exceptions and add some 
destructors to various objects. It makes for a strange mix of C and 
C++, which hopefully isn't too offensive. I was more concerned with 
making the minimum number of changes to make it work in the new 
environment, rather than making stylistic ones for the sake of 
cleanliness etc.
ttconv supports outputting a Type 42 font (without subsetting), so 
matplotlib's old Type 42 output has been replaced with ttconv's. An rc 
value (ps.fonttype) can be used to select Type 3 or Type 42 output. 
This will probably go away (with Type 3 as the only option) in the 
future, but I thought I'd leave it in for testing purposes for now.
(I have been unable to test tex_unicode_demo.py, as my TeX installation 
doesn't seem to have all the necessary stuff (ucs.py) and installing 
that without root access is proving to be difficult... I suspect my 
changes will not affect this demo anyway, since use_tex uses a different 
method to embed the glyph outlines in the file.)
Please kick the tires and let me know how it goes,
Mike

On 7/9/07, Michael Droettboom <md...@st...> wrote:
>
> I am about to update the memory leak question in the FAQ, but I thought I'd
> run it by the list first. I removed language that talked about much earlier
> releases of mpl, and the paragraph about leaks in older versions of Numeric
> and numarray. It seems like we should recommend numpy when the user
> experiences problems with Numeric or numarray, but I wanted to confirm that
> before adding a paragraph to that effect.
vis-a-vis numpy, since we are dropping support entirely for Numeric
and numarray, you can definitely recommend numoy :-)
The looks very good, but I would also point them to memleak_hawaii or
something to diagnose a pure image backend. Many times it is simply
the user script that is failing (they fail to properly close a figure
they have opened) so the FAQ should at least point them to an image
generation script that is written correctly. The GUI leaks are are
current problem, but they are 2nd order. We first need to make sure
the pure MPL part is not leaking (it isn't currently, but it might one
day if we screw something up in extension code) and once we've
verified that, we can isolate the GUI leaks.
Thanks,
JDH

On 7/8/07, ah...@cs... <ah...@cs...> wrote:
> First, I noticed a bug in the version I sent before. I've attached a new
> version in patch form, to be applied from the base directory of a normal
> 0.90.1 installation with 'patch -p1 <patch'.
Thanks Allan, I just applied this to svn. Some time ago I made an
optimization to Agg's draw_markers to used cached rasters of the
markers and this made a dramatic difference in speed. So it is nice
to complement that with this draw_lines optimization.
I was curious to see what approach you took to cull the points outside
the window -- just looking at the vertices of the current point and
last point and dropping if both are outside the x and y interval of
the viewport. As you mention in the comment, we could do a line
segment/rectangle intersection test to cull more, but it is not clear
that that would be faster because of the tradeoff between the cost of
the test and cost of drawing segments we don't need.
I wonder if you cached some of the interval tests whether you might
see some additional speedups. Eg, something like
thisoutsidex = (thisx<0 || thisx>width)
thisoutsidey = (thisy<0 || thisy>height)
if (thisoutsidex && thisoutsidey && lastoutsidex && lastoutsidey) {
 // other stuff as necessary...
 lastoutsidex = thisoutsidex
 lastoutsidey = thisoutsidey
 continue;
}
lastoutsidex = thisoutsidex
lastoutsidey = thisoutsidey
I suspect the comparisons are pretty cheap so I don't know how much
this will save but it might be worth looking into.
There is one case where your culling algorithm will fail, I think
 view window x2
 -----------------------------
 | |
 | |
 | |
 _______________
x1
The segment connecting x1 and x2 will be dropped, but the user should
see a line here, right? If we could do a rectangle/line intersection
test w/o giving up too much in terms of performance, that would be
better I think. That or give users the ability to turn it off. Since
people use mpl to view their scientific data, we should be careful to
make sure they can see it....
> One artifact I can clearly see is there is partly due to skipping lines <
> 1 pixel in length. The original agg renderer did this too, so it is not
> too different from before. My patch makes it worse, though. You can see it
> with plot(rand(10000)) again, but zoom out on only the y axis until the
> data is almost a straight line from right to left. You should see little
> clumps appear as it gets close to a straight line.
>
> Besides this I have not noticed anything. There may be problem cases I
> have not thought of, of course. To see the kind of problem that might
> occur, you can try changing the distance cutoff from 0.25 to something
> huge like 25.0. There you will definitely notice errors.
Do you think it is worth making this a line property so users can
tweak it? There are a couple of ways to get the line property
information into the backend -- the typical way is
to make a corresponding graphics context property and then set it in
the line draw method.
> About doing this for other image formats: I think it is OK for pixel-based
> images, but probably not for SVG or vector-based images. I can imagine
> someone writing to an SVG so that they can use another program to make a
> pretty rendering of it, but if lines are missing this second program does
> not have the full data to work with.
I do think it would be nice to support culling points outside the
viewport across backends. A recurring criticism is that our
postscript files with lots of points, in which the zoom is set to just
a subset of points, are quite large because we use graphics clipping
to get rid of them but the data are still in the file. As you note,
this can be a good thing for users who want to do post-hoc editing,
and culling is tricky because, for example, a point whose center is
outside the viewport could have a edge vertex inside the viewport. So
we would need polygon intersections to do it right (and even this
would not completely solve the case of polygons with thick edge
strokes) but having the option to cull points whose centers fell
outside the viewport, or line segments whose vertices fell outside the
x and y intervals, would be a big win in most common use cases.
Since we already support masks in the line class, probably the best
way to do this is would be some special purpose (possibly extension)
code that efficiently sets the mask. Then we would automatically get
support across backends.
JDH

I am about to update the memory leak question in the FAQ, but I thought 
I'd run it by the list first. I removed language that talked about much 
earlier releases of mpl, and the paragraph about leaks in older versions 
of Numeric and numarray. It seems like we should recommend numpy when 
the user experiences problems with Numeric or numarray, but I wanted to 
confirm that before adding a paragraph to that effect.
The second question (below), can be ommitted -- it really belongs in a 
developer's FAQ. Any recommendations on a better place to post that 
information?
<snip>
 matplotlib appears to be leaking memory, what should I do?
First, determine if it is a true memory leak. Python allocates memory in 
pools, rather than one object at a time, which can make memory leaks 
difficult to diagnose. Memory usage may appear to increase rapidly with 
each iteration, but should eventually reach a steady state if no true 
memory leaks exist. (For more fine-grained memory usage reporting, you 
can build a custom Python with the --without-pymalloc flag, which 
disables pool allocation.) If after sufficient iterations, you still see 
that memory usage is increasing, it is a likely a bonafide memory leak 
that should be reported.
memleak_gui.py 
<http://matplotlib.svn.sourceforge.net/viewvc/matplotlib/trunk/matplotlib/unit/memleak_gui.py?view=markup> 
(in the unit directory of the source tree), contains an example script 
useful for diagnosing and reporting memory leaks. Please use something 
like it when reporting leaks so we get an idea of the magnitude of the 
problem (i.e. bytes per figure). Also please provide your platform, 
matplotlib version, backend and as much information about the versions 
of the libraries you are using: freetype, png and zlib. It would also 
help if you posted code so I could look for any obvious coding errors 
vis-a-vis matplotlib.
There are some known memory leaks in matplotlib-0.90.1 when used in 
conjunction with the Tk, Gtk, Wx, Qt and Qt4 GUI backends. Many of these 
leaks have resolutions in the current SVN version of matplotlib. 
However, the following library versions are known to have leaks that 
matplotlib triggers. If you have one of these versions and are 
experiencing memory leaks, you should upgrade your library. This is not 
an exhaustive list.
 * *Wx backend:*wxPython-2.8.2 or earlier in the 2.8 series
 * *Gtk backend:*pygobject-2.12.x, pygtk-2.4.0
 I'd like to help diagnose a memory leak, rather than just report
 it. How do you recommend I do that?
I thought you'd never ask! Python memory leaks tend to fall into one of 
the following categories:
 * *Uncollectable garbage:* The Python garbage collector is not
 infallible. It can not collect objects that the Python that have
 __del__ methods or weakrefs and contain cycles. If curious, you
 can read about this problem in horrific detail
 <http://svn.python.org/view/python/trunk/Modules/gc_weakref.txt?view=markup>.
 You can obtain a list of all uncollectable objects as follows:
 import gc
 print gc.garbage
 
 To see what cycles these objects participate in, there is a useful
 function in matplotlib.cbook:
 from matplotlib import cbook
 		cbook.print_cycles(gc.garbage)
 
 This will print out all of the reference cycles that are
 preventing the uncollectable objects from being freed. The code
 should then be modified to prevent these cycles, or break the
 cycles during destruction.
 * *Real references:* Sometimes objects are legitimately being held
 onto by other Python objects. When this happens, you would see the
 total number of Python objects in the interpreter increase with
 each iteration of your test, even when you didn't intend any data
 to "stick around". You can print out the total number of objects
 in the interpreter:
 import gc
 		print len(gc.get_objects())
 
 By comparing the objects before and after your test, you can
 determine which of them remain between iterations:
 original_objects = [id(x) for x in gc.get_objects()]
 		# ... do something that leaks objects
 		new_objects = [x for x in gc.get_objects() 
 		 if id(x) not in original_objects]
 
 You can then determine what is referencing those objects and
 causing them to stick around:
 print gc.get_referents(x)
 
 The code should be modified to prevent these unwanted references.
 * *C/C++ leaks in extension objects:* These is the classic problem
 of objects that are "malloc'd/new'd" and never "freed/deleted" in
 C or C++ parlance. There are many memory debuggers available such
 as Purify or Valgrind to help find these errors. I recommend
 reading the documentation about using these tools in conjunction
 with Python
 <http://svn.python.org/view/python/trunk/Misc/README.valgrind?view=markup>
 to avoid a lot of false positives.

On Sun, Jul 08, 2007 at 03:04:41AM -0400, Paul Kienzle wrote:
> I submitted the 'contains' patch to the patch tracker on sourceforge.
I've updated the 'contains' patch to handle figure enter/leave, at
least on wx backends, and fixed a minor hit test bug in ellipse. 
I've also removed the special status of the pick event notifier, 
using the mpl_connect event processing framework to handle it.
You can attach the enter/leave artist highlighting demo to a 
figure using:
 figure.canvas.mpl_connect("motion_notify_event",figure.canvas.hilite)
You can disable pick on a figure using:
 fig.canvas.mpl_disconnect(fig.canvas.button_pick_id)
 fig.canvas.mpl_disconnect(fig.canvas.scroll_pick_id)
Now I can work on an event handling framework which connects 
directly to the artists rather than filtering all callbacks to
the same pick function. I'll mostly follow tk canvas handling 
conventions unless anyone has suggests for a better approach.
 - Paul

On 7/9/07, Michael Droettboom <md...@st...> wrote:
> ...however, there are only about 40 patches between 2.12.3 and 2.13.2.
> Two of them are very crucial memory leak patches:
> When I patched 2.12.3 against these patches, the Python object leaks go
> away. There may be workarounds matplotlib can make -- I'll look into
> that now. Hopefully the workarounds won't be too problematic -- I would
> hate to complicate matplotlib code for the sake of something that should
> resolve itself in the next Gnome release cycle...
I wouldn't spend any time on working around a leak that is already
fixed in gobject. We just need to update the FAQ so GTK users will
know that if they have a problem with leaks in GTK*, they need to
upgrade.
JDH

Eric,
It turns out that the key difference is in pygobject. I was using 
2.13.2 (from source) here at work, and Ubuntu Feisty currently ships 
2.12.3. 2.13.x is an unstable branch, of course, (gnome-related 
packages follow the even/odd versioning scheme), so I shouldn't have 
been using it to test with in the first place...
...however, there are only about 40 patches between 2.12.3 and 2.13.2. 
Two of them are very crucial memory leak patches:
 
http://svn.gnome.org/viewcvs/pygobject?limit_changes=100&view=revision&revision=653
 
http://svn.gnome.org/viewcvs/pygobject?limit_changes=100&view=revision&revision=642
When I patched 2.12.3 against these patches, the Python object leaks go 
away. There may be workarounds matplotlib can make -- I'll look into 
that now. Hopefully the workarounds won't be too problematic -- I would 
hate to complicate matplotlib code for the sake of something that should 
resolve itself in the next Gnome release cycle...
I updated the memleak_gui.py script to display the pygobject/glib 
versions as well when testing the gtk backend.
Cheers,
Mike
Michael Droettboom wrote:
> Eric Firing wrote:
> 
>> Michael Droettboom wrote:
>> 
>>> Interesting...
>>>
>>> When you get a chance, would you mind running the attached script? 
>>> This is how I was finding object leaks before. It takes a single 
>>> commandline argument that is the number of iterations. Can you send 
>>> me the outputs from 1 and 2 iterations? That way we should be able 
>>> to see what type of object is being leaked, which is a good first step.
>>> 
>> And here is the result of the script modified for gtk:
>>
>> efiring@manini:~/programs/py/mpl/tests$ python memleak_gui_gtk.py 1
>> 55352 55417
>> *** <type 'gtk.gdk.Colormap'>
>> *** <class 'matplotlib.backends.backend_gtk.FileChooserDialog'>
>> *** <class 'gtk._gtk.WidgetFlags'>
>> *** <class 'gtk._gtk.WidgetFlags'>
>> *** <type 'gtk.gdk.Window'>
>> *** <class 'matplotlib.backends.backend_gtk.FigureCanvasGTK'>
>> *** <class 'matplotlib.backends.backend_gtk.NavigationToolbar2GTK'>
>> *** <type 'gtk.gdk.Pixmap'>
>> *** <type 'gtk.Tooltips'>
>> *** <type 'gtk.Label'>
>> *** <type 'gtk.Window'>
>> *** <type 'gtk.VBox'>
>>
>> uncollectable list: []
>>
>> efiring@manini:~/programs/py/mpl/tests$ python memleak_gui_gtk.py 2
>> 55352 55421
>> *** <type 'gtk.gdk.Colormap'>
>> *** <class 'matplotlib.backends.backend_gtk.FileChooserDialog'>
>> *** <class 'gtk._gtk.WidgetFlags'>
>> *** <class 'gtk._gtk.WidgetFlags'>
>> *** <type 'gtk.gdk.Window'>
>> *** <class 'matplotlib.backends.backend_gtk.FileChooserDialog'>
>> *** <class 'gtk._gtk.WidgetFlags'>
>> *** <class 'gtk._gtk.WidgetFlags'>
>> *** <type 'gtk.gdk.Window'>
>> *** <class 'matplotlib.backends.backend_gtk.FigureCanvasGTK'>
>> *** <class 'matplotlib.backends.backend_gtk.NavigationToolbar2GTK'>
>> *** <type 'gtk.gdk.Pixmap'>
>> *** <type 'gtk.Tooltips'>
>> *** <type 'gtk.Label'>
>> *** <type 'gtk.Window'>
>> *** <type 'gtk.VBox'>
>>
>> uncollectable list: []
>>
>> Eric
>>
>> 
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
> 

Michael Droettboom wrote:
> You're right: my bad. Should be fixed in r3461.
> 
I meant to type r3484.
> Cheers,
> Mike
>
> Eric Firing wrote:
> 
>> Mike,
>>
>> When I try to save a file as postscript using the FileChooserDialog 
>> with GtkAgg, I get:
>>
>> efiring@manini:~/programs/py/mpl/matplotlib_units/examples$ python 
>> quadmesh_demo.py
>> Traceback (most recent call last):
>> File 
>> "/usr/local/lib/python2.5/site-packages/matplotlib/backends/backend_gtk.py", 
>> line 696, in save_figure
>> fname = self.fileselect.get_filename_from_user()
>> File 
>> "/usr/local/lib/python2.5/site-packages/matplotlib/backends/backend_gtk.py", 
>> line 1095, in get_filename_from_user
>> menu_ext = IMAGE_FORMAT[self.cbox.get_active()]
>> AttributeError: 'FileChooserDialog' object has no attribute 'cbox'
>>
>> I haven't checked, but I suspect this may be a bug introduced during 
>> your recent changes.
>>
>> Eric
>> 
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
> 

You're right: my bad. Should be fixed in r3461.
Cheers,
Mike
Eric Firing wrote:
> Mike,
>
> When I try to save a file as postscript using the FileChooserDialog 
> with GtkAgg, I get:
>
> efiring@manini:~/programs/py/mpl/matplotlib_units/examples$ python 
> quadmesh_demo.py
> Traceback (most recent call last):
> File 
> "/usr/local/lib/python2.5/site-packages/matplotlib/backends/backend_gtk.py", 
> line 696, in save_figure
> fname = self.fileselect.get_filename_from_user()
> File 
> "/usr/local/lib/python2.5/site-packages/matplotlib/backends/backend_gtk.py", 
> line 1095, in get_filename_from_user
> menu_ext = IMAGE_FORMAT[self.cbox.get_active()]
> AttributeError: 'FileChooserDialog' object has no attribute 'cbox'
>
> I haven't checked, but I suspect this may be a bug introduced during 
> your recent changes.
>
> Eric