I just committed changes that add TTF subsetting to the PDF backend. It is completely analogous to the font subsetting recently added to the PS backend. I have added a configuration option, pdf.fonttype, to choose either "Type3" or "Truetype" font output. This may be removed in the future once the "Type3" stuff has been sufficiently tested. Some results: fonts_demo_kw.py: 201744 -> 37326 mathtext_demo.py: 129306 -> 26179 unicode_demo.py: 45303 -> 20084 over all demos in backend_driver.py: 5856001 -> 3390460 The differences aren't as dramatic as with Postscript, but IMHO they are still large enough to be worthwhile. Again, please help by testing with your own favorite PDF tools. Gory details about composite characters follow --> In this new code, composite characters (such as a character composed of a letter and an accent) aren't handled as they should be. According to the PDF spec, a PDF-1.2 (Acrobat 3.x) Type 3 font can reference other glyphs with the "Do" command, to avoid duplicating the components of a composite glyph. I was able to get this to work with Acrobat 7, but xpdf-3.0 and ggv 2.8.0 both choked on the file. Therefore, I decided to err on the side of compatibility by including each component of a composite character inline where it is used. This makes the PDF files larger than they would otherwise have to be. However, it should only be a real problem if a plot contains an inordinate amount of different accented characters. Cheers, Mike
On 7/10/07, Michael Droettboom <md...@st...> wrote: > I just committed changes that add TTF subsetting to the PDF backend. It > is completely analogous to the font subsetting recently added to the PS > backend. > > I have added a configuration option, pdf.fonttype, to choose either > "Type3" or "Truetype" font output. This may be removed in the future > once the "Type3" stuff has been sufficiently tested. > > Some results: > > fonts_demo_kw.py: 201744 -> 37326 > mathtext_demo.py: 129306 -> 26179 > unicode_demo.py: 45303 -> 20084 > over all demos in backend_driver.py: 5856001 -> 3390460 When you say "over all demos" do you mean just over the PS or PDF depending on which you are testing? I'm a bit confused because the single examples you show show between a 5 and 20 fold improvement, but the overall number is less than 2 fold. So I wonder if you are including the other backend driver PNG, SVG, etc... output.... Just curious. JDH
John Hunter wrote: > On 7/10/07, Michael Droettboom <md...@st...> wrote: >> >> fonts_demo_kw.py: 201744 -> 37326 >> mathtext_demo.py: 129306 -> 26179 >> unicode_demo.py: 45303 -> 20084 >> over all demos in backend_driver.py: 5856001 -> 3390460 > > When you say "over all demos" do you mean just over the PS or PDF > depending on which you are testing? I'm a bit confused because the > single examples you show show between a 5 and 20 fold improvement, but > the overall number is less than 2 fold. So I wonder if you are > including the other backend driver PNG, SVG, etc... output.... Just > curious. The comparison is just over the PDF files, old way (Truetype embedding) vs. new way (Type 3 subsetting). The ratios are different because I chose to highlight the examples that are quite "texty". That wasn't a deliberate attempt to mislead, it's just because this change is related to fonts. Most of the other examples use a single font, and the plotting content itself dominates file size. Cheers, Mike
On 7/10/07, Michael Droettboom <md...@st...> wrote: > The comparison is just over the PDF files, old way (Truetype embedding) > vs. new way (Type 3 subsetting). > The ratios are different because I chose to highlight the examples that > are quite "texty". That wasn't a deliberate attempt to mislead, it's > just because this change is related to fonts. Most of the other > examples use a single font, and the plotting content itself dominates > file size. This must be dominated by some weird outliers. I'm seeing great results with the canonical "simple_plot" johnh@flag:examples> python simple_plot.py -dPS johnh@flag:examples> mv simple_plot.ps new.ps johnh@flag:examples> PYTHONPATH=/my/old/site-packages python simple_plot.py -dPS johnh@flag:examples> mv simple_plot.ps old.ps johnh@flag:examples> ls -l old.ps new.ps -rw-r--r-- 1 johnh research 19352 Jul 10 12:11 new.ps -rw-r--r-- 1 johnh research 144227 Jul 10 12:11 old.ps Though for some reason my 90.1 install is picking up Vera Serif and my svn install is picking up Vera Sans., which is mysterious but unrelated to your work In any case, excellent work! JDH
"John Hunter" <jd...@gm...> writes:
> On 7/10/07, Michael Droettboom <md...@st...> wrote:
>
>> The comparison is just over the PDF files, old way (Truetype embedding)
>> vs. new way (Type 3 subsetting).
>
> This must be dominated by some weird outliers. I'm seeing great
> results with the canonical "simple_plot"
>
> -rw-r--r-- 1 johnh research 19352 Jul 10 12:11 new.ps
> -rw-r--r-- 1 johnh research 144227 Jul 10 12:11 old.ps
Probably the difference is just that you are comparing PS files while
Michael was comparing PDF files, where the internal gzip compression
reduces the file size. Even in PDF files you can get large gains if you
use several different fonts (which can easily happen with math formulas)
or you have very big fonts (such as some fonts that come with Mac OS X).
I'm seeing a bug on OS X, whose file system is by default
case-preserving but not case-sensitive:
458 -> fontdictObject = self.embedTTF(
459 filename, self.used_characters[filename])
Here self.used_characters has a key starting with '/Users/jks/...' but
filename is '/users/jks/...'. I'm not sure how to fix this cleanly.
I though os.path.normcase would help, but it doesn't:
>>> normcase('/users'); normcase('/Users')
'/users'
'/Users'
> In any case, excellent work!
Yes, very good! This is a much-needed feature for the PS and PDF
backends.
--
Jouni K. Seppänen
http://www.iki.fi/jks
Jouni K. Seppänen wrote: > "John Hunter" <jd...@gm...> writes: > > >> On 7/10/07, Michael Droettboom <md...@st...> wrote: >> > I'm seeing a bug on OS X, whose file system is by default > case-preserving but not case-sensitive: > > 458 -> fontdictObject = self.embedTTF( > 459 filename, self.used_characters[filename]) > > Here self.used_characters has a key starting with '/Users/jks/...' but > filename is '/users/jks/...'. I'm not sure how to fix this cleanly. > I though os.path.normcase would help, but it doesn't: > > Thanks for that. The Ps backend has the same problem. I'll do a little research and see what a common solution to this might be. Cheers, Mike
Michael Droettboom wrote: > Jouni K. Seppänen wrote: > >> "John Hunter" <jd...@gm...> writes: >> >> >> >>> On 7/10/07, Michael Droettboom <md...@st...> wrote: >>> >>> >> I'm seeing a bug on OS X, whose file system is by default >> case-preserving but not case-sensitive: >> >> 458 -> fontdictObject = self.embedTTF( >> 459 filename, self.used_characters[filename]) >> >> Here self.used_characters has a key starting with '/Users/jks/...' but >> filename is '/users/jks/...'. I'm not sure how to fix this cleanly. >> I though os.path.normcase would help, but it doesn't: >> >> >> > Thanks for that. The Ps backend has the same problem. I'll do a little > research and see what a common solution to this might be. > > I changed the code to use the file's (st_ino, st_dev) pair as a key, rather than the path. This is cached to prevent lots of little stat calls. This approach is reported to work on "Macintosh, Unix, and Windows", but I've only tested on Linux and OS-X. Please let me know if it solves your issue. Along the way, I found an interesting discussion about why normcase does what it does on OS-X. (Summary: because file names are case sensitive on OS-X, but not the default HFS+ filesystem that Macs ship with as their System volume.) http://mail.python.org/pipermail/python-list/2006-January/360098.html Cheers, Mike
Eric,
Thanks for the scripts -- they make it quite clear what the problem is.
As you know, cntr.c is a fairly opaque chunk of code, and I'm not having
much luck so far tracking down why the branch-cutting between the outer
and inner polygons is not working.
In the meantime, I have some more general questions.
It looks like that the backend API isn't really set up to do compound
paths. The PolygonCollection class appears to be just a list of simple
polygons, rather than something that could be used for compound paths.
(Correct me if I'm wrong.) Is the only reason for wanting branch cuts
in contours because matplotlib doesn't support compound paths? Is there
a reason matplotlib doesn't support compound paths? I suspect you guys
have been down that "path" [haha] already. I would anticipate problems
if different backends have different winding rules, but perhaps there
are some ways to around that? This could be useful in general in
matplotlib...
Another option is to remove the mask from the contours in a
post-processing step (i.e. don't worry about trying to do it as part of
the main two passes in cntr.c). The result may be a bit of a
Frankenstein's monster, admittedly, but it is an option of last resort
if we choose to take it. cntr.c is creating the inner and outer
polygons correctly, just not connecting them with a slit, right?
In the meantime, I'll continue to look for the root cause.
Cheers,
Mike
Eric Firing wrote:
> Mike,
>
> The attached file masked_interior.py illustrates masking failure in a
> very simple case; you can see masking working in the plot on the left,
> where a contour intersects the masked region, but when that contour
> level is removed the masked region is getting filled in.
>
> The file contourf_demo.py is slightly modified from the one in the mpl
> examples directory, and shows a failure of masking in a more
> complicated setting. The masked region at the lower-left corner is
> correct, but the masked region in the middle of the plot is getting
> filled with gray instead of being left blank.
>
> In cntr.c there is a function, print_Csite, that may be helpful for
> debugging the simplest case, where the array size is not too large.
>
> Note that the code path for filled contours is quite different, and
> more complicated, than for line contours--and in fact, even neglecting
> branch cuts, the two code paths don't always yield the same contours.
>
> cntr.c is somewhat unusual among contour algorithms in that it works
> with rectangles without subdividing them into triangles.
>
> Eric
> ------------------------------------------------------------------------
>
> #!/usr/bin/env python
> '''
> This is a very simple illustration of what is causing
> the problem with masked interior regions for filled contours.
> The exterior contour and the interior contour are being
> separated into two polygons instead of being joined by the
> branch cut.
> '''
> import sys
> from pylab import *
> import matplotlib.numerix.npyma as ma
> rc('figure', dpi=120)
>
> x = y = arange(5)
> X, Y = meshgrid(x, y)
> Z = Y
>
> ax = (0,4,0,4)
>
> badmask = zeros(shape(X))
>
> badmask[2, 2] = 1
>
> Z = ma.array(Z, mask=badmask)
> print Z
>
> subplot(2,1,1)
> CS = contourf(X, Y, Z, (-0.1, 0.8, 2.0, 6.0),
> colors=('r', 'g', 'b'))
>
> CS.collections[0].set_edgecolor('c')
> CS.collections[0].set_linewidth(6)
> CS.collections[1].set_edgecolor('y')
> CS.collections[1].set_linewidth(2)
> CS.collections[2].set_edgecolor('m')
> CS.collections[2].set_linewidth(4)
> title('Masked correctly')
>
> subplot(2,1,2)
> CS = contourf(X, Y, Z, (-0.1, 0.8, 6.0),
> colors=('r', 'b'))
>
> CS.collections[0].set_edgecolor('c')
> CS.collections[0].set_linewidth(6)
> CS.collections[1].set_edgecolor('y')
> CS.collections[1].set_linewidth(2)
> title('Wrong: masked region filled')
> print "len(CS.collections)", len(CS.collections)
>
> colls = CS.collections
>
> # We want to look at the actual polygons individually.
> for i, coll in enumerate(colls):
> print "level ", i
> for j, T in enumerate(coll._verts):
> print "polygon ", j
> print T
>
> # The second collection has two polygons, but should only have
> # one; the two should be joined by a branch cut.
>
> show()
>
>
> ------------------------------------------------------------------------
>
> #!/usr/bin/env python
> from pylab import *
> import matplotlib.numerix.npyma as ma
> origin = 'lower'
> #origin = 'upper'
>
> test_masking = True # There is a bug in filled contour masking.
>
> if test_masking:
> # Use a coarse grid so only a few masked points are needed.
> delta = 0.5
> else:
> delta = 0.025
>
> x = y = arange(-3.0, 3.01, delta)
> X, Y = meshgrid(x, y)
> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
> Z = 10 * (Z1 - Z2)
>
> # interior badmask doesn't work yet for filled contours
> if test_masking:
> badmask = zeros(shape(Z))
>
> badmask[5,5] = 1
> badmask[5,6] = 1
> Z[5,5] = 0
> Z[5,6] = 0
>
> badmask[0,0] = 1
> Z[0,0] = 0
> Z = ma.array(Z, mask=badmask)
>
> # We are using automatic selection of contour levels;
> # this is usually not such a good idea, because they don't
> # occur on nice boundaries, but we do it here for purposes
> # of illustration.
> CS = contourf(X, Y, Z, 10, # [-1, -0.1, 0, 0.1],
> #alpha=0.5,
> cmap=cm.bone,
> origin=origin)
>
> # Note that in the following, we explicitly pass in a subset of
> # the contour levels used for the filled contours. Alternatively,
> # We could pass in additional levels to provide extra resolution.
>
> CS2 = contour(X, Y, Z, CS.levels[::2],
> colors = 'r',
> origin=origin,
> hold='on')
>
> title('Nonsense')
> xlabel('word length anomaly')
> ylabel('sentence length anomaly')
>
> # Make a colorbar for the ContourSet returned by the contourf call.
> cbar = colorbar(CS)
> cbar.ax.set_ylabel('verbosity coefficient')
> # Add the contour line levels to the colorbar
> cbar.add_lines(CS2)
>
> figure()
>
> # Now make a contour plot with the levels specified,
> # and with the colormap generated automatically from a list
> # of colors.
> levels = [-2, -1.5, -1, -0.5, 0, 0.5, 1, 1.5]
> CS3 = contourf(X, Y, Z, levels,
> colors = ('r', 'g', 'b'),
> origin=origin)
>
> CS4 = contour(X, Y, Z, levels,
> colors = ('k',),
> linewidths = (3,),
> origin = origin)
> title('Listed colors')
> clabel(CS4, fmt = '%2.1f', colors = 'w', fontsize=14)
> colorbar(CS3)
>
> #savefig('contourf_demo')
> show()
>
>
On 7/12/07, Michael Droettboom <md...@st...> wrote: > It looks like that the backend API isn't really set up to do compound > paths. The PolygonCollection class appears to be just a list of simple > polygons, rather than something that could be used for compound paths. > (Correct me if I'm wrong.) Is the only reason for wanting branch cuts > in contours because matplotlib doesn't support compound paths? Is there > a reason matplotlib doesn't support compound paths? I suspect you guys > have been down that "path" [haha] already. I would anticipate problems > if different backends have different winding rules, but perhaps there > are some ways to around that? This could be useful in general in > matplotlib... Actually we haven't. In fact, paths have been a fairly recent addition to mpl. The drawing model is based on GTK, which did not have paths at the time. The renderer API is basically the GTK API, and all the other backends simply implement the GTK drawing model. In the early days, there was only a GTK backend.... So one reason why the backends are a bit of a kludge is because we have tried to throw some extra stuff into the GTK drawing model as an afterthough. We could redo all the collection stuff w/ compounds paths. JDH
Michael Droettboom wrote: > cntr.c is creating the inner and outer > polygons correctly, just not connecting them with a slit, right? > To answer my own question, unfortunately, this isn't always the case. There is also a bug where if a masked region is inside of the *inner* polygon of a contour, the inner contour is incorrect. So, other advantages aside, adding support for compound polygons won't magically fix any of the contour problems. Cheers, Mike
John Hunter wrote: [...] > So one reason why the backends are a bit of a kludge is because we > have tried to throw some extra stuff into the GTK drawing model as an > afterthough. We could redo all the collection stuff w/ compounds > paths. John, Do all the backend devices or libraries we *need* now support compound paths (1) at all, and (2) in a consistent enough way to solve the contouring problem as well as to make something like polygon collections more efficient? I guess this is really part of a larger question about the degree to which the various graphics drawing models have converged with a rich intersection of capabilities, including arbitrary text rotation, precise text positioning, arcs, bezier curves, etc. Eric
On 7/12/07, Eric Firing <ef...@ha...> wrote: > John Hunter wrote: > Do all the backend devices or libraries we *need* now support compound > paths (1) at all, and (2) in a consistent enough way to solve the > contouring problem as well as to make something like polygon collections > more efficient? > > I guess this is really part of a larger question about the degree to > which the various graphics drawing models have converged with a rich > intersection of capabilities, including arbitrary text rotation, precise > text positioning, arcs, bezier curves, etc. Yes, I think they do, though we will encounter some differences between spline handling that might make some of the path stuff tricky. We need agg, cairo, svg, ps and pdf, all of which use vector graphics. We would need to sacrifice native GDK and WX, which would inconvenience some but these users could certainly get by with GTK* or WX*. One reason a few still use native GTK is because it is faster, but this would likely disappear with some intelligent design. One might envision just three or four fundamental objects which the backends need to understand. Path CompoundPath Image Text and maybe use a stateful graphics context (linewidth, facecolor, etc...) numpify everything, traitify everything and rebuild the transformation architecture from scratch. Borrowing some language from ipython, this would truly be a "chainsaw" branch. I'm halfway inclined to take a crack at it...... JDH
Carl Worth wrote: > On 2007年7月12日 10:31:19 -1000, Eric Firing wrote: >> if only cairo would make eps files > > Isn't EPS a trivially change compared to PS? Something like a modified > header and the addition of bounding-box information? Carl, My knowledge of graphics topics is sketchy, but yes, in many cases the difference between PS and EPS is just a few lines. This is true for Matlab files--ps and eps are identical except for something like 3 lines. This works *provided* the ps file doesn't have anything in it that is not allowed in eps. I don't recall the constraints, apart from the file being single-page, but I know there are some. I suspect this is not a problem for cairo output, but I haven't tried to check. > > It sounds like something that should be really easy to change in > cairo. Yes, so much so that I don't understand why it is not already done. (I know you are a cairo dev--is there not a postscript specialist in the cairo dev community?) > > Maybe you could take a cairo-generated .ps file and manually munge it > into the .eps that you'd like? If you did that, I'd be glad to write > the necessary cairo code to get the same result, (and come up with > whatever tiny new API is necessary to instruct cairo to do that). If no one else steps forward, I can try to look at it fairly soon--but I am really not the right person to do it. > > As for the rest of what you say. From my point of view, yes, using > cairo and its multi-backend capabilities seems to make a fair amount > more sense than inventing a new system with multiple backends. (Not a > criticism against the original MPL stuff---cairo 1.0 has only existed > since August 2005). > > But yeah, I also understand that there are licensing concerns. > Anything else? Are there cairo performance concerns? If so, I'd love > to hear about them so we can fix them. Well, I think that for screen display and for png generation, Agg is significantly faster. Backend_driver.py gives Backend agg took 0.76 minutes to complete Backend cairo.png took 0.95 minutes to complete Backend template took 0.46 minutes to complete I haven't tried to test it, but I suspect that for some types of plots the optimizations in Agg and the agg backend will make a much bigger difference than the averages above would indicate. Eric > And even if you don't use cairo directly, I will offer the suggestion > of its drawing model as being very good, (using source and mask > pattern objects and just 5 different drawing operations: stroke, fill, > paint, show_text/glyphs, and mask). > > Whatever you decide to go with, have fun! > > -Carl
Carl Worth wrote: > On 2007年7月12日 11:35:25 -1000, Eric Firing wrote: >> Carl Worth wrote: >> [...] > Later, you said... > >> Aha! Cairo ps output is full of "initclip" commands, so it can't be >> converted to eps without substantial reworking. > > Oh, actually that should be really easy to get rid of. The cairo > semantics match PostScript's very closely: > > cairo_clip -> clip > cairo_reset_clip -> initclip > > So, the only problematic case is if the application is actually > calling cairo_reset_clip, (which could simply be made into an error > when targeting EPS). If the application isn't, and cairo is still > generating initclip in the PostScript output then it should be very > simple to fix that. I have looked at the mpl cairo backend code and at the source code for the pycairo package that mpl uses, and I have not figured out where the initclip call is coming from; it does not appear to be anything explicit. reset_clip is wrapped by pycairo, but an example snippet that generates ps with an initclip does not include any call to reset_clip. Nor does the mpl backend. So I conclude that it is being generated implicitly. I have not checked for any of the other postscript commands that are not allowed in eps, and I really can't pursue this any farther. But thanks for the ideas and encouragement; and in turn, if you are interested, I certainly encourage you to pursue the better utilization of cairo in mpl. The author of the backend, as well as of pycairo, is Steve Chaplin. Eric
John Hunter wrote: > On 7/10/07, Michael Droettboom <md...@st...> wrote: > >> The comparison is just over the PDF files, old way (Truetype embedding) >> vs. new way (Type 3 subsetting). [...] > > In any case, excellent work! > > JDH > Mike, I second that! I greatly appreciate your contributions, first in chasing down memory leaks and now in reducing file sizes by embedding fonts. Eric
Eric Firing wrote: > I second that! I greatly appreciate your contributions, first in > chasing down memory leaks and now in reducing file sizes by embedding > fonts. It's been fun. Now, Eric, I'm just waiting for you to tell me how this latest batch reveals another bug on Ubuntu ;) (with all due respect to Ubuntu) Cheers, Mike
Michael Droettboom wrote: > Eric Firing wrote: >> I second that! I greatly appreciate your contributions, first in >> chasing down memory leaks and now in reducing file sizes by embedding >> fonts. > It's been fun. Mike, Good--what's next? You're ready for more fun, I hope. If you are looking for brain-benders, I know of two bugs lurking deep in extension code (one in cntr.c, the other in the Agg quadmesh rendering) that have completely eluded me. There is also a bug in the Agg image rendering that shows up at high magnification. If you are interested in grand strategy, let's discuss what a suitable target might be--maybe some refactoring and consolidation of backend code to reduce duplication, and make optimizations applicable to all (referring to recent work by Allan). A related idea that has been languishing is to consolidate the various image-like functionality, again so as to make maximum use of the different optimizations and options that are now spread among different functions. John probably has better ideas about what to attack. > > Now, Eric, I'm just waiting for you to tell me how this latest batch > reveals another bug on Ubuntu ;) (with all due respect to Ubuntu) So, you use RHEL4 at work and ubuntu at home; has the former actually managed to put together a set of relatively bug-free versions of gui toolkits and other libraries? Or is it a choice between old bugs and newer bugs? Given all the different constantly-changing libraries, compilers, etc. involved in something like mpl, I am sometimes amazed that it works at all--and worried that at any moment it may cease to work. Eric > > Cheers, > Mike
Michael Droettboom <md...@st...> writes: > This approach is reported to work on "Macintosh, Unix, and Windows", > but I've only tested on Linux and OS-X. Please let me know if it > solves your issue. It solves my problem on OS X. Thanks! -- Jouni K. Seppänen http://www.iki.fi/jks
Mike, The attached file masked_interior.py illustrates masking failure in a very simple case; you can see masking working in the plot on the left, where a contour intersects the masked region, but when that contour level is removed the masked region is getting filled in. The file contourf_demo.py is slightly modified from the one in the mpl examples directory, and shows a failure of masking in a more complicated setting. The masked region at the lower-left corner is correct, but the masked region in the middle of the plot is getting filled with gray instead of being left blank. In cntr.c there is a function, print_Csite, that may be helpful for debugging the simplest case, where the array size is not too large. Note that the code path for filled contours is quite different, and more complicated, than for line contours--and in fact, even neglecting branch cuts, the two code paths don't always yield the same contours. cntr.c is somewhat unusual among contour algorithms in that it works with rectangles without subdividing them into triangles. Eric
On 2007年7月12日 11:35:25 -1000, Eric Firing wrote: > Carl Worth wrote: > > > It sounds like something that should be really easy to change in > > cairo. > > Yes, so much so that I don't understand why it is not already done. (I > know you are a cairo dev--is there not a postscript specialist in the > cairo dev community?) Honestly, there just plain hasn't been much demand for it. We do have several very competent PostScript specialists in the community. I would expect that if you floated the idea on the cairo list, (and maybe pointed out a particular missing issue), the problem could get solved very quickly. Later, you said... > Aha! Cairo ps output is full of "initclip" commands, so it can't be > converted to eps without substantial reworking. Oh, actually that should be really easy to get rid of. The cairo semantics match PostScript's very closely: cairo_clip -> clip cairo_reset_clip -> initclip So, the only problematic case is if the application is actually calling cairo_reset_clip, (which could simply be made into an error when targeting EPS). If the application isn't, and cairo is still generating initclip in the PostScript output then it should be very simple to fix that. > Well, I think that for screen display and for png generation, Agg is > significantly faster. Backend_driver.py gives > > Backend agg took 0.76 minutes to complete > Backend cairo.png took 0.95 minutes to complete > Backend template took 0.46 minutes to complete Ah, thanks. So there's definitely some work to be done there. Does anyone know a list of any optimizations that are specific to the AGG backend, but could in fact be general. (The recently discussed culling of geometry outside the visible region comes to mind for example---that wasn't generalized was it?). And I do understand that the proposal for reworking the backend interface is to help with concerns like this. Anyway, I'll try to get setup to profile the cairo run above and see if any obvious things stick out that could be easily fixed. -Carl
On 7/12/07, Carl Worth <cw...@cw...> wrote: > Anyway, I'll try to get setup to profile the cairo run above and see > if any obvious things stick out that could be easily fixed. Most likely, the biggest win by far will be to implement draw_markers. You will see the start of an attmept in draw_markers_OLD. Not having this function means symbol plots must make poosibly thousands of separate calls to draw_polygon and friends, each with their own set of gc calls. backend_ps is probably the best place to look for inspiration. If you search the devel archives for draw_markers, you'll find lots of discussion .... JDH
John Hunter wrote: > On 7/12/07, Michael Droettboom <md...@st...> wrote: > >> It looks like that the backend API isn't really set up to do compound >> paths. The PolygonCollection class appears to be just a list of simple >> polygons, rather than something that could be used for compound paths. >> (Correct me if I'm wrong.) Is the only reason for wanting branch cuts >> in contours because matplotlib doesn't support compound paths? Is there >> a reason matplotlib doesn't support compound paths? I suspect you guys >> have been down that "path" [haha] already. I would anticipate problems >> if different backends have different winding rules, but perhaps there >> are some ways to around that? This could be useful in general in >> matplotlib... > > Actually we haven't. In fact, paths have been a fairly recent > addition to mpl. The drawing model is based on GTK, which did not > have paths at the time. The renderer API is basically the GTK API, > and all the other backends simply implement the GTK drawing model. In > the early days, there was only a GTK backend.... > > So one reason why the backends are a bit of a kludge is because we > have tried to throw some extra stuff into the GTK drawing model as an > afterthough. We could redo all the collection stuff w/ compounds > paths. This is the sort of thing I was getting at by suggesting that if a big project is possible, rethinking the drawing model and refactoring accordingly would be a candidate. I think there is quite a bit of scope for fundamentally improving mpl internals (and maybe even performance), and reducing future maintenance costs, by doing this. It is at the same level as, and maybe complementary to, redoing the transforms framework. With respect to contouring, yes, a branch cut is a kludge--a way to fake a compound path. It would be much cleaner if the cuts could be eliminated. With a little extra record-keeping to mark the parts of paths that bound masked regions and boundaries, this would also solve the problem that the polygon boundaries don't always coincide with the contour lines that are calculated with cntr.c in non-filled mode. Eric > > JDH
On 2007年7月12日 09:29:06 -1000, Eric Firing wrote: > > So one reason why the backends are a bit of a kludge is because we > > have tried to throw some extra stuff into the GTK drawing model as an > > afterthough. We could redo all the collection stuff w/ compounds > > paths. What's the "GTK drawing model" here that is problematic? Is that pre-cairo stuff? And can using cairo directly help at all? -Carl
On 7/12/07, Carl Worth <cw...@cw...> wrote: > What's the "GTK drawing model" here that is problematic? Is that > pre-cairo stuff? And can using cairo directly help at all? Yes, this is pre cairo and unfortunately, no I don't think cairo can help. When I wrote MPL, it was GTK only and I used GDK for rendering. The backend canvas was a gtk.DrawingArea calls and the renderer was a gdk.Drawable which defined methods like draw_rectangle and draw_polygon http://www.pygtk.org/pygtk2reference/class-gdkdrawable.html As I added other backends, backend_ps first, I abstracted the gtk.DrawingArea, gdk.Drawable and gdk.GC to the FigureCanvas, Renderer and GraphicsContext respectively which are are core backend base classes, and then for each backend did a concrete implemention of the gdk.Drawable API, etc. This was back in 2001. If I were smarter or had done more research at the time, I would have chosen a different drawing mode, eg DisplayPDF or something like that, but I guess I was too busy coding .... (reminds me of a joke we used to tell in the lab "Why spend and hour in the library when you can get the same result in the lab in a year?"). So the fact that GTK now uses Cairo doesn't help us too much, since all of our backends implement the gdk.Drawable API more or less. What we are discussing here is a clean room reimplementation of our backend API and drawing model. JDH
John Hunter wrote: > On 7/12/07, Eric Firing <ef...@ha...> wrote: >> John Hunter wrote: > >> Do all the backend devices or libraries we *need* now support compound >> paths (1) at all, and (2) in a consistent enough way to solve the >> contouring problem as well as to make something like polygon collections >> more efficient? >> >> I guess this is really part of a larger question about the degree to >> which the various graphics drawing models have converged with a rich >> intersection of capabilities, including arbitrary text rotation, precise >> text positioning, arcs, bezier curves, etc. > > Yes, I think they do, though we will encounter some differences > between spline handling that might make some of the path stuff tricky. > We need agg, cairo, svg, ps and pdf, all of which use vector > graphics. We would need to sacrifice native GDK and WX, which would From a purely technical standpoint--getting the desired output formats--perhaps cairo could substitute for pdf and svg, and also for ps if only cairo would make eps files. So, even if eventually you want to restore native pdf, svg, and ps backends, it could be done at a later stage than the initial chainsaw sculpture. > inconvenience some but these users could certainly get by with GTK* or > WX*. One reason a few still use native GTK is because it is faster, > but this would likely disappear with some intelligent design. One > might envision just three or four fundamental objects which the > backends need to understand. > > Path > CompoundPath > Image > Text > > and maybe use a stateful graphics context (linewidth, facecolor, etc...) > > numpify everything, traitify everything and rebuild the > transformation architecture from scratch. Borrowing some language > from ipython, this would truly be a "chainsaw" branch. > > I'm halfway inclined to take a crack at it...... Great! I wonder whether your company could be convinced to give you some slack for that... It might be worth a bit of thought and planning to see how parts of the work could be divided up so that you would not have to do everything yourself. To minimize the interval during which a separate branch would be needed, maybe some initial traitification should be done without a fork. Eric > > JDH