[To summarize an off-list conversation -- Eric and I had been discussing ways to speed up pcolor plots. After some improvements, quadmeshes are still about a factor of 2 slower on the branch than on the trunk. His use case often involves the simpler problem of rectilinear meshes, which can be handled by NonUniformImage, which should be much faster in all cases.] Eric Firing wrote: > you might want > to look at the long-neglected but promising pcolor method in the > _image.cpp, used by the NonUniformImage class in image.py. There is an example on using NonUniformImage in pcolor_nonuniform.py that seems to work -- so this code isn't too neglected ;) > I suspect that the axes pcolor method should actually be using this if > the grid is rectilinear and using quadmesh otherwise. Implementation wise (just to get something working), it was easier to extend imshow to take two 1D-arrays X and Y which define a non-uniform grid for the image. (My changes merely give NonUniformImage a pyplot API, so the user doesn't have to do as much work as in the example.) But if we think this functionality belongs with pcolor, it can be exposed that way instead. However, there should be some way to let pcolor know that the mesh is rectilinear. (It would otherwise be wasteful computation just to determine that and proceed accordingly). As expected, there are significant performance benefits: Branch: nonuniformimage: init: 0.27 fps: 21.37 pcolormesh: init: 0.42 fps: 2.61 Trunk: nonuniformimage: init: 0.25 fps: 22.52 pcolormesh: init: 0.28 fps: 6.64 I've committed my changes on the transforms branch so you can play with it -- I'm holding off on changing the trunk due to the pending release. But if everyone agrees on the way to expose this, it would be nice to merge this over to trunk before the release. Cheers, Mike -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
Mike, Thank you for once again blasting out such an array of improvements. Regarding implementation and API details, such as what should go in imshow versus pcolor versus something with another name, I would like to review the situation (and your branch) and come up with a recommendation, but I can't do it immediately. I can have something waiting for you on Monday morning, however. (But very briefly, an initial thought: an image is a very special "thing", and I am reluctant to overload imshow. We may do best to have separate methods for each distinctly separate case. That can keep both API and implementation simpler than trying to cram too many variations into a single method or function.) (Arg! This brings up the *big* question of what should be a class, and how much functionality, and what kind, should be stuffed into axes methods.) Eric Michael Droettboom wrote: > John Hunter wrote: >> On Nov 9, 2007 1:12 PM, Michael Droettboom <md...@st...> wrote: >> >>> I've committed my changes on the transforms branch so you can play with >>> it -- I'm holding off on changing the trunk due to the pending release. >>> But if everyone agrees on the way to expose this, it would be nice to >>> merge this over to trunk before the release. >> >> Am I right in assuming that the only thing we lose in this approach is >> faceting (which Eric hates but others may care about)? Since it is >> orders of magnitudes faster, we could have a pcolor_faceted which >> pcolor calls the old function if shading='faceted'. Of course the two >> functions would return different types (image vs patch collection) >> which is potentially a bit confusing.... We could play with adding >> faceting to images.... > > pcolor can draw an arbitrary quadmesh (see quadmesh_demo.py, which uses > pcolor), where the edges of the quadrilaterals are not necessarily > parallel to the x or y axes. The NonUniformImage stuff requires that > the quadrilaterals are axis-aligned rectangles. To put it another way, > the X and Y arrays (that define the mesh) can be 2-dimensional for > pcolor, but only 1-dimensional for (the new) imshow. > > pcolormesh, AFAICT, is more-or-less functionally equivalent to pcolor, > but uses optimized quadmesh drawing under the hood, rather than a > PolyCollection. (Though the comments hint at subtle differences related > to masking.) > > But you are right -- NonUniformImage does not support outlining each > quadrilateral -- though that may not be hard to add if needed. > > The difference in return types is perhaps an argument for > NonUniformImages going in imshow, not pcolor. (I was thinking only of > ease of implementation...) > > Cheers, > Mike >
On Nov 9, 2007 1:12 PM, Michael Droettboom <md...@st...> wrote: > I've committed my changes on the transforms branch so you can play with > it -- I'm holding off on changing the trunk due to the pending release. > But if everyone agrees on the way to expose this, it would be nice to > merge this over to trunk before the release. Am I right in assuming that the only thing we lose in this approach is faceting (which Eric hates but others may care about)? Since it is orders of magnitudes faster, we could have a pcolor_faceted which pcolor calls the old function if shading='faceted'. Of course the two functions would return different types (image vs patch collection) which is potentially a bit confusing.... We could play with adding faceting to images.... JDH
John Hunter wrote: > On Nov 9, 2007 1:12 PM, Michael Droettboom <md...@st...> wrote: > >> I've committed my changes on the transforms branch so you can play with >> it -- I'm holding off on changing the trunk due to the pending release. >> But if everyone agrees on the way to expose this, it would be nice to >> merge this over to trunk before the release. > > Am I right in assuming that the only thing we lose in this approach is > faceting (which Eric hates but others may care about)? Since it is > orders of magnitudes faster, we could have a pcolor_faceted which > pcolor calls the old function if shading='faceted'. Of course the two > functions would return different types (image vs patch collection) > which is potentially a bit confusing.... We could play with adding > faceting to images.... pcolor can draw an arbitrary quadmesh (see quadmesh_demo.py, which uses pcolor), where the edges of the quadrilaterals are not necessarily parallel to the x or y axes. The NonUniformImage stuff requires that the quadrilaterals are axis-aligned rectangles. To put it another way, the X and Y arrays (that define the mesh) can be 2-dimensional for pcolor, but only 1-dimensional for (the new) imshow. pcolormesh, AFAICT, is more-or-less functionally equivalent to pcolor, but uses optimized quadmesh drawing under the hood, rather than a PolyCollection. (Though the comments hint at subtle differences related to masking.) But you are right -- NonUniformImage does not support outlining each quadrilateral -- though that may not be hard to add if needed. The difference in return types is perhaps an argument for NonUniformImages going in imshow, not pcolor. (I was thinking only of ease of implementation...) Cheers, Mike -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
On Fri, Nov 09, 2007 at 01:39:10PM -0600, John Hunter wrote: > On Nov 9, 2007 1:12 PM, Michael Droettboom <md...@st...> wrote: > > > I've committed my changes on the transforms branch so you can play with > > it -- I'm holding off on changing the trunk due to the pending release. > > But if everyone agrees on the way to expose this, it would be nice to > > merge this over to trunk before the release. > > Am I right in assuming that the only thing we lose in this approach is > faceting (which Eric hates but others may care about)? Since it is > orders of magnitudes faster, we could have a pcolor_faceted which > pcolor calls the old function if shading='faceted'. Of course the two > functions would return different types (image vs patch collection) > which is potentially a bit confusing.... We could play with adding > faceting to images.... It is important for us that the entire cell have the same colour. Is this what you get with shading='flat'? Sometimes shading='faceted' is useful. You should be able to do this faster by simply blasting an array of lines over top of shading='flat' than by drawing each individual quadrilateral. Note that I found 'faceted' to be much nicer when the lines are drawn with transparency. Without transparency, a large array turns the entire image black --- not very useful. - Paul
John Hunter wrote: > On Nov 9, 2007 1:12 PM, Michael Droettboom <md...@st...> wrote: > >> I've committed my changes on the transforms branch so you can play with >> it -- I'm holding off on changing the trunk due to the pending release. >> But if everyone agrees on the way to expose this, it would be nice to >> merge this over to trunk before the release. > > Am I right in assuming that the only thing we lose in this approach is > faceting (which Eric hates but others may care about)? Since it is John, I agree that others may want faceting, but I have not yet heard from anyone who does. When I proposed making "flat" the default I got only positive comments, and there have been no complaints about the change. > orders of magnitudes faster, we could have a pcolor_faceted which > pcolor calls the old function if shading='faceted'. Of course the two > functions would return different types (image vs patch collection) > which is potentially a bit confusing.... We could play with adding > faceting to images.... Not worth the trouble; retaining an old-style pcolor with faceting that can be turned on is good enough support for the faceted use case--if anyone is even using it. Eric
Michael Droettboom wrote: > John Hunter wrote: >> On Nov 9, 2007 1:12 PM, Michael Droettboom <md...@st...> wrote: >> >>> I've committed my changes on the transforms branch so you can play with >>> it -- I'm holding off on changing the trunk due to the pending release. >>> But if everyone agrees on the way to expose this, it would be nice to >>> merge this over to trunk before the release. >> >> Am I right in assuming that the only thing we lose in this approach is >> faceting (which Eric hates but others may care about)? Since it is >> orders of magnitudes faster, we could have a pcolor_faceted which >> pcolor calls the old function if shading='faceted'. Of course the two >> functions would return different types (image vs patch collection) >> which is potentially a bit confusing.... We could play with adding >> faceting to images.... > > pcolor can draw an arbitrary quadmesh (see quadmesh_demo.py, which uses > pcolor), where the edges of the quadrilaterals are not necessarily > parallel to the x or y axes. The NonUniformImage stuff requires that > the quadrilaterals are axis-aligned rectangles. To put it another way, > the X and Y arrays (that define the mesh) can be 2-dimensional for > pcolor, but only 1-dimensional for (the new) imshow. > > pcolormesh, AFAICT, is more-or-less functionally equivalent to pcolor, > but uses optimized quadmesh drawing under the hood, rather than a > PolyCollection. (Though the comments hint at subtle differences related > to masking.) > > But you are right -- NonUniformImage does not support outlining each > quadrilateral -- though that may not be hard to add if needed. > > The difference in return types is perhaps an argument for > NonUniformImages going in imshow, not pcolor. (I was thinking only of > ease of implementation...) > > Cheers, > Mike > Mike, I have looked at NonUniformImage and found that although in the extension code it is called "pcolor", it is not doing the right thing for pcolor; and I think not quite the right thing for anything "image-like" either. For the latter, the problem is that it extrapolates the edges out indefinitely. I can't think of any circumstance in which this would be desirable. For the former, which is the application that really interests me, this extrapolation is one problem, but worse than that, it does not allow easy specification of the rectangle *boundaries*, which is what pcolor really needs. So, it is a strange hybrid. I thought I might simply make a modified version to do what pcolor really needs, but between general distraction and floundering in C++ I have not succeeded, although it looks like it should be very easy--especially for someone comfortable with C++ and Agg, which I am not. So, if you can get to it, that would be great. If not, I can probably do it within a week or two. The pcolor variant of the nonuniform image code should accept as X and Y vectors giving the *boundaries* of the rectangles, so they should have lengths 1 greater than the corresponding dimensions of the color array. Any pixels that are not within the rectangles should be given the background color, so this needs to be passed in as well, just as it is for the regular image initializer. I will discuss higher level API questions later. Eric