Hello everybody,
I have sent this message to the user group, but thinking of it, it may be more
relevant to the development mailing list...so here it is again.
We are looking for the best way to plot a waterfall diagram in
Matplotlib. The 2 functions which could be used
to do that are (as far as I have found) imshow and pcolormesh. Here is a
small script that use both to compare the output:
-----------------
from pylab import *
delta = 0.2
x = arange(-3.0, 3.0, delta)
y = arange(-2.0, 2.0, delta)
X, Y = meshgrid(x, y)
Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
# difference of Gaussians
Z = 10.0 * (Z2 - Z1)
figure(1)
im = imshow(Z,extent=(-3,3,-2,2))
CS = contour(X, -Y, Z, 6,
colors='k', # negative contours will be dashed by default
)
clabel(CS, fontsize=9, inline=1)
title('Using imshow')
figure(2)
im = pcolormesh(X,-Y,Z)
CS = contour(X, -Y, Z, 6,
colors='k', # negative contours will be dashed by default
)
clabel(CS, fontsize=9, inline=1)
title('Using pcolormesh')
show()
---------------------
The problem is that we need some of the flexibility of pcolormesh (which
is able to map the matrix of value on any deformed mesh), while
we would like to use the interpolations available in imshow (which
explain why the imshow version is much "smoother" than the pcolormesh
one).
In fact, what would be needed is not the full flexibility of pcolormesh
(which can map the grid to any kind of shape), we "only" have to deal
with rectangular grids where x- and y- graduations are irregularly spaced.
Is there a drawing function in Matplotlib which would be able to work
with such a rectangular non-uniform grid?
And if not (and a quick look at the example and the code make me think
that indeed the capability is currently not present),
what about an extension of imshow which would work as this:
im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
position of the grid's nodes, instead of giving the extend and assuming
uniform spacing.
Longer term, would a pcolormesh accepting interpolation be possible? The
current behavior, averaging the color of the grids node to get a uniform
cell color,
is quite rough except for a large number of cells...And even then, it
soon shows when you zoom in...
The best would be to allow the same interpolations as in imshow (or a
subset of it), and also allows to use interpolation before colormap
lookup (or after),
like in Matlab. Indeed, Matlab allows to finely tune interpolation by
specifying Gouraud (interpolation after color
lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
Phong is usually much better but also more CPU intensive. Phong is
especially when using discrete colormap, producing banded colors
equivalent to countour lines, while Gouraud does not work in those
cases.
Of course, the performance will be impacted by some of those
interpolation options, which would degrade performance in animations for
example.... but I think that having the different options available
would be very useful, it allows to have the highest map quality, or have
a "quick and dirty" map depending on situation (grid spacing, type of
map, animation or not, ...).
Best regards,
Greg.
On Fri, 2008年08月08日 at 16:05 +0200, Grégory Lielens wrote:
> Hello everybody,
>
> I have sent this message to the user group, but thinking of it, it may be more
> relevant to the development mailing list...so here it is again.
>
>
>
> We are looking for the best way to plot a waterfall diagram in
> Matplotlib. The 2 functions which could be used
> to do that are (as far as I have found) imshow and pcolormesh. Here is a
> small script that use both to compare the output:
>
> -----------------
>
> from pylab import *
>
>
> delta = 0.2
> x = arange(-3.0, 3.0, delta)
> y = arange(-2.0, 2.0, delta)
> X, Y = meshgrid(x, y)
> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
> # difference of Gaussians
> Z = 10.0 * (Z2 - Z1)
> figure(1)
> im = imshow(Z,extent=(-3,3,-2,2))
> CS = contour(X, -Y, Z, 6,
> colors='k', # negative contours will be dashed by default
> )
> clabel(CS, fontsize=9, inline=1)
> title('Using imshow')
> figure(2)
> im = pcolormesh(X,-Y,Z)
> CS = contour(X, -Y, Z, 6,
> colors='k', # negative contours will be dashed by default
> )
> clabel(CS, fontsize=9, inline=1)
> title('Using pcolormesh')
> show()
>
> ---------------------
>
>
> The problem is that we need some of the flexibility of pcolormesh (which
> is able to map the matrix of value on any deformed mesh), while
> we would like to use the interpolations available in imshow (which
> explain why the imshow version is much "smoother" than the pcolormesh
> one).
>
> In fact, what would be needed is not the full flexibility of pcolormesh
> (which can map the grid to any kind of shape), we "only" have to deal
> with rectangular grids where x- and y- graduations are irregularly spaced.
>
> Is there a drawing function in Matplotlib which would be able to work
> with such a rectangular non-uniform grid?
> And if not (and a quick look at the example and the code make me think
> that indeed the capability is currently not present),
> what about an extension of imshow which would work as this:
>
> im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
> position of the grid's nodes, instead of giving the extend and assuming
> uniform spacing.
>
> Longer term, would a pcolormesh accepting interpolation be possible? The
> current behavior, averaging the color of the grids node to get a uniform
> cell color,
> is quite rough except for a large number of cells...And even then, it
> soon shows when you zoom in...
>
> The best would be to allow the same interpolations as in imshow (or a
> subset of it), and also allows to use interpolation before colormap
> lookup (or after),
> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
> specifying Gouraud (interpolation after color
> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
> Phong is usually much better but also more CPU intensive. Phong is
> especially when using discrete colormap, producing banded colors
> equivalent to countour lines, while Gouraud does not work in those
> cases.
>
> Of course, the performance will be impacted by some of those
> interpolation options, which would degrade performance in animations for
> example.... but I think that having the different options available
> would be very useful, it allows to have the highest map quality, or have
> a "quick and dirty" map depending on situation (grid spacing, type of
> map, animation or not, ...).
>
> Best regards,
>
> Greg.
I have found a method which implement the proposed extension to imshow:
NonUniformImage...
However, this image instance support only nearest neighbor
interpolation. Trying to set the interpolation (using the
set_interpolation method)
to something akin imshow throw a "NotImplementedError: Only nearest
neighbor supported" exception....
So basically I am still stuck, it seems that currently there is no way
in matplotlib to plot interpolated
colormap on irregular rectangular grid, and even less on arbitrarily
mapped grid...
Is there any plans to add support for more interpolation in
NonUniformImage in the future? Or maybe there is another
drawing function that I did not find yet, with this ability?
Best regards,
Greg.
Hi all,
here is a patch which implement bilinear interpolation on irregular grid
( i.e. it allows NonUniformImage
to accept both 'nearest' and 'bilinear' interpoaltion, instead of only
'nearest'.)
It is not perfect, given the current architecture of the image module I
think there is not simple way
to specify other interpolations (except for 'nearest', all
interpolations are implemented at the AGG level, not in
matplotlib itself).
For the same reason, it is not possible to interpolate before colormap
lookup instead of after (and this
is usually where the biggest effect is, when dealing with coarse grid).
However, I think it is already quite useful and the best one ca do
without a -very- extensive rewrite
of the matrix map modules....
BTW, I think it also corrects a small bug in the 'nearest'
interpolation: the last intervals was ignored
in the CVS version, now it is taken into account.
BOTH nearest and bilinear interpolation do the same for values oustside
the data grid: they just use boundary values (constant extrapolation).
I avoided linear extrapolation for 'bilinear', as extrapolation is
usually dangerous or meaningless (we can go outside the colormap very
fast)...
I have included a small example showing how both interpolation works....
Any remarks, could this be added before the next release? ;-)
Greg.
On Mon, 2008年08月11日 at 16:50 +0200, Grégory Lielens wrote:
> On Fri, 2008年08月08日 at 16:05 +0200, Grégory Lielens wrote:
> > Hello everybody,
> >
> > I have sent this message to the user group, but thinking of it, it may be more
> > relevant to the development mailing list...so here it is again.
> >
> >
> >
> > We are looking for the best way to plot a waterfall diagram in
> > Matplotlib. The 2 functions which could be used
> > to do that are (as far as I have found) imshow and pcolormesh. Here is a
> > small script that use both to compare the output:
> >
> > -----------------
> >
> > from pylab import *
> >
> >
> > delta = 0.2
> > x = arange(-3.0, 3.0, delta)
> > y = arange(-2.0, 2.0, delta)
> > X, Y = meshgrid(x, y)
> > Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
> > Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
> > # difference of Gaussians
> > Z = 10.0 * (Z2 - Z1)
> > figure(1)
> > im = imshow(Z,extent=(-3,3,-2,2))
> > CS = contour(X, -Y, Z, 6,
> > colors='k', # negative contours will be dashed by default
> > )
> > clabel(CS, fontsize=9, inline=1)
> > title('Using imshow')
> > figure(2)
> > im = pcolormesh(X,-Y,Z)
> > CS = contour(X, -Y, Z, 6,
> > colors='k', # negative contours will be dashed by default
> > )
> > clabel(CS, fontsize=9, inline=1)
> > title('Using pcolormesh')
> > show()
> >
> > ---------------------
> >
> >
> > The problem is that we need some of the flexibility of pcolormesh (which
> > is able to map the matrix of value on any deformed mesh), while
> > we would like to use the interpolations available in imshow (which
> > explain why the imshow version is much "smoother" than the pcolormesh
> > one).
> >
> > In fact, what would be needed is not the full flexibility of pcolormesh
> > (which can map the grid to any kind of shape), we "only" have to deal
> > with rectangular grids where x- and y- graduations are irregularly spaced.
> >
> > Is there a drawing function in Matplotlib which would be able to work
> > with such a rectangular non-uniform grid?
> > And if not (and a quick look at the example and the code make me think
> > that indeed the capability is currently not present),
> > what about an extension of imshow which would work as this:
> >
> > im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
> > position of the grid's nodes, instead of giving the extend and assuming
> > uniform spacing.
> >
> > Longer term, would a pcolormesh accepting interpolation be possible? The
> > current behavior, averaging the color of the grids node to get a uniform
> > cell color,
> > is quite rough except for a large number of cells...And even then, it
> > soon shows when you zoom in...
> >
> > The best would be to allow the same interpolations as in imshow (or a
> > subset of it), and also allows to use interpolation before colormap
> > lookup (or after),
> > like in Matlab. Indeed, Matlab allows to finely tune interpolation by
> > specifying Gouraud (interpolation after color
> > lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
> > Phong is usually much better but also more CPU intensive. Phong is
> > especially when using discrete colormap, producing banded colors
> > equivalent to countour lines, while Gouraud does not work in those
> > cases.
> >
> > Of course, the performance will be impacted by some of those
> > interpolation options, which would degrade performance in animations for
> > example.... but I think that having the different options available
> > would be very useful, it allows to have the highest map quality, or have
> > a "quick and dirty" map depending on situation (grid spacing, type of
> > map, animation or not, ...).
> >
> > Best regards,
> >
> > Greg.
>
> I have found a method which implement the proposed extension to imshow:
> NonUniformImage...
>
> However, this image instance support only nearest neighbor
> interpolation. Trying to set the interpolation (using the
> set_interpolation method)
> to something akin imshow throw a "NotImplementedError: Only nearest
> neighbor supported" exception....
>
> So basically I am still stuck, it seems that currently there is no way
> in matplotlib to plot interpolated
> colormap on irregular rectangular grid, and even less on arbitrarily
> mapped grid...
>
> Is there any plans to add support for more interpolation in
> NonUniformImage in the future? Or maybe there is another
> drawing function that I did not find yet, with this ability?
>
>
> Best regards,
>
> Greg.
>
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Thanks for all the work you put into this patch.
As you say, it would be nice to have a generic framework for this so
that new types of interpolation could be easily added and to be able to
support arbitrary (non-axis-aligned) quadmeshes as well. But that's
even more work -- if we keep waiting for everything we want, we'll never
get it... ;) I agree that Agg probably won't be much help with that.
There are a couple of comments with the patch as it stands -->
There seems to be a gap extrapolating over the left and bottom edge (see
attached screenshot from pcolor_nonuniform.py).
Memory management looks problematic, some of which I think you inherited
from earlier code. For example, arows and acols are never freed.
Personally, I think these temporary buffers should be std::vector's so
they'll be free'd automatically when scope is left. It might also be
nice to move all of the Py_XDECREF's that happen when exceptions are
thrown to either a master try/catch block or an "exit" goto label at the
bottom. The amount of duplication and care required to ensure
everything will be freed by all of the different exit paths is a little
cumbersome.
Also, acols and arows are only used in BILINEAR interpolation, but they
are allocated always.
Once these issues are addressed, it would be great to have someone who
*uses* the nonuniform pcolor functionality (Eric Firing?) have a look at
this patch for any regressions etc.. Assuming none, I'll be happy to
commit it (but I won't be around for a week or so).
Cheers,
Mike
Grégory Lielens wrote:
> Hi all,
>
> here is a patch which implement bilinear interpolation on irregular grid
> ( i.e. it allows NonUniformImage
> to accept both 'nearest' and 'bilinear' interpoaltion, instead of only
> 'nearest'.)
>
> It is not perfect, given the current architecture of the image module I
> think there is not simple way
> to specify other interpolations (except for 'nearest', all
> interpolations are implemented at the AGG level, not in
> matplotlib itself).
> For the same reason, it is not possible to interpolate before colormap
> lookup instead of after (and this
> is usually where the biggest effect is, when dealing with coarse grid).
> However, I think it is already quite useful and the best one ca do
> without a -very- extensive rewrite
> of the matrix map modules....
>
> BTW, I think it also corrects a small bug in the 'nearest'
> interpolation: the last intervals was ignored
> in the CVS version, now it is taken into account.
>
> BOTH nearest and bilinear interpolation do the same for values oustside
> the data grid: they just use boundary values (constant extrapolation).
> I avoided linear extrapolation for 'bilinear', as extrapolation is
> usually dangerous or meaningless (we can go outside the colormap very
> fast)...
>
> I have included a small example showing how both interpolation works....
>
> Any remarks, could this be added before the next release? ;-)
>
>
> Greg.
>
>
>
>
>
> On Mon, 2008年08月11日 at 16:50 +0200, Grégory Lielens wrote:
>
>> On Fri, 2008年08月08日 at 16:05 +0200, Grégory Lielens wrote:
>>
>>> Hello everybody,
>>>
>>> I have sent this message to the user group, but thinking of it, it may be more
>>> relevant to the development mailing list...so here it is again.
>>>
>>>
>>>
>>> We are looking for the best way to plot a waterfall diagram in
>>> Matplotlib. The 2 functions which could be used
>>> to do that are (as far as I have found) imshow and pcolormesh. Here is a
>>> small script that use both to compare the output:
>>>
>>> -----------------
>>>
>>> from pylab import *
>>>
>>>
>>> delta = 0.2
>>> x = arange(-3.0, 3.0, delta)
>>> y = arange(-2.0, 2.0, delta)
>>> X, Y = meshgrid(x, y)
>>> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
>>> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
>>> # difference of Gaussians
>>> Z = 10.0 * (Z2 - Z1)
>>> figure(1)
>>> im = imshow(Z,extent=(-3,3,-2,2))
>>> CS = contour(X, -Y, Z, 6,
>>> colors='k', # negative contours will be dashed by default
>>> )
>>> clabel(CS, fontsize=9, inline=1)
>>> title('Using imshow')
>>> figure(2)
>>> im = pcolormesh(X,-Y,Z)
>>> CS = contour(X, -Y, Z, 6,
>>> colors='k', # negative contours will be dashed by default
>>> )
>>> clabel(CS, fontsize=9, inline=1)
>>> title('Using pcolormesh')
>>> show()
>>>
>>> ---------------------
>>>
>>>
>>> The problem is that we need some of the flexibility of pcolormesh (which
>>> is able to map the matrix of value on any deformed mesh), while
>>> we would like to use the interpolations available in imshow (which
>>> explain why the imshow version is much "smoother" than the pcolormesh
>>> one).
>>>
>>> In fact, what would be needed is not the full flexibility of pcolormesh
>>> (which can map the grid to any kind of shape), we "only" have to deal
>>> with rectangular grids where x- and y- graduations are irregularly spaced.
>>>
>>> Is there a drawing function in Matplotlib which would be able to work
>>> with such a rectangular non-uniform grid?
>>> And if not (and a quick look at the example and the code make me think
>>> that indeed the capability is currently not present),
>>> what about an extension of imshow which would work as this:
>>>
>>> im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
>>> position of the grid's nodes, instead of giving the extend and assuming
>>> uniform spacing.
>>>
>>> Longer term, would a pcolormesh accepting interpolation be possible? The
>>> current behavior, averaging the color of the grids node to get a uniform
>>> cell color,
>>> is quite rough except for a large number of cells...And even then, it
>>> soon shows when you zoom in...
>>>
>>> The best would be to allow the same interpolations as in imshow (or a
>>> subset of it), and also allows to use interpolation before colormap
>>> lookup (or after),
>>> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
>>> specifying Gouraud (interpolation after color
>>> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
>>> Phong is usually much better but also more CPU intensive. Phong is
>>> especially when using discrete colormap, producing banded colors
>>> equivalent to countour lines, while Gouraud does not work in those
>>> cases.
>>>
>>> Of course, the performance will be impacted by some of those
>>> interpolation options, which would degrade performance in animations for
>>> example.... but I think that having the different options available
>>> would be very useful, it allows to have the highest map quality, or have
>>> a "quick and dirty" map depending on situation (grid spacing, type of
>>> map, animation or not, ...).
>>>
>>> Best regards,
>>>
>>> Greg.
>>>
>> I have found a method which implement the proposed extension to imshow:
>> NonUniformImage...
>>
>> However, this image instance support only nearest neighbor
>> interpolation. Trying to set the interpolation (using the
>> set_interpolation method)
>> to something akin imshow throw a "NotImplementedError: Only nearest
>> neighbor supported" exception....
>>
>> So basically I am still stuck, it seems that currently there is no way
>> in matplotlib to plot interpolated
>> colormap on irregular rectangular grid, and even less on arbitrarily
>> mapped grid...
>>
>> Is there any plans to add support for more interpolation in
>> NonUniformImage in the future? Or maybe there is another
>> drawing function that I did not find yet, with this ability?
>>
>>
>> Best regards,
>>
>> Greg.
>>
>>
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>> ------------------------------------------------------------------------
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA
Thanks a lot for reviewing my patch!
I have corrected most of the problems (I think ;-) )
I indeed introduced memory leak, I think it is fixed now, I have also
reorganized the code to avoid duplication of the cleanup code. I used an
helper function instead of the goto, because this cleanup is needed for
normal exit too, so the helper function can come in handy for that.
std::vector would have been nice, but I tried to respect the original
coding style, maybe this could be changed but I guess the original
author should have something to say about it....
Only thing I did not change is the allocation of acols/arows even when
they are not used. It is not difficult to do, but the code will be a
little more complex and I think that the extra memory used is not
significant: The memory for those mapping structure is doubled (1 float
and 1 int vector of size N, instead of a single int vector of size N),
but those mapping structures are an order of magnitude smaller than the
image buffer of size N*N of 4 char anyway...
If one agree that this is to be saved at the (slight) expense of code
conciseness/simplicity, I can add this optimisation...
Thanks for pointing the error at the left/bottom pixel lines, it is
corrected now :-)
And I set interpolation to NEAREST by default, so that the behavior of
NonUniformImage remains the same as before if the user do not specify
otherwise (I forgot to do it in the first patch)...
I include the new patch,
Best regards,
Greg.
On Fri, 2008年08月15日 at 15:45 -0400, Michael Droettboom wrote:
> Thanks for all the work you put into this patch.
>
> As you say, it would be nice to have a generic framework for this so
> that new types of interpolation could be easily added and to be able to
> support arbitrary (non-axis-aligned) quadmeshes as well. But that's
> even more work -- if we keep waiting for everything we want, we'll never
> get it... ;) I agree that Agg probably won't be much help with that.
>
> There are a couple of comments with the patch as it stands -->
>
> There seems to be a gap extrapolating over the left and bottom edge (see
> attached screenshot from pcolor_nonuniform.py).
>
> Memory management looks problematic, some of which I think you inherited
> from earlier code. For example, arows and acols are never freed.
> Personally, I think these temporary buffers should be std::vector's so
> they'll be free'd automatically when scope is left. It might also be
> nice to move all of the Py_XDECREF's that happen when exceptions are
> thrown to either a master try/catch block or an "exit" goto label at the
> bottom. The amount of duplication and care required to ensure
> everything will be freed by all of the different exit paths is a little
> cumbersome.
>
> Also, acols and arows are only used in BILINEAR interpolation, but they
> are allocated always.
>
> Once these issues are addressed, it would be great to have someone who
> *uses* the nonuniform pcolor functionality (Eric Firing?) have a look at
> this patch for any regressions etc.. Assuming none, I'll be happy to
> commit it (but I won't be around for a week or so).
>
> Cheers,
> Mike
>
> Grégory Lielens wrote:
> > Hi all,
> >
> > here is a patch which implement bilinear interpolation on irregular grid
> > ( i.e. it allows NonUniformImage
> > to accept both 'nearest' and 'bilinear' interpoaltion, instead of only
> > 'nearest'.)
> >
> > It is not perfect, given the current architecture of the image module I
> > think there is not simple way
> > to specify other interpolations (except for 'nearest', all
> > interpolations are implemented at the AGG level, not in
> > matplotlib itself).
> > For the same reason, it is not possible to interpolate before colormap
> > lookup instead of after (and this
> > is usually where the biggest effect is, when dealing with coarse grid).
> > However, I think it is already quite useful and the best one ca do
> > without a -very- extensive rewrite
> > of the matrix map modules....
> >
> > BTW, I think it also corrects a small bug in the 'nearest'
> > interpolation: the last intervals was ignored
> > in the CVS version, now it is taken into account.
> >
> > BOTH nearest and bilinear interpolation do the same for values oustside
> > the data grid: they just use boundary values (constant extrapolation).
> > I avoided linear extrapolation for 'bilinear', as extrapolation is
> > usually dangerous or meaningless (we can go outside the colormap very
> > fast)...
> >
> > I have included a small example showing how both interpolation works....
> >
> > Any remarks, could this be added before the next release? ;-)
> >
> >
> > Greg.
> >
> >
> >
> >
> >
> > On Mon, 2008年08月11日 at 16:50 +0200, Grégory Lielens wrote:
> >
> >> On Fri, 2008年08月08日 at 16:05 +0200, Grégory Lielens wrote:
> >>
> >>> Hello everybody,
> >>>
> >>> I have sent this message to the user group, but thinking of it, it may be more
> >>> relevant to the development mailing list...so here it is again.
> >>>
> >>>
> >>>
> >>> We are looking for the best way to plot a waterfall diagram in
> >>> Matplotlib. The 2 functions which could be used
> >>> to do that are (as far as I have found) imshow and pcolormesh. Here is a
> >>> small script that use both to compare the output:
> >>>
> >>> -----------------
> >>>
> >>> from pylab import *
> >>>
> >>>
> >>> delta = 0.2
> >>> x = arange(-3.0, 3.0, delta)
> >>> y = arange(-2.0, 2.0, delta)
> >>> X, Y = meshgrid(x, y)
> >>> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
> >>> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
> >>> # difference of Gaussians
> >>> Z = 10.0 * (Z2 - Z1)
> >>> figure(1)
> >>> im = imshow(Z,extent=(-3,3,-2,2))
> >>> CS = contour(X, -Y, Z, 6,
> >>> colors='k', # negative contours will be dashed by default
> >>> )
> >>> clabel(CS, fontsize=9, inline=1)
> >>> title('Using imshow')
> >>> figure(2)
> >>> im = pcolormesh(X,-Y,Z)
> >>> CS = contour(X, -Y, Z, 6,
> >>> colors='k', # negative contours will be dashed by default
> >>> )
> >>> clabel(CS, fontsize=9, inline=1)
> >>> title('Using pcolormesh')
> >>> show()
> >>>
> >>> ---------------------
> >>>
> >>>
> >>> The problem is that we need some of the flexibility of pcolormesh (which
> >>> is able to map the matrix of value on any deformed mesh), while
> >>> we would like to use the interpolations available in imshow (which
> >>> explain why the imshow version is much "smoother" than the pcolormesh
> >>> one).
> >>>
> >>> In fact, what would be needed is not the full flexibility of pcolormesh
> >>> (which can map the grid to any kind of shape), we "only" have to deal
> >>> with rectangular grids where x- and y- graduations are irregularly spaced.
> >>>
> >>> Is there a drawing function in Matplotlib which would be able to work
> >>> with such a rectangular non-uniform grid?
> >>> And if not (and a quick look at the example and the code make me think
> >>> that indeed the capability is currently not present),
> >>> what about an extension of imshow which would work as this:
> >>>
> >>> im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
> >>> position of the grid's nodes, instead of giving the extend and assuming
> >>> uniform spacing.
> >>>
> >>> Longer term, would a pcolormesh accepting interpolation be possible? The
> >>> current behavior, averaging the color of the grids node to get a uniform
> >>> cell color,
> >>> is quite rough except for a large number of cells...And even then, it
> >>> soon shows when you zoom in...
> >>>
> >>> The best would be to allow the same interpolations as in imshow (or a
> >>> subset of it), and also allows to use interpolation before colormap
> >>> lookup (or after),
> >>> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
> >>> specifying Gouraud (interpolation after color
> >>> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
> >>> Phong is usually much better but also more CPU intensive. Phong is
> >>> especially when using discrete colormap, producing banded colors
> >>> equivalent to countour lines, while Gouraud does not work in those
> >>> cases.
> >>>
> >>> Of course, the performance will be impacted by some of those
> >>> interpolation options, which would degrade performance in animations for
> >>> example.... but I think that having the different options available
> >>> would be very useful, it allows to have the highest map quality, or have
> >>> a "quick and dirty" map depending on situation (grid spacing, type of
> >>> map, animation or not, ...).
> >>>
> >>> Best regards,
> >>>
> >>> Greg.
> >>>
> >> I have found a method which implement the proposed extension to imshow:
> >> NonUniformImage...
> >>
> >> However, this image instance support only nearest neighbor
> >> interpolation. Trying to set the interpolation (using the
> >> set_interpolation method)
> >> to something akin imshow throw a "NotImplementedError: Only nearest
> >> neighbor supported" exception....
> >>
> >> So basically I am still stuck, it seems that currently there is no way
> >> in matplotlib to plot interpolated
> >> colormap on irregular rectangular grid, and even less on arbitrarily
> >> mapped grid...
> >>
> >> Is there any plans to add support for more interpolation in
> >> NonUniformImage in the future? Or maybe there is another
> >> drawing function that I did not find yet, with this ability?
> >>
> >>
> >> Best regards,
> >>
> >> Greg.
> >>
> >>
> >>
> >> -------------------------------------------------------------------------
> >> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> >> Build the coolest Linux based applications with Moblin SDK & win great prizes
> >> Grand prize is a trip for two to an Open Source event anywhere in the world
> >> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> >> _______________________________________________
> >> Matplotlib-devel mailing list
> >> Mat...@li...
> >> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> >>
> >> ------------------------------------------------------------------------
> >>
> >> -------------------------------------------------------------------------
> >> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> >> Build the coolest Linux based applications with Moblin SDK & win great prizes
> >> Grand prize is a trip for two to an Open Source event anywhere in the world
> >> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> >> ------------------------------------------------------------------------
> >>
> >> _______________________________________________
> >> Matplotlib-devel mailing list
> >> Mat...@li...
> >> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________ Matplotlib-devel mailing list Mat...@li... https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Michael Droettboom wrote: > I think we want to use std::vector where possible for new code, and > convert as we go through old code. Manual malloc/free is just too error > prone. (We're forced to do some by interfacing with Python, but it > should be kept to a minimum). My understanding is that this is exactly the issue -- with std::vector and friends you can't both get the data pointer and create a vector with a data pointer, which would be what you'd want to do to interface efficiently with numpy arrays. I've been looking for a way to address this problem, but my C++ is pretty week. maybe smart pointers and/or other boost classes could help. I do have an idea, though. The goal is a set of classes for working with data that are convenient to use with pure C++, but also play with numpy arrays. I imagine a simple vector/array set of classes build on top of a simple object: a reference counted data block. This data block object would simply hold a pointer to a block of data, maybe know how large it is, maybe know what type the data is, and hold a reference count. when that count drops to zero it deletes itself. The vector/array classes built on top of it would use the data block to store their data, and increment/decrement the reference count as need be. This would allow them to share data, have one array that is a subset of another, using the same data block, etc. It should be easy to build numpy arrays from a system like this. Ideally (but I'm not sure how), the data blocks reference counting system could be integrated with Python's, so that when a numpy array was constructed from one, its reference count would be managed by python as well as your C++ code. But maybe that's all just too much work... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no...
It's funny you should mention this. One of the things I realized after SciPy this year is that there is a lot of interfacing of Numpy with C++ (as opposed to C) that could really benefit from a standard C++ wrapper around Numpy. A bunch of different projects all had home-grown, semi-complete solutions. A standard wrapper would be good for all of the reasons you mention (basically resource management), but also to have a cleaner syntax for accessing the elements that would abstract away striding, ordering etc. It may actually be possible to build it around boost::multiarray (the closest thing the upcoming C++-0x standard has to a n-dimensional array). Can't imagine I'll find time for it anytime soon, but it would be fun to work on... Cheers, Mike Christopher Barker wrote: > Michael Droettboom wrote: > >> I think we want to use std::vector where possible for new code, and >> convert as we go through old code. Manual malloc/free is just too error >> prone. (We're forced to do some by interfacing with Python, but it >> should be kept to a minimum). >> > > My understanding is that this is exactly the issue -- with std::vector > and friends you can't both get the data pointer and create a vector with > a data pointer, which would be what you'd want to do to interface > efficiently with numpy arrays. > > I've been looking for a way to address this problem, but my C++ is > pretty week. maybe smart pointers and/or other boost classes could help. > > > I do have an idea, though. The goal is a set of classes for working with > data that are convenient to use with pure C++, but also play with numpy > arrays. > > I imagine a simple vector/array set of classes build on top of a simple > object: a reference counted data block. This data block object would > simply hold a pointer to a block of data, maybe know how large it is, > maybe know what type the data is, and hold a reference count. when that > count drops to zero it deletes itself. > > The vector/array classes built on top of it would use the data block to > store their data, and increment/decrement the reference count as need > be. This would allow them to share data, have one array that is a subset > of another, using the same data block, etc. > > It should be easy to build numpy arrays from a system like this. > > Ideally (but I'm not sure how), the data blocks reference counting > system could be integrated with Python's, so that when a numpy array was > constructed from one, its reference count would be managed by python as > well as your C++ code. > > But maybe that's all just too much work... > > > -Chris > > > >
Michael Droettboom wrote: > It's funny you should mention this. > > One of the things I realized after SciPy this year is that there is a > lot of interfacing of Numpy with C++ (as opposed to C) that could really > benefit from a standard C++ wrapper around Numpy. Absolutely. Though now that you mention it, isn't there one in boost::python? or at least one that worked with Numeric. > projects all had home-grown, semi-complete solutions. > A standard wrapper would be good for all of the reasons you mention > (basically resource management), but also to have a cleaner syntax for > accessing the elements that would abstract away striding, ordering etc. I think I'm thinking of something slightly different. What I'd like is set of array classes that can interface well with numpy, and also be quite usable all by themselves in C++ code that has nothing to do with python. I don't really envision a C++ ndarray -- I generally think of C++ as static enough that I'm happy to define that I need, for example, a NX2 array of doubles. I don't see using a generic array like ndarray that could hold any type, shape, etc. But maybe others do -- it would be pretty cool. I guess to some extent the question is whether you are writing extensions for python, or a C++ that you want to use from C++, and also wrap for python. But maybe both need can be met by the same C++ classes (templates?) > It may actually be possible to build it around boost::multiarray Do you know off hand if multiarray can be constructed from an existing pointer to a data block? > Can't imagine I'll find time for it anytime soon, too bad, but if you do -- I'd love to hear about it! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no...
Christopher Barker wrote: > Michael Droettboom wrote: > >> It's funny you should mention this. >> >> One of the things I realized after SciPy this year is that there is a >> lot of interfacing of Numpy with C++ (as opposed to C) that could really >> benefit from a standard C++ wrapper around Numpy. >> > > Absolutely. Though now that you mention it, isn't there one in > boost::python? or at least one that worked with Numeric. > > Yes, but AFAICT it forces the use of boost::python, at least to convert the object. It would be nicer (in some cases) to have something that could be wrapper-system-agnostic. I spent a little time working on this today, and I was actually pleasantly surprised by how far I got. I have something that exposes Numpy arrays as boost::multi_arrays on the C++ side, and also allows for creating new Numpy-backed arrays from C++. The memory is still managed with Python reference counting, and the references are automatically freed from the constructor. It requires boost and Numpy, of course, but not boost::python. A standalone C++ class for this would perhaps have been nicer, but that's not an afternoon's 200-line hack like this is. I wouldn't really consider this a "wrapper" around Numpy, since it doesn't actually use Numpy for any of the indexing or computation. It simply converts the Numpy description (pointer + strides, etc.) to a boost::multi_array, without copying the underlying data. > >> projects all had home-grown, semi-complete solutions. >> A standard wrapper would be good for all of the reasons you mention >> (basically resource management), but also to have a cleaner syntax for >> accessing the elements that would abstract away striding, ordering etc. >> > > I think I'm thinking of something slightly different. What I'd like is > set of array classes that can interface well with numpy, and also be > quite usable all by themselves in C++ code that has nothing to do with > python. > This should fit that bill. Just templatize all your algorithms to accept anything with the "boost::multi_array" interface. When building extensions for Numpy, pass in a wrapped numpy array -- when not, use a pure boost::multi_array. Of course, the pure boost::multi_array doesn't have reference-counted memory management. Use of boost::shared_ptr may help, but it's not "built-in" -- views of boost::multi_arrays don't automatically keep their parent data alive. It's the usual C++ "you're on your own" approach. > I don't really envision a C++ ndarray -- I generally think of C++ as > static enough that I'm happy to define that I need, for example, a NX2 > array of doubles. I don't see using a generic array like ndarray that > could hold any type, shape, etc. But maybe others do -- it would be > pretty cool. > My approach (being based on boost::multi_array) has a fixed type and number of dimensions at compile time. If you try to pass in a Numpy array that can't be converted to that, an exception is thrown. Personally, I feel that's good enough for most domain-specific algorithms. Theoretically, one could write algorithms that are templatized on the data type and then dispatch at run time to different instantiations of that template -- but my code doesn't do anything like that yet. > I guess to some extent the question is whether you are writing > extensions for python, or a C++ that you want to use from C++, and also > wrap for python. > I'm just doing this out of curiosity rather than to meet a need. We don't have much C++ code at the Institute. I could see this being very handy in matplotlib, however, but the dependency on Boost is probably a showstopper... :( > But maybe both need can be met by the same C++ classes (templates?) > I think so. See above. > >> It may actually be possible to build it around boost::multiarray >> > > Do you know off hand if multiarray can be constructed from an existing > pointer to a data block? > Yep. multi_array_ref seems custom-built for that purpose. > >> Can't imagine I'll find time for it anytime soon, >> I went ahead and created a Google Code project for this here: http://code.google.com/p/numpy-boost/ Just because it's up there doesn't mean it's anything more than an experimental hack. It needs documentation and more complete unit tests... etc... Mike
Hi John, Hi Eric,
In case this has slipped under the radar, I repost my patch for bilinear interp
in NonUniformImage (including the discussion around it). I think Eric is the most concerned by this, but you were both
on holidays when I raised the issue in matplotlib newsgroup.
Best regards,
Greg.
-------------
Thanks a lot for reviewing my patch!
I have corrected most of the problems (I think ;-) )
I indeed introduced memory leak, I think it is fixed now, I have also
reorganized the code to avoid duplication of the cleanup code. I used an
helper function instead of the goto, because this cleanup is needed for
normal exit too, so the helper function can come in handy for that.
std::vector would have been nice, but I tried to respect the original
coding style, maybe this could be changed but I guess the original
author should have something to say about it....
Only thing I did not change is the allocation of acols/arows even when
they are not used. It is not difficult to do, but the code will be a
little more complex and I think that the extra memory used is not
significant: The memory for those mapping structure is doubled (1 float
and 1 int vector of size N, instead of a single int vector of size N),
but those mapping structures are an order of magnitude smaller than the
image buffer of size N*N of 4 char anyway...
If one agree that this is to be saved at the (slight) expense of code
conciseness/simplicity, I can add this optimisation...
Thanks for pointing the error at the left/bottom pixel lines, it is
corrected now :-)
And I set interpolation to NEAREST by default, so that the behavior of
NonUniformImage remains the same as before if the user do not specify
otherwise (I forgot to do it in the first patch)...
I include the new patch,
Best regards,
Greg.
On Fri, 2008年08月15日 at 15:45 -0400, Michael Droettboom wrote:
> Thanks for all the work you put into this patch.
>
> As you say, it would be nice to have a generic framework for this so
> that new types of interpolation could be easily added and to be able to
> support arbitrary (non-axis-aligned) quadmeshes as well. But that's
> even more work -- if we keep waiting for everything we want, we'll never
> get it... ;) I agree that Agg probably won't be much help with that.
>
> There are a couple of comments with the patch as it stands -->
>
> There seems to be a gap extrapolating over the left and bottom edge (see
> attached screenshot from pcolor_nonuniform.py).
>
> Memory management looks problematic, some of which I think you inherited
> from earlier code. For example, arows and acols are never freed.
> Personally, I think these temporary buffers should be std::vector's so
> they'll be free'd automatically when scope is left. It might also be
> nice to move all of the Py_XDECREF's that happen when exceptions are
> thrown to either a master try/catch block or an "exit" goto label at the
> bottom. The amount of duplication and care required to ensure
> everything will be freed by all of the different exit paths is a little
> cumbersome.
>
> Also, acols and arows are only used in BILINEAR interpolation, but they
> are allocated always.
>
> Once these issues are addressed, it would be great to have someone who
> *uses* the nonuniform pcolor functionality (Eric Firing?) have a look at
> this patch for any regressions etc.. Assuming none, I'll be happy to
> commit it (but I won't be around for a week or so).
>
> Cheers,
> Mike
>
> Grégory Lielens wrote:
> > Hi all,
> >
> > here is a patch which implement bilinear interpolation on irregular grid
> > ( i.e. it allows NonUniformImage
> > to accept both 'nearest' and 'bilinear' interpoaltion, instead of only
> > 'nearest'.)
> >
> > It is not perfect, given the current architecture of the image module I
> > think there is not simple way
> > to specify other interpolations (except for 'nearest', all
> > interpolations are implemented at the AGG level, not in
> > matplotlib itself).
> > For the same reason, it is not possible to interpolate before colormap
> > lookup instead of after (and this
> > is usually where the biggest effect is, when dealing with coarse grid).
> > However, I think it is already quite useful and the best one ca do
> > without a -very- extensive rewrite
> > of the matrix map modules....
> >
> > BTW, I think it also corrects a small bug in the 'nearest'
> > interpolation: the last intervals was ignored
> > in the CVS version, now it is taken into account.
> >
> > BOTH nearest and bilinear interpolation do the same for values oustside
> > the data grid: they just use boundary values (constant extrapolation).
> > I avoided linear extrapolation for 'bilinear', as extrapolation is
> > usually dangerous or meaningless (we can go outside the colormap very
> > fast)...
> >
> > I have included a small example showing how both interpolation works....
> >
> > Any remarks, could this be added before the next release? ;-)
> >
> >
> > Greg.
> >
> >
> >
> >
> >
> > On Mon, 2008年08月11日 at 16:50 +0200, Grégory Lielens wrote:
> >
> >> On Fri, 2008年08月08日 at 16:05 +0200, Grégory Lielens wrote:
> >>
> >>> Hello everybody,
> >>>
> >>> I have sent this message to the user group, but thinking of it, it may be more
> >>> relevant to the development mailing list...so here it is again.
> >>>
> >>>
> >>>
> >>> We are looking for the best way to plot a waterfall diagram in
> >>> Matplotlib. The 2 functions which could be used
> >>> to do that are (as far as I have found) imshow and pcolormesh. Here is a
> >>> small script that use both to compare the output:
> >>>
> >>> -----------------
> >>>
> >>> from pylab import *
> >>>
> >>>
> >>> delta = 0.2
> >>> x = arange(-3.0, 3.0, delta)
> >>> y = arange(-2.0, 2.0, delta)
> >>> X, Y = meshgrid(x, y)
> >>> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
> >>> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
> >>> # difference of Gaussians
> >>> Z = 10.0 * (Z2 - Z1)
> >>> figure(1)
> >>> im = imshow(Z,extent=(-3,3,-2,2))
> >>> CS = contour(X, -Y, Z, 6,
> >>> colors='k', # negative contours will be dashed by default
> >>> )
> >>> clabel(CS, fontsize=9, inline=1)
> >>> title('Using imshow')
> >>> figure(2)
> >>> im = pcolormesh(X,-Y,Z)
> >>> CS = contour(X, -Y, Z, 6,
> >>> colors='k', # negative contours will be dashed by default
> >>> )
> >>> clabel(CS, fontsize=9, inline=1)
> >>> title('Using pcolormesh')
> >>> show()
> >>>
> >>> ---------------------
> >>>
> >>>
> >>> The problem is that we need some of the flexibility of pcolormesh (which
> >>> is able to map the matrix of value on any deformed mesh), while
> >>> we would like to use the interpolations available in imshow (which
> >>> explain why the imshow version is much "smoother" than the pcolormesh
> >>> one).
> >>>
> >>> In fact, what would be needed is not the full flexibility of pcolormesh
> >>> (which can map the grid to any kind of shape), we "only" have to deal
> >>> with rectangular grids where x- and y- graduations are irregularly spaced.
> >>>
> >>> Is there a drawing function in Matplotlib which would be able to work
> >>> with such a rectangular non-uniform grid?
> >>> And if not (and a quick look at the example and the code make me think
> >>> that indeed the capability is currently not present),
> >>> what about an extension of imshow which would work as this:
> >>>
> >>> im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
> >>> position of the grid's nodes, instead of giving the extend and assuming
> >>> uniform spacing.
> >>>
> >>> Longer term, would a pcolormesh accepting interpolation be possible? The
> >>> current behavior, averaging the color of the grids node to get a uniform
> >>> cell color,
> >>> is quite rough except for a large number of cells...And even then, it
> >>> soon shows when you zoom in...
> >>>
> >>> The best would be to allow the same interpolations as in imshow (or a
> >>> subset of it), and also allows to use interpolation before colormap
> >>> lookup (or after),
> >>> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
> >>> specifying Gouraud (interpolation after color
> >>> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
> >>> Phong is usually much better but also more CPU intensive. Phong is
> >>> especially when using discrete colormap, producing banded colors
> >>> equivalent to countour lines, while Gouraud does not work in those
> >>> cases.
> >>>
> >>> Of course, the performance will be impacted by some of those
> >>> interpolation options, which would degrade performance in animations for
> >>> example.... but I think that having the different options available
> >>> would be very useful, it allows to have the highest map quality, or have
> >>> a "quick and dirty" map depending on situation (grid spacing, type of
> >>> map, animation or not, ...).
> >>>
> >>> Best regards,
> >>>
> >>> Greg.
> >>>
> >> I have found a method which implement the proposed extension to imshow:
> >> NonUniformImage...
> >>
> >> However, this image instance support only nearest neighbor
> >> interpolation. Trying to set the interpolation (using the
> >> set_interpolation method)
> >> to something akin imshow throw a "NotImplementedError: Only nearest
> >> neighbor supported" exception....
> >>
> >> So basically I am still stuck, it seems that currently there is no way
> >> in matplotlib to plot interpolated
> >> colormap on irregular rectangular grid, and even less on arbitrarily
> >> mapped grid...
> >>
> >> Is there any plans to add support for more interpolation in
> >> NonUniformImage in the future? Or maybe there is another
> >> drawing function that I did not find yet, with this ability?
> >>
> >>
> >> Best regards,
> >>
> >> Greg.
I'm going to again defer the real question of the functionality and
interface etc. to Eric and/or John etc., though it seems good in general.
I just have a couple of follow on comments related to implementation
details below.
Grégory Lielens wrote:
> std::vector would have been nice, but I tried to respect the original
> coding style, maybe this could be changed but I guess the original
> author should have something to say about it....
>
I think we want to use std::vector where possible for new code, and
convert as we go through old code. Manual malloc/free is just too error
prone. (We're forced to do some by interfacing with Python, but it
should be kept to a minimum). Maybe this should be added to the coding
guidelines?
> Only thing I did not change is the allocation of acols/arows even when
> they are not used. It is not difficult to do, but the code will be a
> little more complex and I think that the extra memory used is not
> significant: The memory for those mapping structure is doubled (1 float
> and 1 int vector of size N, instead of a single int vector of size N),
> but those mapping structures are an order of magnitude smaller than the
> image buffer of size N*N of 4 char anyway...
>
> If one agree that this is to be saved at the (slight) expense of code
> conciseness/simplicity, I can add this optimisation...
>
I don't understand the complexity. Isn't it as simple as moving:
acols = reinterpret_cast<float*>(PyMem_Malloc(sizeof(float)*cols));
(and the following NULL check) inside of the "if (interpolation ==
Image::BILINEAR)" block?
Heap allocations can be expensive, so it would be great to minimize them.
> Thanks for pointing the error at the left/bottom pixel lines, it is
> corrected now :-)
>
Great!
> And I set interpolation to NEAREST by default, so that the behavior of
> NonUniformImage remains the same as before if the user do not specify
> otherwise (I forgot to do it in the first patch)...
>
Sounds reasonable -- though perhaps we want to be consistent instead
with the standard (uniform) images, which default to the
"image.interpolation" setting in matplotlibrc. Of course,
"image.interpolation" has a bunch of options that nonuniform doesn't
currently support... What do others think?
Cheers,
Mike
>
> I include the new patch,
>
> Best regards,
>
> Greg.
>
> On Fri, 2008年08月15日 at 15:45 -0400, Michael Droettboom wrote:
>
>> Thanks for all the work you put into this patch.
>>
>> As you say, it would be nice to have a generic framework for this so
>> that new types of interpolation could be easily added and to be able to
>> support arbitrary (non-axis-aligned) quadmeshes as well. But that's
>> even more work -- if we keep waiting for everything we want, we'll never
>> get it... ;) I agree that Agg probably won't be much help with that.
>>
>> There are a couple of comments with the patch as it stands -->
>>
>> There seems to be a gap extrapolating over the left and bottom edge (see
>> attached screenshot from pcolor_nonuniform.py).
>>
>> Memory management looks problematic, some of which I think you inherited
>> from earlier code. For example, arows and acols are never freed.
>> Personally, I think these temporary buffers should be std::vector's so
>> they'll be free'd automatically when scope is left. It might also be
>> nice to move all of the Py_XDECREF's that happen when exceptions are
>> thrown to either a master try/catch block or an "exit" goto label at the
>> bottom. The amount of duplication and care required to ensure
>> everything will be freed by all of the different exit paths is a little
>> cumbersome.
>>
>> Also, acols and arows are only used in BILINEAR interpolation, but they
>> are allocated always.
>>
>> Once these issues are addressed, it would be great to have someone who
>> *uses* the nonuniform pcolor functionality (Eric Firing?) have a look at
>> this patch for any regressions etc.. Assuming none, I'll be happy to
>> commit it (but I won't be around for a week or so).
>>
>> Cheers,
>> Mike
>>
>> Grégory Lielens wrote:
>>
>>> Hi all,
>>>
>>> here is a patch which implement bilinear interpolation on irregular grid
>>> ( i.e. it allows NonUniformImage
>>> to accept both 'nearest' and 'bilinear' interpoaltion, instead of only
>>> 'nearest'.)
>>>
>>> It is not perfect, given the current architecture of the image module I
>>> think there is not simple way
>>> to specify other interpolations (except for 'nearest', all
>>> interpolations are implemented at the AGG level, not in
>>> matplotlib itself).
>>> For the same reason, it is not possible to interpolate before colormap
>>> lookup instead of after (and this
>>> is usually where the biggest effect is, when dealing with coarse grid).
>>> However, I think it is already quite useful and the best one ca do
>>> without a -very- extensive rewrite
>>> of the matrix map modules....
>>>
>>> BTW, I think it also corrects a small bug in the 'nearest'
>>> interpolation: the last intervals was ignored
>>> in the CVS version, now it is taken into account.
>>>
>>> BOTH nearest and bilinear interpolation do the same for values oustside
>>> the data grid: they just use boundary values (constant extrapolation).
>>> I avoided linear extrapolation for 'bilinear', as extrapolation is
>>> usually dangerous or meaningless (we can go outside the colormap very
>>> fast)...
>>>
>>> I have included a small example showing how both interpolation works....
>>>
>>> Any remarks, could this be added before the next release? ;-)
>>>
>>>
>>> Greg.
>>>
>>>
>>>
>>>
>>>
>>> On Mon, 2008年08月11日 at 16:50 +0200, Grégory Lielens wrote:
>>>
>>>
>>>> On Fri, 2008年08月08日 at 16:05 +0200, Grégory Lielens wrote:
>>>>
>>>>
>>>>> Hello everybody,
>>>>>
>>>>> I have sent this message to the user group, but thinking of it, it may be more
>>>>> relevant to the development mailing list...so here it is again.
>>>>>
>>>>>
>>>>>
>>>>> We are looking for the best way to plot a waterfall diagram in
>>>>> Matplotlib. The 2 functions which could be used
>>>>> to do that are (as far as I have found) imshow and pcolormesh. Here is a
>>>>> small script that use both to compare the output:
>>>>>
>>>>> -----------------
>>>>>
>>>>> from pylab import *
>>>>>
>>>>>
>>>>> delta = 0.2
>>>>> x = arange(-3.0, 3.0, delta)
>>>>> y = arange(-2.0, 2.0, delta)
>>>>> X, Y = meshgrid(x, y)
>>>>> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
>>>>> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
>>>>> # difference of Gaussians
>>>>> Z = 10.0 * (Z2 - Z1)
>>>>> figure(1)
>>>>> im = imshow(Z,extent=(-3,3,-2,2))
>>>>> CS = contour(X, -Y, Z, 6,
>>>>> colors='k', # negative contours will be dashed by default
>>>>> )
>>>>> clabel(CS, fontsize=9, inline=1)
>>>>> title('Using imshow')
>>>>> figure(2)
>>>>> im = pcolormesh(X,-Y,Z)
>>>>> CS = contour(X, -Y, Z, 6,
>>>>> colors='k', # negative contours will be dashed by default
>>>>> )
>>>>> clabel(CS, fontsize=9, inline=1)
>>>>> title('Using pcolormesh')
>>>>> show()
>>>>>
>>>>> ---------------------
>>>>>
>>>>>
>>>>> The problem is that we need some of the flexibility of pcolormesh (which
>>>>> is able to map the matrix of value on any deformed mesh), while
>>>>> we would like to use the interpolations available in imshow (which
>>>>> explain why the imshow version is much "smoother" than the pcolormesh
>>>>> one).
>>>>>
>>>>> In fact, what would be needed is not the full flexibility of pcolormesh
>>>>> (which can map the grid to any kind of shape), we "only" have to deal
>>>>> with rectangular grids where x- and y- graduations are irregularly spaced.
>>>>>
>>>>> Is there a drawing function in Matplotlib which would be able to work
>>>>> with such a rectangular non-uniform grid?
>>>>> And if not (and a quick look at the example and the code make me think
>>>>> that indeed the capability is currently not present),
>>>>> what about an extension of imshow which would work as this:
>>>>>
>>>>> im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
>>>>> position of the grid's nodes, instead of giving the extend and assuming
>>>>> uniform spacing.
>>>>>
>>>>> Longer term, would a pcolormesh accepting interpolation be possible? The
>>>>> current behavior, averaging the color of the grids node to get a uniform
>>>>> cell color,
>>>>> is quite rough except for a large number of cells...And even then, it
>>>>> soon shows when you zoom in...
>>>>>
>>>>> The best would be to allow the same interpolations as in imshow (or a
>>>>> subset of it), and also allows to use interpolation before colormap
>>>>> lookup (or after),
>>>>> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
>>>>> specifying Gouraud (interpolation after color
>>>>> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
>>>>> Phong is usually much better but also more CPU intensive. Phong is
>>>>> especially when using discrete colormap, producing banded colors
>>>>> equivalent to countour lines, while Gouraud does not work in those
>>>>> cases.
>>>>>
>>>>> Of course, the performance will be impacted by some of those
>>>>> interpolation options, which would degrade performance in animations for
>>>>> example.... but I think that having the different options available
>>>>> would be very useful, it allows to have the highest map quality, or have
>>>>> a "quick and dirty" map depending on situation (grid spacing, type of
>>>>> map, animation or not, ...).
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Greg.
>>>>>
>>>>>
>>>> I have found a method which implement the proposed extension to imshow:
>>>> NonUniformImage...
>>>>
>>>> However, this image instance support only nearest neighbor
>>>> interpolation. Trying to set the interpolation (using the
>>>> set_interpolation method)
>>>> to something akin imshow throw a "NotImplementedError: Only nearest
>>>> neighbor supported" exception....
>>>>
>>>> So basically I am still stuck, it seems that currently there is no way
>>>> in matplotlib to plot interpolated
>>>> colormap on irregular rectangular grid, and even less on arbitrarily
>>>> mapped grid...
>>>>
>>>> Is there any plans to add support for more interpolation in
>>>> NonUniformImage in the future? Or maybe there is another
>>>> drawing function that I did not find yet, with this ability?
>>>>
>>>>
>>>> Best regards,
>>>>
>>>> Greg.
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Mat...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA
Grégory Lielens wrote:
> Thanks a lot for reviewing my patch!
> I have corrected most of the problems (I think ;-) )
> I indeed introduced memory leak, I think it is fixed now, I have also
> reorganized the code to avoid duplication of the cleanup code. I used an
> helper function instead of the goto, because this cleanup is needed for
> normal exit too, so the helper function can come in handy for that.
> std::vector would have been nice, but I tried to respect the original
> coding style, maybe this could be changed but I guess the original
> author should have something to say about it....
> Only thing I did not change is the allocation of acols/arows even when
> they are not used. It is not difficult to do, but the code will be a
> little more complex and I think that the extra memory used is not
> significant: The memory for those mapping structure is doubled (1 float
> and 1 int vector of size N, instead of a single int vector of size N),
> but those mapping structures are an order of magnitude smaller than the
> image buffer of size N*N of 4 char anyway...
>
> If one agree that this is to be saved at the (slight) expense of code
> conciseness/simplicity, I can add this optimisation...
>
> Thanks for pointing the error at the left/bottom pixel lines, it is
> corrected now :-)
>
> And I set interpolation to NEAREST by default, so that the behavior of
> NonUniformImage remains the same as before if the user do not specify
> otherwise (I forgot to do it in the first patch)...
>
>
> I include the new patch,
Gregory,
I am sorry to be late in joining in, but I have a basic question about
what you are trying to do. It arises from the fact that there are two
styles of image-like plots:
1) In the original Image and NonuniformImage, the colors and locations
of "pixel" (or patch or subregion or whatever) centers are specified.
2) In pcolor, quadmesh, and pcolorimage, the patch *boundaries* are
specified. pcolorimage is my modification of the original
NonuniformImage code to work with boundaries. I left the original in
place, since presumably it meets some people's needs, even though it
does not meet mine.
When the grid is uniform, it doesn't really make any difference; but
when the grid is nonuniform, then I don't think style #1 makes much
sense for pcolor-style plotting, in which there is no interpolation;
there is no unique way to specify the boundaries, so it is up to
the plotting routine, and that is bad.
For an image, it is clear how the interpolation should work: the color
values are given at the centers of the "pixels" (meaning on the original
image grid, not on the screen or other output device grid), which are
uniformly spaced, and interpolated between.
Exactly what is your patch supposed to do for the nonuniform case? Would
you describe and demonstrate it, please, for a simple case with a 2x2
array of colors and 3x3 arrays for boundary x and y?
Thanks.
Eric
>
> Best regards,
>
> Greg.
>
> On Fri, 2008年08月15日 at 15:45 -0400, Michael Droettboom wrote:
>> Thanks for all the work you put into this patch.
>>
>> As you say, it would be nice to have a generic framework for this so
>> that new types of interpolation could be easily added and to be able to
>> support arbitrary (non-axis-aligned) quadmeshes as well. But that's
>> even more work -- if we keep waiting for everything we want, we'll never
>> get it... ;) I agree that Agg probably won't be much help with that.
>>
>> There are a couple of comments with the patch as it stands -->
>>
>> There seems to be a gap extrapolating over the left and bottom edge (see
>> attached screenshot from pcolor_nonuniform.py).
>>
>> Memory management looks problematic, some of which I think you inherited
>> from earlier code. For example, arows and acols are never freed.
>> Personally, I think these temporary buffers should be std::vector's so
>> they'll be free'd automatically when scope is left. It might also be
>> nice to move all of the Py_XDECREF's that happen when exceptions are
>> thrown to either a master try/catch block or an "exit" goto label at the
>> bottom. The amount of duplication and care required to ensure
>> everything will be freed by all of the different exit paths is a little
>> cumbersome.
>>
>> Also, acols and arows are only used in BILINEAR interpolation, but they
>> are allocated always.
>>
>> Once these issues are addressed, it would be great to have someone who
>> *uses* the nonuniform pcolor functionality (Eric Firing?) have a look at
>> this patch for any regressions etc.. Assuming none, I'll be happy to
>> commit it (but I won't be around for a week or so).
>>
>> Cheers,
>> Mike
>>
>> Grégory Lielens wrote:
>>> Hi all,
>>>
>>> here is a patch which implement bilinear interpolation on irregular grid
>>> ( i.e. it allows NonUniformImage
>>> to accept both 'nearest' and 'bilinear' interpoaltion, instead of only
>>> 'nearest'.)
>>>
>>> It is not perfect, given the current architecture of the image module I
>>> think there is not simple way
>>> to specify other interpolations (except for 'nearest', all
>>> interpolations are implemented at the AGG level, not in
>>> matplotlib itself).
>>> For the same reason, it is not possible to interpolate before colormap
>>> lookup instead of after (and this
>>> is usually where the biggest effect is, when dealing with coarse grid).
>>> However, I think it is already quite useful and the best one ca do
>>> without a -very- extensive rewrite
>>> of the matrix map modules....
>>>
>>> BTW, I think it also corrects a small bug in the 'nearest'
>>> interpolation: the last intervals was ignored
>>> in the CVS version, now it is taken into account.
>>>
>>> BOTH nearest and bilinear interpolation do the same for values oustside
>>> the data grid: they just use boundary values (constant extrapolation).
>>> I avoided linear extrapolation for 'bilinear', as extrapolation is
>>> usually dangerous or meaningless (we can go outside the colormap very
>>> fast)...
>>>
>>> I have included a small example showing how both interpolation works....
>>>
>>> Any remarks, could this be added before the next release? ;-)
>>>
>>>
>>> Greg.
>>>
>>>
>>>
>>>
>>>
>>> On Mon, 2008年08月11日 at 16:50 +0200, Grégory Lielens wrote:
>>>
>>>> On Fri, 2008年08月08日 at 16:05 +0200, Grégory Lielens wrote:
>>>>
>>>>> Hello everybody,
>>>>>
>>>>> I have sent this message to the user group, but thinking of it, it may be more
>>>>> relevant to the development mailing list...so here it is again.
>>>>>
>>>>>
>>>>>
>>>>> We are looking for the best way to plot a waterfall diagram in
>>>>> Matplotlib. The 2 functions which could be used
>>>>> to do that are (as far as I have found) imshow and pcolormesh. Here is a
>>>>> small script that use both to compare the output:
>>>>>
>>>>> -----------------
>>>>>
>>>>> from pylab import *
>>>>>
>>>>>
>>>>> delta = 0.2
>>>>> x = arange(-3.0, 3.0, delta)
>>>>> y = arange(-2.0, 2.0, delta)
>>>>> X, Y = meshgrid(x, y)
>>>>> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
>>>>> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
>>>>> # difference of Gaussians
>>>>> Z = 10.0 * (Z2 - Z1)
>>>>> figure(1)
>>>>> im = imshow(Z,extent=(-3,3,-2,2))
>>>>> CS = contour(X, -Y, Z, 6,
>>>>> colors='k', # negative contours will be dashed by default
>>>>> )
>>>>> clabel(CS, fontsize=9, inline=1)
>>>>> title('Using imshow')
>>>>> figure(2)
>>>>> im = pcolormesh(X,-Y,Z)
>>>>> CS = contour(X, -Y, Z, 6,
>>>>> colors='k', # negative contours will be dashed by default
>>>>> )
>>>>> clabel(CS, fontsize=9, inline=1)
>>>>> title('Using pcolormesh')
>>>>> show()
>>>>>
>>>>> ---------------------
>>>>>
>>>>>
>>>>> The problem is that we need some of the flexibility of pcolormesh (which
>>>>> is able to map the matrix of value on any deformed mesh), while
>>>>> we would like to use the interpolations available in imshow (which
>>>>> explain why the imshow version is much "smoother" than the pcolormesh
>>>>> one).
>>>>>
>>>>> In fact, what would be needed is not the full flexibility of pcolormesh
>>>>> (which can map the grid to any kind of shape), we "only" have to deal
>>>>> with rectangular grids where x- and y- graduations are irregularly spaced.
>>>>>
>>>>> Is there a drawing function in Matplotlib which would be able to work
>>>>> with such a rectangular non-uniform grid?
>>>>> And if not (and a quick look at the example and the code make me think
>>>>> that indeed the capability is currently not present),
>>>>> what about an extension of imshow which would work as this:
>>>>>
>>>>> im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
>>>>> position of the grid's nodes, instead of giving the extend and assuming
>>>>> uniform spacing.
>>>>>
>>>>> Longer term, would a pcolormesh accepting interpolation be possible? The
>>>>> current behavior, averaging the color of the grids node to get a uniform
>>>>> cell color,
>>>>> is quite rough except for a large number of cells...And even then, it
>>>>> soon shows when you zoom in...
>>>>>
>>>>> The best would be to allow the same interpolations as in imshow (or a
>>>>> subset of it), and also allows to use interpolation before colormap
>>>>> lookup (or after),
>>>>> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
>>>>> specifying Gouraud (interpolation after color
>>>>> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
>>>>> Phong is usually much better but also more CPU intensive. Phong is
>>>>> especially when using discrete colormap, producing banded colors
>>>>> equivalent to countour lines, while Gouraud does not work in those
>>>>> cases.
>>>>>
>>>>> Of course, the performance will be impacted by some of those
>>>>> interpolation options, which would degrade performance in animations for
>>>>> example.... but I think that having the different options available
>>>>> would be very useful, it allows to have the highest map quality, or have
>>>>> a "quick and dirty" map depending on situation (grid spacing, type of
>>>>> map, animation or not, ...).
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Greg.
>>>>>
>>>> I have found a method which implement the proposed extension to imshow:
>>>> NonUniformImage...
>>>>
>>>> However, this image instance support only nearest neighbor
>>>> interpolation. Trying to set the interpolation (using the
>>>> set_interpolation method)
>>>> to something akin imshow throw a "NotImplementedError: Only nearest
>>>> neighbor supported" exception....
>>>>
>>>> So basically I am still stuck, it seems that currently there is no way
>>>> in matplotlib to plot interpolated
>>>> colormap on irregular rectangular grid, and even less on arbitrarily
>>>> mapped grid...
>>>>
>>>> Is there any plans to add support for more interpolation in
>>>> NonUniformImage in the future? Or maybe there is another
>>>> drawing function that I did not find yet, with this ability?
>>>>
>>>>
>>>> Best regards,
>>>>
>>>> Greg.
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Mat...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Mat...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________ Matplotlib-devel mailing list Mat...@li... https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>> ------------------------------------------------------------------------
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Hi Eric, I think I understand the two approaches you mention, I have about the same mental distinction although with a different spin, maybe because I come from finite element/data visualisation instead of image manipulation: I see basically two type of 2D data that can be visualized as a colormapped image: a) point data, where you provide a collection of values with associated coordinates. Algorithm is then fully responsible for computing the "best" value to associate to other coordinates (not provided by the user, but nonetheless needed to build a full image, because there was a change of resolution or just some data lacking). One can distinguish sub categories: are the points given on - an regular rectangular grid, * - a regular hexagonal/triangular grid, - an irregular rectangular grid * - another pattern? like mapped rectangular grid, ... - at arbitrary locations, without pattern for now in matplotlib only *-marked subcategories are supported, AFAIK b) cell data, where you provide a collection of cell (polygons), with vertex coordinates and color-mapped data. Again we can distinguish su categories: - cell-wise data, no interpolation needed * - or vertex-wise data, interpolation needed -averaging on the cell -proximity based interpolation -(bi)linear interpolation -higher order interpolation and also it can be subdivided using cell geometry criterium - cell part of a regular rectangular grid * - cell part of an irregular rectangular grid * - cell part of a mapped rectangular * - arbitray cells, but paving the plane (vertex coodinate + connectivity table) - fully arbitrary cells, not necessarily continuously connected Again, AFAIK for now the *-marked categories are currently suported in matplotlib. In addition to those categories, one additional thing can be considered when any kind of interpolation/extrapolation is performed: Does the interpolation happen on the data, and then the interpolated data is mapped to color (phong mapping/interpolation/shading in matlab speach), or are is the interpolation performed on colors after the initial user-provided data have been mapped (gouraud mapping/interpolation/shading in matlab speach). Some equivalences exists between cell based image and point based image, in particular my patch can be seen as irregular rectangular grid patches bith bilinear interpolation, or point data with bilinear interpolation. It is more of the second one, as for example in cell-data, one can expect to be able to set line styles for cell boundaries, while in point data this would have no sense. I do not see the fact that for point data the interpolation is choosen by the algorithm as a bad thing: in fact, if one use point data, it is because there is no cell information as such, so the fact that the algorithm choose automatically the boundaries is an advantage. For example, your NonUniformImage is a point-data image generator, which work for non-uniform rectangular grids. It associate with each pixel the data of the closest (using non-cartesian distance, but a "manhatan distance" given by abs(x-x0) + abs(y-y0)) available point data. My patch added bilinear interpolation, using x and y bilinear interpolation between the four closest. Other type of interpolation are possible, but more complex to implement. The context in which we want to use NonUniformImage is not for image processing, but for data visualisation: we have measurement data dependent on two variables (RPM and frequency), and a classic way to present such data is what is known in the field (acoustic radiation) as a "waterfall diagram". However, the samples are not necessarily taken on a regular grid. They are taken at best on an irregular grid (especially for the RPM), and at worst using a non-obvious pattern. There is no cell data here, boudaries are not really relevant, the only thing that matter is to provide an attractive and visualy-easy-to-parse image of 2D data which was sampled (or simulated) on a set of points. In this context both interpolation (nearest, usefull because if present only measured data extended in the best way so that they cover the whole figure, without any actual interpolation), and bilinear (show smoother variations, making more easy to visually detect some pattern and interresting RPM/FREQUENCY regions ) are interresting for our users... A small example as you requested (a is withing the data, b is outside, c is completely oustide) c 11 12 13 a b 21 22 23 31 32 33 Nearest interpolation (patched (small error), but already present): color at a = color at 12 or 13 or 22 or 23, whichever is closest to a color at b = color at 13 or 23, whichever is closest to b color at c = color at 13 Bilinear interpolation (new): S=(x23-x12)(y12-y23) color at a = color(12) * (x23-xa)(ya-y23)/S + color(13) * (xa-x22)(ya-y22)/S + color(22) * (x13-xa)(y13-ya)/S + color(23) * (xa-x12)(y12-ya)/S color at b = color(13) * (yb-y23)/(y13-y23) + color(23) * (y13-yb)/(y13-y23) color at c = color at 13 As I mentioned in my first message, to complete the full array of image generation (point data, cell data, various interpolation scheme and geometric distribution of points or cell geometries, interpolation before or after the color mapping) is a tremendous work. Not even matlab has it completed, although it is far more complete than matploblib for now and can be considered a worthy goal... But I think this linear interpolation after color mapping on irregular rectangular point data is already a usefull addition ;-) Regards, Greg. Quoting Eric Firing <ef...@ha...>: > Grégory Lielens wrote: >> Thanks a lot for reviewing my patch! >> I have corrected most of the problems (I think ;-) ) >> I indeed introduced memory leak, I think it is fixed now, I have also >> reorganized the code to avoid duplication of the cleanup code. I used an >> helper function instead of the goto, because this cleanup is needed for >> normal exit too, so the helper function can come in handy for that. >> std::vector would have been nice, but I tried to respect the original >> coding style, maybe this could be changed but I guess the original >> author should have something to say about it.... >> Only thing I did not change is the allocation of acols/arows even when >> they are not used. It is not difficult to do, but the code will be a >> little more complex and I think that the extra memory used is not >> significant: The memory for those mapping structure is doubled (1 float >> and 1 int vector of size N, instead of a single int vector of size N), >> but those mapping structures are an order of magnitude smaller than the >> image buffer of size N*N of 4 char anyway... >> >> If one agree that this is to be saved at the (slight) expense of code >> conciseness/simplicity, I can add this optimisation... >> >> Thanks for pointing the error at the left/bottom pixel lines, it is >> corrected now :-) >> >> And I set interpolation to NEAREST by default, so that the behavior of >> NonUniformImage remains the same as before if the user do not specify >> otherwise (I forgot to do it in the first patch)... >> >> >> I include the new patch, > > Gregory, > > I am sorry to be late in joining in, but I have a basic question about > what you are trying to do. It arises from the fact that there are two > styles of image-like plots: > > 1) In the original Image and NonuniformImage, the colors and locations > of "pixel" (or patch or subregion or whatever) centers are specified. > > 2) In pcolor, quadmesh, and pcolorimage, the patch *boundaries* are > specified. pcolorimage is my modification of the original > NonuniformImage code to work with boundaries. I left the original in > place, since presumably it meets some people's needs, even though it > does not meet mine. > > When the grid is uniform, it doesn't really make any difference; but > when the grid is nonuniform, then I don't think style #1 makes much > sense for pcolor-style plotting, in which there is no interpolation; > there is no unique way to specify the boundaries, so it is up to > the plotting routine, and that is bad. > > For an image, it is clear how the interpolation should work: the color > values are given at the centers of the "pixels" (meaning on the original > image grid, not on the screen or other output device grid), which are > uniformly spaced, and interpolated between. > > Exactly what is your patch supposed to do for the nonuniform case? Would > you describe and demonstrate it, please, for a simple case with a 2x2 > array of colors and 3x3 arrays for boundary x and y? > > Thanks. > > Eric
gl...@ff... wrote: > Hi Eric, > > I think I understand the two approaches you mention, I have about the > same mental distinction although with a different spin, maybe because > I come from finite element/data visualisation instead of image > manipulation: I see basically two type of 2D data that can be > visualized as a colormapped image: > > a) point data, where you provide a collection of values with > associated coordinates. Algorithm is then fully responsible for > computing the "best" value to associate to other coordinates (not > provided by the user, but nonetheless needed to build a full image, > because there was a change of resolution or just some data lacking). > One can distinguish sub categories: are the points given on > - an regular rectangular grid, * > - a regular hexagonal/triangular grid, > - an irregular rectangular grid * > - another pattern? like mapped rectangular grid, ... > - at arbitrary locations, without pattern > for now in matplotlib only *-marked subcategories are supported, AFAIK > b) cell data, where you provide a collection of cell (polygons), with > vertex coordinates and color-mapped data. Again we can distinguish su > categories: > - cell-wise data, no interpolation needed * > - or vertex-wise data, interpolation needed > -averaging on the cell > -proximity based interpolation > -(bi)linear interpolation > -higher order interpolation > and also it can be subdivided using cell geometry criterium > - cell part of a regular rectangular grid * > - cell part of an irregular rectangular grid * > - cell part of a mapped rectangular * > - arbitray cells, but paving the plane (vertex coodinate + > connectivity table) > - fully arbitrary cells, not necessarily continuously connected > Again, AFAIK for now the *-marked categories are currently suported > in matplotlib. > > In addition to those categories, one additional thing can be > considered when any kind of interpolation/extrapolation is performed: > Does the interpolation happen on the data, and then the interpolated > data is mapped to color (phong mapping/interpolation/shading in matlab > speach), or are is the interpolation performed on colors after the > initial user-provided data have been mapped (gouraud > mapping/interpolation/shading in matlab speach). > > Some equivalences exists between cell based image and point based > image, in particular my patch can be seen as irregular rectangular > grid patches bith bilinear interpolation, or point data with bilinear > interpolation. It is more of the second one, as for example in > cell-data, one can expect to be able to set line styles for cell > boundaries, while in point data this would have no sense. > > I do not see the fact that for point data the interpolation is choosen > by the algorithm as a bad thing: in fact, if one use point data, it is > because there is no cell information as such, so the fact that the > algorithm choose automatically the boundaries is an advantage. > > For example, your NonUniformImage is a point-data image generator, > which work for non-uniform rectangular grids. It associate with each > pixel the data of the closest (using non-cartesian distance, but a > "manhatan distance" given by abs(x-x0) + abs(y-y0)) available point > data. > My patch added bilinear interpolation, using x and y bilinear > interpolation between the four closest. Other type of interpolation > are possible, but more complex to implement. > > The context in which we want to use NonUniformImage is not for image > processing, but for data visualisation: we have measurement data > dependent on two variables (RPM and frequency), and a classic way to > present such data is what is known in the field (acoustic radiation) > as a "waterfall diagram". However, the samples are not necessarily > taken on a regular grid. They are taken at best on an irregular grid > (especially for the RPM), and at worst using a non-obvious pattern. > There is no cell data here, boudaries are not really relevant, the > only thing that matter is to provide an attractive and > visualy-easy-to-parse image of 2D data which was sampled (or > simulated) on a set of points. In this context both interpolation > (nearest, usefull because if present only measured data extended in > the best way so that they cover the whole figure, without any actual > interpolation), and bilinear (show smoother variations, making more > easy to visually detect some pattern and interresting RPM/FREQUENCY > regions ) are interresting for our users... > > A small example as you requested (a is withing the data, b is outside, > c is completely oustide) > c > 11 12 13 > a b > 21 22 23 > 31 32 33 > > Nearest interpolation (patched (small error), but already present): > color at a = color at 12 or 13 or 22 or 23, whichever is closest to a > color at b = color at 13 or 23, whichever is closest to b > color at c = color at 13 > Bilinear interpolation (new): > S=(x23-x12)(y12-y23) > color at a = color(12) * (x23-xa)(ya-y23)/S > + color(13) * (xa-x22)(ya-y22)/S > + color(22) * (x13-xa)(y13-ya)/S > + color(23) * (xa-x12)(y12-ya)/S > color at b = color(13) * (yb-y23)/(y13-y23) > + color(23) * (y13-yb)/(y13-y23) > color at c = color at 13 > > > As I mentioned in my first message, to complete the full array of > image generation (point data, cell data, various interpolation scheme > and geometric distribution of points or cell geometries, interpolation > before or after the color mapping) is a tremendous work. Not even > matlab has it completed, although it is far more complete than > matploblib for now and can be considered a worthy goal... But I think > this linear interpolation after color mapping on irregular rectangular > point data is already a usefull addition ;-) > > Regards, > > Greg. Greg, Thank you for your very clear and complete explanation. I have committed your patch with only a few modifications: 0) I fixed a bug with non-agg backends by setting im.is_grayscale. 1) I changed the handling of the interpolation kwarg. The default is still 'nearest'. 2) I took Mike's suggestion to allocate acols and arows only if needed. 3) I renamed pcolor_nonuniform to image_nonuniform, modified it to show both types of interpolation, and added it to the set run by backend_driver.py. This is the most minimal test; one could write a whole test suite to exercise various inputs and options. Among the questions that occur to me: 1) Should the functionality be exposed as an Axes method, and from there as a pyplot function? This could be done by integrating it into imshow, either via the argument signature or via kwargs for X and Y, or it could be a separate method. 2) If X and Y are in fact uniform, should the displayed image be the same as the present imshow? The big difference now is the boundary: NonUniformImage extends boundary values indefinitely while Image uses the background color for anything outside the image. I find the NonUniformImage behavior disconcerting. 3) Should caching be added to NonUniformImage? Every other class in the image.py module uses it. 4) Can we do anything with the internal function naming, at least, to make the distinction between point data functions and cell data functions? My thought was to use "image" for point data and "pcolor" for cell data, but this may reflect my ignorance and particular usage patterns from matlab days. What I would do is rename the _image.cpp pcolor function to nonuniform_image, or something like that, to distinguish it more clearly from pcolor2, which works with cell data. Eric
> Greg, > > Thank you for your very clear and complete explanation. > > I have committed your patch with only a few modifications: > > 0) I fixed a bug with non-agg backends by setting im.is_grayscale. > > 1) I changed the handling of the interpolation kwarg. The default is > still 'nearest'. > > 2) I took Mike's suggestion to allocate acols and arows only if needed. > > 3) I renamed pcolor_nonuniform to image_nonuniform, modified it to show > both types of interpolation, and added it to the set run by > backend_driver.py. This is the most minimal test; one could write a > whole test suite to exercise various inputs and options. Thanks for this! > > Among the questions that occur to me: > > 1) Should the functionality be exposed as an Axes method, and from there > as a pyplot function? This could be done by integrating it into imshow, > either via the argument signature or via kwargs for X and Y, or it could > be a separate method. it would be nice, at first I tried to find kwargs in imshow to do exactly that (specify X and Y grid positions), and I did find NonUniformImage after quite a lot of documentation digging and exploring the example directory... > > 2) If X and Y are in fact uniform, should the displayed image be the > same as the present imshow? The big difference now is the boundary: > NonUniformImage extends boundary values indefinitely while Image uses > the background color for anything outside the image. I find the > NonUniformImage behavior disconcerting. You are right, it should be the same. The extrapolation present in NonUniformImage was disconcerting to me at first too, but I now enjoy it: it is the same as what we do in our code for user-provided parameter dependent data: linear interpolation between points, and flat extrapolation outside the min/max: this avoid nasty surprises (linear extrapolation) and sidestep the problem of providing a "default" value outside min/max bounds. But here it is not a strong argument, as background color/full transparency is a good default value in the matplotlib case. I think the best approach would be to give a special kwarg extrapolation= None/flat (and eventually add other options if needed). > > 3) Should caching be added to NonUniformImage? Every other class in the > image.py module uses it. Hum, there I can not help: I do not know what caching is...Is it an optimisation to avoid unneeded re-render, when image size is kept unchanged? If it is, I guess optimisation can never hurt ;-) > > 4) Can we do anything with the internal function naming, at least, to > make the distinction between point data functions and cell data > functions? My thought was to use "image" for point data and "pcolor" > for cell data, but this may reflect my ignorance and particular usage > patterns from matlab days. What I would do is rename the _image.cpp > pcolor function to nonuniform_image, or something like that, to > distinguish it more clearly from pcolor2, which works with cell data. Some rationalization would ne nice indeed. I do not have strong prefences on this, image and pcolor would work fine for me, as any other names used consistently. It could be the opportunity to prepare the addition of other point/cell data function (like delaunay triangulation of arbitrary point data in matlab, or non-uniform cell data with color interpolation on the cell. For the "phong" mapping (mapping data to color after interpolating the data for each pixel ), I am affraid the work would be even greater, as I do not see how to use the imshow filters in this case....except maybe using a fake "greyscale" image working on the data directly, and they mapping the greyscale image to color image using the color mapper pixel per pixel... Performance would suffer, though... Regards, Greg.