matplotlib-devel Mailing List for matplotlib

On Thu, Aug 28, 2008 at 4:18 PM, John Hunter <jd...@gm...> wrote:
> On Thu, Aug 28, 2008 at 2:57 PM, Jae-Joon Lee <lee...@gm...> wrote:
>
>> First of all, I borrowed this idea from the PyX which is in GPL.
>> Although there is little of copying, other than the basic idea, I'm
>> not 100% sure if this could be BSD-compatible.
>
> I think it is fine to borrow the idea; what we need to do is a clean
> room implementation with no copying. You can best answer that, so if
> you tell us your patch is cleanly implemented, we can accept it.
>
> JDH
>
Thanks for the response.
Well, the only part I borrowed from PyX is TeX related commands they
use (there is not much of implementation as far as TeX-related code is
concerned). From their code, I learned the meaning and usage of the
following TeX commands
\newbox
\setbox
\immediate\write16
And I used the same TeX commands in my code.
But I personally think this is not a (code) copy.
Other than this, the code is clean.
Regards,
-JJ

On Thu, Aug 28, 2008 at 2:57 PM, Jae-Joon Lee <lee...@gm...> wrote:
> First of all, I borrowed this idea from the PyX which is in GPL.
> Although there is little of copying, other than the basic idea, I'm
> not 100% sure if this could be BSD-compatible.
I think it is fine to borrow the idea; what we need to do is a clean
room implementation with no copying. You can best answer that, so if
you tell us your patch is cleanly implemented, we can accept it.
JDH

Hello,
AFAIK, current backends (I only tested agg, pdf, and ps) do not
properly respect the text baseline when text is rendered using TeX.
The get_text_width_height_descent() method in Agg and PS backends
simply return 0 for the descent value. While PDF backend uses the
dviread module to figure out correct descent values, there are cases
this does not work well (e.g. $\frac{1}{2}\pi$).
As an example, the attached figure shows the result for the Agg
backend. In all cases, the texts are placed at (0,0) with
baseline-alignment. Leftmost one is when usetex=False, which has a
correct baseline. The middle one is when usetex=True. It is bottom
aligned, which is not intended. The rightmost one is also when
usetex=True but after the patch I describe below.
First of all, I borrowed this idea from the PyX which is in GPL.
Although there is little of copying, other than the basic idea, I'm
not 100% sure if this could be BSD-compatible.
Anyhow, the idea is that you can have LateX to print out the width,
height, and descent (=depth) of a given text by enclosing the text in
a box. For example,
 \newbox\MatplotlibBox%
 \setbox\MatplotlibBox=\hbox{$\frac{1}{2}\pi$}%
 \copy\MatplotlibBox
 \immediate\write16{MatplotlibBox:\the\wd\MatplotlibBox,\the\ht\MatplotlibBox,\the\dp\MatplotlibBox}%
I define a newbox (called MatplotlibBox) which encloses
$\frac{1}{2}\pi$. And then print out the width, height and depth of
the box.
Attached is a patch of a texmanager.py which utilize above method to
figure out the dimension of the text. The template string to generate
a ".tex" file is slightly modified. After latex is run, the
dimensional information of the text is extracted and saved in
".baseline" file and get_text_width_height_descent() method is added
under the TexManager class, which reads in the ".baseline" file and
return its content. (you need to empty out the tex.cache directory for
this work correctly).
A backend can simply call the get_text_width_height_descent() of
texmanager (a simple patch for the Agg backend is attached). I also
tested this with PS and PDF backends and they worked out fine.
So if the license issue is okay, I wonder if this patch can be
reviewed and applied (after any necessary modifications) to improve
the baseline handling in matploltib.
Regards,
-JJ

Michael Droettboom wrote:
> I think we want to use std::vector where possible for new code, and 
> convert as we go through old code. Manual malloc/free is just too error 
> prone. (We're forced to do some by interfacing with Python, but it 
> should be kept to a minimum).
My understanding is that this is exactly the issue -- with std::vector 
and friends you can't both get the data pointer and create a vector with 
a data pointer, which would be what you'd want to do to interface 
efficiently with numpy arrays.
I've been looking for a way to address this problem, but my C++ is 
pretty week. maybe smart pointers and/or other boost classes could help.
I do have an idea, though. The goal is a set of classes for working with 
data that are convenient to use with pure C++, but also play with numpy 
arrays.
I imagine a simple vector/array set of classes build on top of a simple 
object: a reference counted data block. This data block object would 
simply hold a pointer to a block of data, maybe know how large it is, 
maybe know what type the data is, and hold a reference count. when that 
count drops to zero it deletes itself.
The vector/array classes built on top of it would use the data block to 
store their data, and increment/decrement the reference count as need 
be. This would allow them to share data, have one array that is a subset 
of another, using the same data block, etc.
It should be easy to build numpy arrays from a system like this.
Ideally (but I'm not sure how), the data blocks reference counting 
system could be integrated with Python's, so that when a numpy array was 
constructed from one, its reference count would be managed by python as 
well as your C++ code.
But maybe that's all just too much work...
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

Michael Droettboom wrote:
> On RHEL4, I get the following message:
> 
> This OpenGL does not support framebuffer objects
> 
> I understand that I'm probably just suffering from a relatively old 
> Mesa/OpenGL stack here. And obviously, just because some older systems 
> won't support this is not a reason to not include it as an optional 
> backend.
> 
> Can you possibly send a screen shot to the list for those of us who 
> can't see it? Is the quadmesh interpolated? That would be a huge 
> improvement over anything we can do currently.
> 
> As for developing a full OpenGL backend, note that the process of 
> writing a backend has been greatly simplified as of 0.98.x (there's only 
> about four rendering methods to implement now, with three more optional 
> ones to re-implement for performance reasons). You can start by 
> looking at backend_bases.py. I'm happy to help with any questions you 
> have as you go along, but as I don't have a working pyglet/OpenGL, I 
> can't be of much help for testing.
> 
I'll let you know if I need any help. 0.98 definitely seems simpler and 
I think Pyglet will handle a lot of the context stuff to make things 
simpler as well.
Your error on RHEL4 indicates more that it doesn't support our 
particular approach, which uses offscreen rendering to render to an 
image which is then blitted in. It's an expedient approach to get a 
drop in quadmesh replacement. A full pyglet backend would not suffer 
from this limitation.
I don't have this hooked up fully within matplotlib yet, so right now it 
just generates a test image with a 2000x2000 mesh of quadrilaterals 
which is displayed using imshow(). (simple example attached.) I have 
not looked at interpolation or even turned on antialiasing (should just 
be one or two function calls). I can tell you that most of the time is 
spent generating the list of vertices in a form suitable for OpenGL 
QUAD_LISTS. The actual rendering and handing off to the graphics card 
is *really* fast.
I have a couple other things on my plate ATM, but hopefully next week 
some time I will get back to this, as I really, really, really want/need 
to pursue the opengl stuff further.
Ryan
-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

On RHEL4, I get the following message:
 This OpenGL does not support framebuffer objects
I understand that I'm probably just suffering from a relatively old 
Mesa/OpenGL stack here. And obviously, just because some older systems 
won't support this is not a reason to not include it as an optional backend.
Can you possibly send a screen shot to the list for those of us who 
can't see it? Is the quadmesh interpolated? That would be a huge 
improvement over anything we can do currently.
As for developing a full OpenGL backend, note that the process of 
writing a backend has been greatly simplified as of 0.98.x (there's only 
about four rendering methods to implement now, with three more optional 
ones to re-implement for performance reasons). You can start by 
looking at backend_bases.py. I'm happy to help with any questions you 
have as you go along, but as I don't have a working pyglet/OpenGL, I 
can't be of much help for testing.
Cheers,
Mike
Ryan May wrote:
> Paul Kienzle wrote:
> 
>> Hi,
>>
>> There was a recent discussion about opengl and matplotlib in the
>> context of matplotlib rendering speeds.
>>
>> At the scipy sprints we put together a proof of concept renderer
>> for quad meshes using the opengl frame buffer object, which we
>> then render as a matplotlib image. Once the data is massaged
>> to the correct form we were seeing rendering speeds of about
>> 10 million quads per second. See below.
>>
>> Using this technique we can get the advantage of opengl without
>> having to write a full backend. Please let me know if you have
>> time to contribute to writing a more complete backend.
>> 
>
> I know that I (as the other contributer :) ) plan on making this into a 
> full backend provided:
>
> 1) The quadmesh code can be shown yield the gains when integrated into 
> matplotlib more fully
>
> 2) I find the time (which is a matter of *when*, not if)
>
> I'm certainly finding thus far that pyglet makes it a lot easier to do a 
> full backend than some of the other python->opengl methods I'd explored 
> in the past.
>
> Ryan
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Thanks!
It works as I expected.
-JJ
On Thu, Aug 28, 2008 at 8:43 AM, Michael Droettboom <md...@st...> wrote:
> This should now be fixed in SVN r6052.
>
> Cheers,
> Mike
>
> Jae-Joon Lee wrote:
>>
>> Hi,
>>
>> The clip_on and clip_box arguments (and maybe clip_path also) in
>> plot() command seem to have no effect. For example,
>>
>> In [29]: p, =plot([1,2,3], clip_on=False)
>>
>> In [30]: p.get_clip_on()
>> Out[30]: True
>>
>>
>> It seems that the line object is created with the given arguments but
>> gets overwritten later when it is added to the axes (Axes.add_line),
>>
>> def add_line(self, line):
>> '''
>> Add a :class:`~matplotlib.lines.Line2D` to the list of plot
>> lines
>> '''
>> self._set_artist_props(line)
>> line.set_clip_path(self.patch)
>>
>> "line.set_clip_path(self.patch)" update clip_path, clip_box, and clip_on.
>>
>> So, isn't it better to do something explicitly when these arguments
>> are ignored? Maybe to print out some warnings or even raise an
>> exception, or do not call set_clip_path when clip_* argument is given,
>> or etc.?
>>
>> A slightly related behavior I found unexpected is that set_clip_path()
>> and set_clip_rect() method also update clip_on.
>>
>> Regards,
>>
>> -JJ
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>> challenge
>> Build the coolest Linux based applications with Moblin SDK & win great
>> prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the
>> world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>
>

I'm going to again defer the real question of the functionality and 
interface etc. to Eric and/or John etc., though it seems good in general.
I just have a couple of follow on comments related to implementation 
details below.
Grégory Lielens wrote:
> std::vector would have been nice, but I tried to respect the original
> coding style, maybe this could be changed but I guess the original
> author should have something to say about it....
> 
I think we want to use std::vector where possible for new code, and 
convert as we go through old code. Manual malloc/free is just too error 
prone. (We're forced to do some by interfacing with Python, but it 
should be kept to a minimum). Maybe this should be added to the coding 
guidelines?
> Only thing I did not change is the allocation of acols/arows even when
> they are not used. It is not difficult to do, but the code will be a
> little more complex and I think that the extra memory used is not
> significant: The memory for those mapping structure is doubled (1 float
> and 1 int vector of size N, instead of a single int vector of size N), 
> but those mapping structures are an order of magnitude smaller than the
> image buffer of size N*N of 4 char anyway...
> 
> If one agree that this is to be saved at the (slight) expense of code
> conciseness/simplicity, I can add this optimisation...
> 
I don't understand the complexity. Isn't it as simple as moving:
 acols = reinterpret_cast<float*>(PyMem_Malloc(sizeof(float)*cols));
(and the following NULL check) inside of the "if (interpolation == 
Image::BILINEAR)" block?
Heap allocations can be expensive, so it would be great to minimize them.
> Thanks for pointing the error at the left/bottom pixel lines, it is
> corrected now :-)
> 
Great!
> And I set interpolation to NEAREST by default, so that the behavior of
> NonUniformImage remains the same as before if the user do not specify
> otherwise (I forgot to do it in the first patch)...
> 
Sounds reasonable -- though perhaps we want to be consistent instead 
with the standard (uniform) images, which default to the 
"image.interpolation" setting in matplotlibrc. Of course, 
"image.interpolation" has a bunch of options that nonuniform doesn't 
currently support... What do others think?
Cheers,
Mike
>
> I include the new patch,
>
> Best regards,
>
> Greg.
>
> On Fri, 2008年08月15日 at 15:45 -0400, Michael Droettboom wrote:
> 
>> Thanks for all the work you put into this patch.
>>
>> As you say, it would be nice to have a generic framework for this so 
>> that new types of interpolation could be easily added and to be able to 
>> support arbitrary (non-axis-aligned) quadmeshes as well. But that's 
>> even more work -- if we keep waiting for everything we want, we'll never 
>> get it... ;) I agree that Agg probably won't be much help with that. 
>>
>> There are a couple of comments with the patch as it stands -->
>>
>> There seems to be a gap extrapolating over the left and bottom edge (see 
>> attached screenshot from pcolor_nonuniform.py).
>>
>> Memory management looks problematic, some of which I think you inherited 
>> from earlier code. For example, arows and acols are never freed. 
>> Personally, I think these temporary buffers should be std::vector's so 
>> they'll be free'd automatically when scope is left. It might also be 
>> nice to move all of the Py_XDECREF's that happen when exceptions are 
>> thrown to either a master try/catch block or an "exit" goto label at the 
>> bottom. The amount of duplication and care required to ensure 
>> everything will be freed by all of the different exit paths is a little 
>> cumbersome.
>>
>> Also, acols and arows are only used in BILINEAR interpolation, but they 
>> are allocated always.
>>
>> Once these issues are addressed, it would be great to have someone who 
>> *uses* the nonuniform pcolor functionality (Eric Firing?) have a look at 
>> this patch for any regressions etc.. Assuming none, I'll be happy to 
>> commit it (but I won't be around for a week or so).
>>
>> Cheers,
>> Mike
>>
>> Grégory Lielens wrote:
>> 
>>> Hi all,
>>>
>>> here is a patch which implement bilinear interpolation on irregular grid
>>> ( i.e. it allows NonUniformImage
>>> to accept both 'nearest' and 'bilinear' interpoaltion, instead of only
>>> 'nearest'.)
>>>
>>> It is not perfect, given the current architecture of the image module I
>>> think there is not simple way
>>> to specify other interpolations (except for 'nearest', all
>>> interpolations are implemented at the AGG level, not in
>>> matplotlib itself).
>>> For the same reason, it is not possible to interpolate before colormap
>>> lookup instead of after (and this 
>>> is usually where the biggest effect is, when dealing with coarse grid).
>>> However, I think it is already quite useful and the best one ca do
>>> without a -very- extensive rewrite
>>> of the matrix map modules....
>>>
>>> BTW, I think it also corrects a small bug in the 'nearest'
>>> interpolation: the last intervals was ignored
>>> in the CVS version, now it is taken into account.
>>>
>>> BOTH nearest and bilinear interpolation do the same for values oustside
>>> the data grid: they just use boundary values (constant extrapolation).
>>> I avoided linear extrapolation for 'bilinear', as extrapolation is
>>> usually dangerous or meaningless (we can go outside the colormap very
>>> fast)...
>>>
>>> I have included a small example showing how both interpolation works....
>>>
>>> Any remarks, could this be added before the next release? ;-)
>>>
>>>
>>> Greg.
>>>
>>>
>>>
>>>
>>>
>>> On Mon, 2008年08月11日 at 16:50 +0200, Grégory Lielens wrote:
>>> 
>>> 
>>>> On Fri, 2008年08月08日 at 16:05 +0200, Grégory Lielens wrote:
>>>> 
>>>> 
>>>>> Hello everybody,
>>>>>
>>>>> I have sent this message to the user group, but thinking of it, it may be more
>>>>> relevant to the development mailing list...so here it is again.
>>>>>
>>>>>
>>>>>
>>>>> We are looking for the best way to plot a waterfall diagram in
>>>>> Matplotlib. The 2 functions which could be used 
>>>>> to do that are (as far as I have found) imshow and pcolormesh. Here is a
>>>>> small script that use both to compare the output:
>>>>>
>>>>> -----------------
>>>>>
>>>>> from pylab import *
>>>>>
>>>>>
>>>>> delta = 0.2
>>>>> x = arange(-3.0, 3.0, delta)
>>>>> y = arange(-2.0, 2.0, delta)
>>>>> X, Y = meshgrid(x, y)
>>>>> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
>>>>> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
>>>>> # difference of Gaussians
>>>>> Z = 10.0 * (Z2 - Z1)
>>>>> figure(1)
>>>>> im = imshow(Z,extent=(-3,3,-2,2))
>>>>> CS = contour(X, -Y, Z, 6,
>>>>> colors='k', # negative contours will be dashed by default
>>>>> )
>>>>> clabel(CS, fontsize=9, inline=1)
>>>>> title('Using imshow')
>>>>> figure(2)
>>>>> im = pcolormesh(X,-Y,Z)
>>>>> CS = contour(X, -Y, Z, 6,
>>>>> colors='k', # negative contours will be dashed by default
>>>>> )
>>>>> clabel(CS, fontsize=9, inline=1)
>>>>> title('Using pcolormesh')
>>>>> show()
>>>>>
>>>>> ---------------------
>>>>>
>>>>>
>>>>> The problem is that we need some of the flexibility of pcolormesh (which
>>>>> is able to map the matrix of value on any deformed mesh), while
>>>>> we would like to use the interpolations available in imshow (which
>>>>> explain why the imshow version is much "smoother" than the pcolormesh
>>>>> one).
>>>>>
>>>>> In fact, what would be needed is not the full flexibility of pcolormesh
>>>>> (which can map the grid to any kind of shape), we "only" have to deal
>>>>> with rectangular grids where x- and y- graduations are irregularly spaced.
>>>>>
>>>>> Is there a drawing function in Matplotlib which would be able to work
>>>>> with such a rectangular non-uniform grid?
>>>>> And if not (and a quick look at the example and the code make me think 
>>>>> that indeed the capability is currently not present),
>>>>> what about an extension of imshow which would work as this:
>>>>> 
>>>>> im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
>>>>> position of the grid's nodes, instead of giving the extend and assuming
>>>>> uniform spacing.
>>>>>
>>>>> Longer term, would a pcolormesh accepting interpolation be possible? The
>>>>> current behavior, averaging the color of the grids node to get a uniform
>>>>> cell color, 
>>>>> is quite rough except for a large number of cells...And even then, it
>>>>> soon shows when you zoom in...
>>>>>
>>>>> The best would be to allow the same interpolations as in imshow (or a
>>>>> subset of it), and also allows to use interpolation before colormap
>>>>> lookup (or after), 
>>>>> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
>>>>> specifying Gouraud (interpolation after color
>>>>> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
>>>>> Phong is usually much better but also more CPU intensive. Phong is
>>>>> especially when using discrete colormap, producing banded colors
>>>>> equivalent to countour lines, while Gouraud does not work in those
>>>>> cases.
>>>>>
>>>>> Of course, the performance will be impacted by some of those
>>>>> interpolation options, which would degrade performance in animations for
>>>>> example.... but I think that having the different options available
>>>>> would be very useful, it allows to have the highest map quality, or have
>>>>> a "quick and dirty" map depending on situation (grid spacing, type of
>>>>> map, animation or not, ...).
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Greg.
>>>>> 
>>>>> 
>>>> I have found a method which implement the proposed extension to imshow:
>>>> NonUniformImage...
>>>>
>>>> However, this image instance support only nearest neighbor
>>>> interpolation. Trying to set the interpolation (using the
>>>> set_interpolation method)
>>>> to something akin imshow throw a "NotImplementedError: Only nearest
>>>> neighbor supported" exception....
>>>>
>>>> So basically I am still stuck, it seems that currently there is no way
>>>> in matplotlib to plot interpolated
>>>> colormap on irregular rectangular grid, and even less on arbitrarily
>>>> mapped grid...
>>>>
>>>> Is there any plans to add support for more interpolation in
>>>> NonUniformImage in the future? Or maybe there is another 
>>>> drawing function that I did not find yet, with this ability?
>>>>
>>>>
>>>> Best regards,
>>>>
>>>> Greg.
>>>> 
>>>> ------------------------------------------------------------------------
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Mat...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Sorry about that, and thanks for your investigation. It's easy enough 
to change the default. 
I think (and it was a while ago, so I may be misremembering) I probably 
made that change myself in error. (There used to be two completely 
separate code paths for doing step plots, that I merged into one). 
Any objections? Is it possible that change was made deliberately (to 
conform with matlab or gnuplot etc.)?
Cheers,
Mike
Neil Crighton wrote:
> Now I see there are more options in 0.98 - 'steps-pre', 'steps-post',
> 'steps-mid'. The default should be steps-post for backwards
> compatibility. In 0.98.3 the default is steps-pre. And sorry for the
> testy tone of the previous email :)
>
> Neil
>
> 2008年8月28日 Neil Crighton <nei...@gm...>:
> 
>> linestyle='steps' has changed behaviour between 0.91.2 and 0.98.3. The
>> 'step' between two points used to move horizontally and then
>> vertically from the left point neighbouring right point, now it moves
>> vertically then horizontally.
>>
>> Was this change intentional? I hope not, because I've just spent the
>> past hour working out it was the reason for my plotting routines not
>> working properly :-/
>>
>> Neil
>>
>> 
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Thu, Aug 28, 2008 at 08:17:26AM -0500, John Hunter wrote:
> On Thu, Aug 28, 2008 at 6:19 AM, Sandro Tosi <mat...@gm...> wrote:
> > Enthought suite is used a lot in scientific area, so mpl and enth
> > almost share their users, so have it enabled would be a plus, but we
> > are mainly interested in generate "no harm", so we'd like to ask you
> > to confirm that enabling that support won't brake anything (but indeed
> > provide a valuable asset for users).
> Well, one thing to make sure of is that you do not install the version
> of traits that ships with matplotlib, since it will break other
> enthought packages.
Debian and Ubuntu are doing the right thing here currently. This is very
nice, as it provides a clean upgrade path for traits under Debian
derivatives.
Gaël

Hi John, Hi Eric,
In case this has slipped under the radar, I repost my patch for bilinear interp 
in NonUniformImage (including the discussion around it). I think Eric is the most concerned by this, but you were both 
on holidays when I raised the issue in matplotlib newsgroup.
Best regards,
Greg.
-------------
Thanks a lot for reviewing my patch!
I have corrected most of the problems (I think ;-) )
I indeed introduced memory leak, I think it is fixed now, I have also
reorganized the code to avoid duplication of the cleanup code. I used an
helper function instead of the goto, because this cleanup is needed for
normal exit too, so the helper function can come in handy for that.
std::vector would have been nice, but I tried to respect the original
coding style, maybe this could be changed but I guess the original
author should have something to say about it....
Only thing I did not change is the allocation of acols/arows even when
they are not used. It is not difficult to do, but the code will be a
little more complex and I think that the extra memory used is not
significant: The memory for those mapping structure is doubled (1 float
and 1 int vector of size N, instead of a single int vector of size N), 
but those mapping structures are an order of magnitude smaller than the
image buffer of size N*N of 4 char anyway...
If one agree that this is to be saved at the (slight) expense of code
conciseness/simplicity, I can add this optimisation...
Thanks for pointing the error at the left/bottom pixel lines, it is
corrected now :-)
And I set interpolation to NEAREST by default, so that the behavior of
NonUniformImage remains the same as before if the user do not specify
otherwise (I forgot to do it in the first patch)...
I include the new patch,
Best regards,
Greg.
On Fri, 2008年08月15日 at 15:45 -0400, Michael Droettboom wrote:
> Thanks for all the work you put into this patch.
> 
> As you say, it would be nice to have a generic framework for this so 
> that new types of interpolation could be easily added and to be able to 
> support arbitrary (non-axis-aligned) quadmeshes as well. But that's 
> even more work -- if we keep waiting for everything we want, we'll never 
> get it... ;) I agree that Agg probably won't be much help with that. 
> 
> There are a couple of comments with the patch as it stands -->
> 
> There seems to be a gap extrapolating over the left and bottom edge (see 
> attached screenshot from pcolor_nonuniform.py).
> 
> Memory management looks problematic, some of which I think you inherited 
> from earlier code. For example, arows and acols are never freed. 
> Personally, I think these temporary buffers should be std::vector's so 
> they'll be free'd automatically when scope is left. It might also be 
> nice to move all of the Py_XDECREF's that happen when exceptions are 
> thrown to either a master try/catch block or an "exit" goto label at the 
> bottom. The amount of duplication and care required to ensure 
> everything will be freed by all of the different exit paths is a little 
> cumbersome.
> 
> Also, acols and arows are only used in BILINEAR interpolation, but they 
> are allocated always.
> 
> Once these issues are addressed, it would be great to have someone who 
> *uses* the nonuniform pcolor functionality (Eric Firing?) have a look at 
> this patch for any regressions etc.. Assuming none, I'll be happy to 
> commit it (but I won't be around for a week or so).
> 
> Cheers,
> Mike
> 
> Grégory Lielens wrote:
> > Hi all,
> >
> > here is a patch which implement bilinear interpolation on irregular grid
> > ( i.e. it allows NonUniformImage
> > to accept both 'nearest' and 'bilinear' interpoaltion, instead of only
> > 'nearest'.)
> >
> > It is not perfect, given the current architecture of the image module I
> > think there is not simple way
> > to specify other interpolations (except for 'nearest', all
> > interpolations are implemented at the AGG level, not in
> > matplotlib itself).
> > For the same reason, it is not possible to interpolate before colormap
> > lookup instead of after (and this 
> > is usually where the biggest effect is, when dealing with coarse grid).
> > However, I think it is already quite useful and the best one ca do
> > without a -very- extensive rewrite
> > of the matrix map modules....
> >
> > BTW, I think it also corrects a small bug in the 'nearest'
> > interpolation: the last intervals was ignored
> > in the CVS version, now it is taken into account.
> >
> > BOTH nearest and bilinear interpolation do the same for values oustside
> > the data grid: they just use boundary values (constant extrapolation).
> > I avoided linear extrapolation for 'bilinear', as extrapolation is
> > usually dangerous or meaningless (we can go outside the colormap very
> > fast)...
> >
> > I have included a small example showing how both interpolation works....
> >
> > Any remarks, could this be added before the next release? ;-)
> >
> >
> > Greg.
> >
> >
> >
> >
> >
> > On Mon, 2008年08月11日 at 16:50 +0200, Grégory Lielens wrote:
> > 
> >> On Fri, 2008年08月08日 at 16:05 +0200, Grégory Lielens wrote:
> >> 
> >>> Hello everybody,
> >>>
> >>> I have sent this message to the user group, but thinking of it, it may be more
> >>> relevant to the development mailing list...so here it is again.
> >>>
> >>>
> >>>
> >>> We are looking for the best way to plot a waterfall diagram in
> >>> Matplotlib. The 2 functions which could be used 
> >>> to do that are (as far as I have found) imshow and pcolormesh. Here is a
> >>> small script that use both to compare the output:
> >>>
> >>> -----------------
> >>>
> >>> from pylab import *
> >>>
> >>>
> >>> delta = 0.2
> >>> x = arange(-3.0, 3.0, delta)
> >>> y = arange(-2.0, 2.0, delta)
> >>> X, Y = meshgrid(x, y)
> >>> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
> >>> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
> >>> # difference of Gaussians
> >>> Z = 10.0 * (Z2 - Z1)
> >>> figure(1)
> >>> im = imshow(Z,extent=(-3,3,-2,2))
> >>> CS = contour(X, -Y, Z, 6,
> >>> colors='k', # negative contours will be dashed by default
> >>> )
> >>> clabel(CS, fontsize=9, inline=1)
> >>> title('Using imshow')
> >>> figure(2)
> >>> im = pcolormesh(X,-Y,Z)
> >>> CS = contour(X, -Y, Z, 6,
> >>> colors='k', # negative contours will be dashed by default
> >>> )
> >>> clabel(CS, fontsize=9, inline=1)
> >>> title('Using pcolormesh')
> >>> show()
> >>>
> >>> ---------------------
> >>>
> >>>
> >>> The problem is that we need some of the flexibility of pcolormesh (which
> >>> is able to map the matrix of value on any deformed mesh), while
> >>> we would like to use the interpolations available in imshow (which
> >>> explain why the imshow version is much "smoother" than the pcolormesh
> >>> one).
> >>>
> >>> In fact, what would be needed is not the full flexibility of pcolormesh
> >>> (which can map the grid to any kind of shape), we "only" have to deal
> >>> with rectangular grids where x- and y- graduations are irregularly spaced.
> >>>
> >>> Is there a drawing function in Matplotlib which would be able to work
> >>> with such a rectangular non-uniform grid?
> >>> And if not (and a quick look at the example and the code make me think 
> >>> that indeed the capability is currently not present),
> >>> what about an extension of imshow which would work as this:
> >>> 
> >>> im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
> >>> position of the grid's nodes, instead of giving the extend and assuming
> >>> uniform spacing.
> >>>
> >>> Longer term, would a pcolormesh accepting interpolation be possible? The
> >>> current behavior, averaging the color of the grids node to get a uniform
> >>> cell color, 
> >>> is quite rough except for a large number of cells...And even then, it
> >>> soon shows when you zoom in...
> >>>
> >>> The best would be to allow the same interpolations as in imshow (or a
> >>> subset of it), and also allows to use interpolation before colormap
> >>> lookup (or after), 
> >>> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
> >>> specifying Gouraud (interpolation after color
> >>> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
> >>> Phong is usually much better but also more CPU intensive. Phong is
> >>> especially when using discrete colormap, producing banded colors
> >>> equivalent to countour lines, while Gouraud does not work in those
> >>> cases.
> >>>
> >>> Of course, the performance will be impacted by some of those
> >>> interpolation options, which would degrade performance in animations for
> >>> example.... but I think that having the different options available
> >>> would be very useful, it allows to have the highest map quality, or have
> >>> a "quick and dirty" map depending on situation (grid spacing, type of
> >>> map, animation or not, ...).
> >>>
> >>> Best regards,
> >>>
> >>> Greg.
> >>> 
> >> I have found a method which implement the proposed extension to imshow:
> >> NonUniformImage...
> >>
> >> However, this image instance support only nearest neighbor
> >> interpolation. Trying to set the interpolation (using the
> >> set_interpolation method)
> >> to something akin imshow throw a "NotImplementedError: Only nearest
> >> neighbor supported" exception....
> >>
> >> So basically I am still stuck, it seems that currently there is no way
> >> in matplotlib to plot interpolated
> >> colormap on irregular rectangular grid, and even less on arbitrarily
> >> mapped grid...
> >>
> >> Is there any plans to add support for more interpolation in
> >> NonUniformImage in the future? Or maybe there is another 
> >> drawing function that I did not find yet, with this ability?
> >>
> >>
> >> Best regards,
> >>
> >> Greg.

Hi,
I just committed a patch that allows for multiple histogram where each 
data-set might have a different length (see example 
histogram_demo_extended.py). I don't think that it breaks any existing 
code, but I would feel much better if someone could check that ...
Manuel

Now I see there are more options in 0.98 - 'steps-pre', 'steps-post',
'steps-mid'. The default should be steps-post for backwards
compatibility. In 0.98.3 the default is steps-pre. And sorry for the
testy tone of the previous email :)
Neil
2008年8月28日 Neil Crighton <nei...@gm...>:
> linestyle='steps' has changed behaviour between 0.91.2 and 0.98.3. The
> 'step' between two points used to move horizontally and then
> vertically from the left point neighbouring right point, now it moves
> vertically then horizontally.
>
> Was this change intentional? I hope not, because I've just spent the
> past hour working out it was the reason for my plotting routines not
> working properly :-/
>
> Neil
>

linestyle='steps' has changed behaviour between 0.91.2 and 0.98.3. The
'step' between two points used to move horizontally and then
vertically from the left point neighbouring right point, now it moves
vertically then horizontally.
Was this change intentional? I hope not, because I've just spent the
past hour working out it was the reason for my plotting routines not
working properly :-/
Neil

On Thu, Aug 28, 2008 at 6:19 AM, Sandro Tosi <mat...@gm...> wrote:
> Enthought suite is used a lot in scientific area, so mpl and enth
> almost share their users, so have it enabled would be a plus, but we
> are mainly interested in generate "no harm", so we'd like to ask you
> to confirm that enabling that support won't brake anything (but indeed
> provide a valuable asset for users).
Well, one thing to make sure of is that you do not install the version
of traits that ships with matplotlib, since it will break other
enthought packages. My understanding is that one either uses the
default rc config or the traits enabled one. Since the latter is
experimental and we are not sure if we will eventually adopt it, I
would not recommend enabling it in the debian distro. Is this your
view Darren?
JDH

This should now be fixed in SVN r6052.
Cheers,
Mike
Jae-Joon Lee wrote:
> Hi,
>
> The clip_on and clip_box arguments (and maybe clip_path also) in
> plot() command seem to have no effect. For example,
>
> In [29]: p, =plot([1,2,3], clip_on=False)
>
> In [30]: p.get_clip_on()
> Out[30]: True
>
>
> It seems that the line object is created with the given arguments but
> gets overwritten later when it is added to the axes (Axes.add_line),
>
> def add_line(self, line):
> '''
> Add a :class:`~matplotlib.lines.Line2D` to the list of plot
> lines
> '''
> self._set_artist_props(line)
> line.set_clip_path(self.patch)
>
> "line.set_clip_path(self.patch)" update clip_path, clip_box, and clip_on.
>
> So, isn't it better to do something explicitly when these arguments
> are ignored? Maybe to print out some warnings or even raise an
> exception, or do not call set_clip_path when clip_* argument is given,
> or etc.?
>
> A slightly related behavior I found unexpected is that set_clip_path()
> and set_clip_rect() method also update clip_on.
>
> Regards,
>
> -JJ
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 

I think this is a bug created by the conversion from 0.91 to 0.98. I'll 
look into this and let you know when it has been resolved.
Mike
Jae-Joon Lee wrote:
> Hi,
>
> The clip_on and clip_box arguments (and maybe clip_path also) in
> plot() command seem to have no effect. For example,
>
> In [29]: p, =plot([1,2,3], clip_on=False)
>
> In [30]: p.get_clip_on()
> Out[30]: True
>
>
> It seems that the line object is created with the given arguments but
> gets overwritten later when it is added to the axes (Axes.add_line),
>
> def add_line(self, line):
> '''
> Add a :class:`~matplotlib.lines.Line2D` to the list of plot
> lines
> '''
> self._set_artist_props(line)
> line.set_clip_path(self.patch)
>
> "line.set_clip_path(self.patch)" update clip_path, clip_box, and clip_on.
>
> So, isn't it better to do something explicitly when these arguments
> are ignored? Maybe to print out some warnings or even raise an
> exception, or do not call set_clip_path when clip_* argument is given,
> or etc.?
>
> A slightly related behavior I found unexpected is that set_clip_path()
> and set_clip_rect() method also update clip_on.
>
> Regards,
>
> -JJ
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 

Hello guys,
in a joint effort between Debian and Ubuntu on matplotlib packaging,
we were wondering if include the experimental support for
python-enthought-traits.
...
EXPERIMENTAL CONFIG PACKAGE DEPENDENCIES
 configobj: 4.5.2
 enthought.traits: 2.0.5
...
Enthought suite is used a lot in scientific area, so mpl and enth
almost share their users, so have it enabled would be a plus, but we
are mainly interested in generate "no harm", so we'd like to ask you
to confirm that enabling that support won't brake anything (but indeed
provide a valuable asset for users).
Additionally, we have developed a patch for the build process
(attached) that you might be interested in merge in your codebase, in
particular hunk #2 and #3.
Cheers,
Sandro, Benjamin & Jordan
--
Sandro Tosi (aka morph, Morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

Paul Kienzle wrote:
> Hi,
> 
> There was a recent discussion about opengl and matplotlib in the
> context of matplotlib rendering speeds.
> 
> At the scipy sprints we put together a proof of concept renderer
> for quad meshes using the opengl frame buffer object, which we
> then render as a matplotlib image. Once the data is massaged
> to the correct form we were seeing rendering speeds of about
> 10 million quads per second. See below.
> 
> Using this technique we can get the advantage of opengl without
> having to write a full backend. Please let me know if you have
> time to contribute to writing a more complete backend.
I know that I (as the other contributer :) ) plan on making this into a 
full backend provided:
1) The quadmesh code can be shown yield the gains when integrated into 
matplotlib more fully
2) I find the time (which is a matter of *when*, not if)
I'm certainly finding thus far that pyglet makes it a lot easier to do a 
full backend than some of the other python->opengl methods I'd explored 
in the past.
Ryan
-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Hi,
There was a recent discussion about opengl and matplotlib in the
context of matplotlib rendering speeds.
At the scipy sprints we put together a proof of concept renderer
for quad meshes using the opengl frame buffer object, which we
then render as a matplotlib image. Once the data is massaged
to the correct form we were seeing rendering speeds of about
10 million quads per second. See below.
Using this technique we can get the advantage of opengl without
having to write a full backend. Please let me know if you have
time to contribute to writing a more complete backend.
 - Paul
-- meshgl.py --
# This program is public domain
import numpy
from pyglet.gl import *
import pyglet
from ctypes import POINTER,c_long,c_ubyte,c_int
def buildmesh(x,y):
 """
 x,y are corners of the mesh quadrants
 """
 # Quadlist is a list of pairs of vertices, left edge, right edge, right
 # edge, one for each row. The resulting map will look like:
 # [[p00 p10 p01 p11 p02 p12 ...] [p10 p20 p11 p21 ...] ...]
 # Note that the x,y shape will be one more than the number of data values.
 nr,nc = x.shape
 quadstrips = numpy.empty((nr-1,nc*4),'f')
 quadstrips[:,0::4] = x[:-1,:]
 quadstrips[:,2::4] = x[1:,:]
 quadstrips[:,1::4] = y[:-1,:]
 quadstrips[:,3::4] = y[1:,:]
 return quadstrips
def colorindex(z,clim,cmap):
 # Calc colour indices
 ncolors = cmap.shape[0]
 step = float(clim[1]-clim[0])/(ncolors-1)
 idx = numpy.asarray((z-clim[0])/step, dtype='i')
 idx = numpy.clip(idx, 0, ncolors-1)
 return idx
 
def buildcolor(z,clim=[0,1],cmap=None):
 """
 z are centers of the mesh quadrants.
 Note there size of z are less than size of x,y by 1 in each dim.
 """
 # Generate colours
 idx = colorindex(z, clim, cmap)
 nr,nc = z.shape
 # Quadlist is a list of pairs of vertices, left edge, right edge, right
 # edge. The resulting map will look like:
 # [[p00 p10 p01 p11 p02 p12 ...] [p10 p20 p11 p21 ...] ...]
 # The z values correspond to the bottom right corner of each quad. This
 # means we do not need the first two quads of each row and every odd
 # quad after that. The resulting color list looks like the following:
 # [[0 0 0 z00 0 z01 0 z02 ...] [0 0 0 z10 0 z11 0 z12 ...] ...]
 # Each i,j has a four byte colour value.
 colors = numpy.zeros((nr,2*nc+2,4),'B')
 colors[:,3::2,:] = cmap[idx,:]
 return colors
def draw_quads(x,y,z,clim=[0,1],cmap=None):
 strips = buildmesh(x,y)
 colors = buildcolor(z,clim,cmap)
 nr,nc = strips.shape
 nc = nc/2 # pair of vertices per point
 starts = numpy.arange(0,nr*nc,nc,'i')
 counts = numpy.ones(nr,'i')*nc
 glEnableClientState(GL_VERTEX_ARRAY)
 glEnableClientState(GL_COLOR_ARRAY)
 glVertexPointer(2,GL_FLOAT,0,strips.ctypes.data)
 glColorPointer(4,GL_UNSIGNED_BYTE,0,colors.ctypes.data)
 glMultiDrawArrays(GL_QUAD_STRIP, #starts, counts, nr)
 starts.ctypes.data_as(POINTER(c_int)),
 counts.ctypes.data_as(POINTER(c_int)),
 nr)
 glDisableClientState(GL_VERTEX_ARRAY)
 glDisableClientState(GL_COLOR_ARRAY)
 
def set_scene(w,h):
 """Set up the openGL transform matrices"""
 glViewport(0, 0, w, h)
 glMatrixMode(GL_PROJECTION)
 glLoadIdentity()
 glMatrixMode(GL_MODELVIEW)
 glLoadIdentity()
 glShadeModel(GL_FLAT)
 glEnable(GL_BLEND)
 glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA)
 # Clear scene
 glClearColor(1, 1, 1, 0)
 glClear(GL_COLOR_BUFFER_BIT)
def renderbuffer(w,h,draw):
 """Render buffer to off screen framebuffer object"""
 # Make sure we support frame buffers
 if not pyglet.gl.gl_info.have_extension('GL_EXT_framebuffer_object'):
 raise RuntimeError("This OpenGL does not support framebuffer objects")
 # Create a frame buffer context
 framebuffers = (GLuint*1)(0)
 glGenFramebuffersEXT(1, framebuffers)
 glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, framebuffers[0])
 # Create a render buffer
 renderbuffers = (GLuint*1)(0)
 glGenRenderbuffersEXT(1, renderbuffers)
 glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, renderbuffers[0])
 glRenderbufferStorageEXT(GL_RENDERBUFFER_EXT, GL_RGBA8, w, h)
 glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, 
 GL_RENDERBUFFER_EXT, renderbuffers[0])
 status = glCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT)
 if status != GL_FRAMEBUFFER_COMPLETE_EXT:
 raise RuntimeError("Could not create GL framebuffer")
 # Render scene
 set_scene(w,h)
 draw()
 glFlush()
 # Retrieve buffer
 buffer = numpy.empty((h,w,4),'B')
 glReadPixels(0, 0, w, h, GL_RGBA, GL_UNSIGNED_BYTE, buffer.ctypes.data)
 # View the buffer in the window
 #glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0)
 # Release buffer
 glDeleteRenderbuffersEXT(1, renderbuffers)
 return buffer
# === DEMO CODE BELOW ===
def testmesh(shape=(3,2)):
 """sample scene"""
 # Quad mesh vertices (nr x nc)
 nr,nc = shape
 x = numpy.arange(nr)[None,:] + numpy.arange(nc)[:,None]
 y = numpy.arange(nr)[None,:] + numpy.zeros((nc,1))
 # Quad mesh values (nr-1 x nc-1)
 z = x[:-1,:-1]
 # Grayscale RGBA colormap
 R = numpy.linspace(0,255,64)
 alpha = numpy.ones(64,'B')*255
 cmap = numpy.array([R,R,R,alpha],'B').T
 if True:
 # Green box in center using default -1,1 coords
 glColor3f(0., 1., 0.)
 glRectf(-0.5, 0.5, 0.5, -0.5)
 
 # Set data coordinates
 glPushMatrix()
 glOrtho(0,x.max()*2,0,y.max()*2,-1,1)
 if True:
 # Blue box unit size at 1,1 in data coords
 glColor4f(0., 0., 1., 1.)
 glRectf(1, 2, 2, 1)
 draw_quads(x,y,z,clim=[z.min(),z.max()],cmap=cmap)
 glPopMatrix()
class Window(pyglet.window.Window):
 """Pyglet driver"""
 def __init__(self, w, h):
 super(Window, self).__init__(width=w, height=h, resizable=True)
 glClearColor(1.,1.,1.,1.)
 self._dirty = True
 def on_resize(self, w, h):
 print "resize"
 set_scene(w,h);
 self._dirty = True
 def draw(self):
 if self._dirty:
 print "draw"
 #window.clear()
 testmesh()
 self._dirty = False
 self.flip()
 
 def run(self):
 counter = 0
 while not self.has_exit and counter < 1000:
 self.dispatch_events()
 self.draw()
 counter += 1
 
def plot():
 """Matplotlib driver"""
 import pylab
 print "start rendering"
 b = renderbuffer(400,500,lambda: testmesh((1000,1000)))
 print "done rendering"
 pylab.imshow(b,origin='lower')
 pylab.show()
if __name__ == "__main__":
 #Window(400,500).run()
 plot()