matplotlib-devel Mailing List for matplotlib

On Tue, Aug 25, 2009 at 4:35 PM, Brian Granger<ell...@gm...> wrote:
> Hello all,
>
> While at SciPy this year, the IPython devs began discussing dropping Python
> 2.4 support. Or rational is this:
>
> * New generator features in 2.5 would dramatically simplify our testing of
> our Twisted using components.
>
> * Being able to use absolute imports would simplify the packaging of some
> external deps.
>
> * The faster we get rid of 2.4 and 2.5 support (we are not getting rid of
> 2.5 yet) the faster we can transition to py3k.
>
> But, this would mean that pylab mode for matplotlib would require either:
>
> * Using IPython v 0.10 or below
>
> * Using Python 2.5/2.6
>
> We know that there are people still using Python 2.4, but at this point, we
> feel the benefits outweight the costs. How do the matplotlib devs feel
> about this?
>
> As a side note, as of IPython 0.11, the IPython threaded shells (pylab
> stuff) will be completely refactored. This will require matplotlib to make
> some moderate changes to support the new interface. Thus, even if we don't
> drop 2.4 support, matplotlib will have to decide how to handle the IPython
> 0.10->0.11 transition.
My sense is that most people still using python-2.4 probably put a
premium on stability and wouldn't be interested in (or aware of) the
refactored codebase until it has seen some use and had a chance to
work out the bugs. I think it is reasonable to provide a ipy-0.10
based on the old codebase that is compatible with py-2.4 and make use
of the newer language features in the trunk.
Hopefully someone will come up with a 3to2 tool that will support
python-2.5. That might provide a faster route to ipython for python-3.
Darren

Michael Hearne wrote:
> I just built matplotlib and basemap from source on a RHEL system, with 
> EPD as my base Python installation.
>
> The build procedure for matplotlib was fairly straightforward, as was 
> basemap (once I read Jeff's documentation on installing).
>
> However, once I try to import basemap, I get an error about dbflib 
> (ipython session below):
> 
Mike: Try editing setup.cfg and changing
pyshapelib = True
It's set to 'auto' by default. I bet it's detecting an existing 
pyshapelib install, but then can find all the parts of it that it needs.
-Jeff
> In [1]: from mpl_toolkits.basemap import Basemap
> ---------------------------------------------------------------------------
> ImportError Traceback (most recent call 
> last)
>
> /home/shake/losspager/1.15/<ipython console> in <module>()
>
> /usr/local/epd/lib/python2.5/site-packages/mpl_toolkits/basemap/ 
> __init__.py in <module>()
> 36 from matplotlib.lines import Line2D
> 37 from matplotlib.transforms import Bbox
> ---> 38 import pyproj, sys, os, math, dbflib
> 39 from proj import Proj
> 40 import numpy as np
>
> ImportError: No module named dbflib
>
> Where is dbflib supposed to be?
>
> Thanks,
>
> Mike
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
> trial. Simplify your report design, integration and deployment - and focus on 
> what you do best, core application coding. Discover what's new with 
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jef...@no...
325 Broadway Office : Skaggs Research Cntr 1D-113
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg

I just built matplotlib and basemap from source on a RHEL system, with 
EPD as my base Python installation.
The build procedure for matplotlib was fairly straightforward, as was 
basemap (once I read Jeff's documentation on installing).
However, once I try to import basemap, I get an error about dbflib 
(ipython session below):
In [1]: from mpl_toolkits.basemap import Basemap
---------------------------------------------------------------------------
ImportError Traceback (most recent call 
last)
/home/shake/losspager/1.15/<ipython console> in <module>()
/usr/local/epd/lib/python2.5/site-packages/mpl_toolkits/basemap/ 
__init__.py in <module>()
 36 from matplotlib.lines import Line2D
 37 from matplotlib.transforms import Bbox
---> 38 import pyproj, sys, os, math, dbflib
 39 from proj import Proj
 40 import numpy as np
ImportError: No module named dbflib
Where is dbflib supposed to be?
Thanks,
Mike

Hello all,
While at SciPy this year, the IPython devs began discussing dropping Python
2.4 support. Or rational is this:
* New generator features in 2.5 would dramatically simplify our testing of
our Twisted using components.
* Being able to use absolute imports would simplify the packaging of some
external deps.
* The faster we get rid of 2.4 and 2.5 support (we are not getting rid of
2.5 yet) the faster we can transition to py3k.
But, this would mean that pylab mode for matplotlib would require either:
* Using IPython v 0.10 or below
* Using Python 2.5/2.6
We know that there are people still using Python 2.4, but at this point, we
feel the benefits outweight the costs. How do the matplotlib devs feel
about this?
As a side note, as of IPython 0.11, the IPython threaded shells (pylab
stuff) will be completely refactored. This will require matplotlib to make
some moderate changes to support the new interface. Thus, even if we don't
drop 2.4 support, matplotlib will have to decide how to handle the IPython
0.10->0.11 transition.
Cheers,
Brian

Hmm,
That might be nothing to do with Sphinx's being very recent. Since that
thread from sphinx-dev list is from Oct 2008 and the guy reporting the
similar issue seemingly using an even older Sphinx version (4.3)
I don't exactly know what would be the advantage of making docstrings numpy
standardized.
On Tue, Aug 25, 2009 at 3:01 PM, Michael Droettboom <md...@st...> wrote:
> The error is caused by recent versions of Sphinx being more picky about
> headers in docstrings. I had to update my version of Sphinx in order to
> reproduce your error.
>
> Your fix is adding support for Numpy's custom docstring format? I'm not
> sure we want to go down that road just yet. Easier to just remove the
> headers in the cohere_pairs docstring (since it's the only one!) and move
> on. I'll have a fix for this once my doc build is done.
>
> I'm not saying using the Numpy format wouldn't be a good idea. But it's a
> lot of manual labor to move to that format, make sure things are consistent
> etc. It's interesting that your fix doesn't produce new problems, though.
>
> Mike
>
> Gökhan Sever wrote:
>
>> OK,
>>
>> Here comes a self fix:
>>
>> http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/docscrape.py
>> http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/docscrape_sphinx.py
>> http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/numpydoc.py
>>
>> added the above files under /doc/sphinxext and updated conf.py.
>>
>> Then works.
>>
>> This said, I am not sure is this the natural way to fix this problem?
>>
>> Thanks.
>>
>> On Tue, Aug 25, 2009 at 1:10 PM, Gökhan Sever <gok...@gm...<mailto:
>> gok...@gm...>> wrote:
>>
>> Hello,
>>
>> The trunk is giving the following error while trying to build the
>> documentation via "python make.py all"
>>
>> reading sources... [ 4%]
>> api/mlab_api
>>
>> reST markup error:
>>
>> /home/gsever/Desktop/python-repo/matplotlib/lib/matplotlib/mlab.py:docstring
>> of matplotlib.mlab.cohere_pairs:6: (SEVERE/4) Unexpected section
>> title.
>>
>> A similar error reported at:
>>
>>
>> http://groups.google.com/group/sphinx-dev/browse_thread/thread/300153957f2902f9?pli=1
>>
>> however, couldn't figure to fix the issue.
>>
>> Any comments?
>>
>> -- Gökhan
>>
>>
>>
>>
>> --
>> Gökhan
>> ------------------------------------------------------------------------
>>
>>
>> ------------------------------------------------------------------------------
>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
>> 30-Day trial. Simplify your report design, integration and deployment - and
>> focus on what you do best, core application coding. Discover what's new with
>> Crystal Reports now. http://p.sf.net/sfu/bobj-july
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>>
>
> --
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
>
-- 
Gökhan

The error is caused by recent versions of Sphinx being more picky about 
headers in docstrings. I had to update my version of Sphinx in order to 
reproduce your error.
Your fix is adding support for Numpy's custom docstring format? I'm not 
sure we want to go down that road just yet. Easier to just remove the 
headers in the cohere_pairs docstring (since it's the only one!) and 
move on. I'll have a fix for this once my doc build is done.
I'm not saying using the Numpy format wouldn't be a good idea. But it's 
a lot of manual labor to move to that format, make sure things are 
consistent etc. It's interesting that your fix doesn't produce new 
problems, though.
Mike
Gökhan Sever wrote:
> OK,
>
> Here comes a self fix:
>
> http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/docscrape.py
> http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/docscrape_sphinx.py
> http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/numpydoc.py
>
> added the above files under /doc/sphinxext and updated conf.py.
>
> Then works.
>
> This said, I am not sure is this the natural way to fix this problem?
>
> Thanks.
>
> On Tue, Aug 25, 2009 at 1:10 PM, Gökhan Sever <gok...@gm... 
> <mailto:gok...@gm...>> wrote:
>
> Hello,
>
> The trunk is giving the following error while trying to build the
> documentation via "python make.py all"
>
> reading sources... [ 4%]
> api/mlab_api 
>
> reST markup error:
> /home/gsever/Desktop/python-repo/matplotlib/lib/matplotlib/mlab.py:docstring
> of matplotlib.mlab.cohere_pairs:6: (SEVERE/4) Unexpected section
> title.
>
> A similar error reported at:
>
> http://groups.google.com/group/sphinx-dev/browse_thread/thread/300153957f2902f9?pli=1
>
> however, couldn't figure to fix the issue.
>
> Any comments?
>
> -- 
> Gökhan
>
>
>
>
> -- 
> Gökhan
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
> trial. Simplify your report design, integration and deployment - and focus on 
> what you do best, core application coding. Discover what's new with 
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> ------------------------------------------------------------------------
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

OK,
Here comes a self fix:
http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/docscrape.py
http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/docscrape_sphinx.py
http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/numpydoc.py
added the above files under /doc/sphinxext and updated conf.py.
Then works.
This said, I am not sure is this the natural way to fix this problem?
Thanks.
On Tue, Aug 25, 2009 at 1:10 PM, Gökhan Sever <gok...@gm...> wrote:
> Hello,
>
> The trunk is giving the following error while trying to build the
> documentation via "python make.py all"
>
> reading sources... [ 4%]
> api/mlab_api
>
> reST markup error:
> /home/gsever/Desktop/python-repo/matplotlib/lib/matplotlib/mlab.py:docstring
> of matplotlib.mlab.cohere_pairs:6: (SEVERE/4) Unexpected section title.
>
> A similar error reported at:
>
>
> http://groups.google.com/group/sphinx-dev/browse_thread/thread/300153957f2902f9?pli=1
>
> however, couldn't figure to fix the issue.
>
> Any comments?
>
> --
> Gökhan
>
-- 
Gökhan

md...@us... wrote:
> Revision: 7567
> http://matplotlib.svn.sourceforge.net/matplotlib/?rev=7567&view=rev
> Author: mdboom
> Date: 2009年08月25日 15:31:10 +0000 (2009年8月25日)
> 
> Log Message:
> -----------
> Support Line2D objects without associated Axes objects.
> 
> Modified Paths:
> --------------
> branches/v0_99_maint/lib/matplotlib/lines.py
> 
> Modified: branches/v0_99_maint/lib/matplotlib/lines.py
> ===================================================================
> --- branches/v0_99_maint/lib/matplotlib/lines.py	2009年08月25日 11:54:24 UTC (rev 7566)
> +++ branches/v0_99_maint/lib/matplotlib/lines.py	2009年08月25日 15:31:10 UTC (rev 7567)
> @@ -459,7 +459,7 @@
> self._y = self._xy[:, 1] # just a view
> 
> self._subslice = False
> - if len(x) > 100 and self._is_sorted(x):
> + if self.axes and len(x) > 100 and self._is_sorted(x):
> self._subslice = True
> if hasattr(self, '_path'):
> interpolation_steps = self._path._interpolation_steps
> @@ -496,7 +496,7 @@
> def draw(self, renderer):
> if self._invalid:
> self.recache()
> - if self._subslice:
> + if self._subslice and self.axes:
> # Need to handle monotonically decreasing case also...
> x0, x1 = self.axes.get_xbound()
> i0, = self._x.searchsorted([x0], 'left')
Mike,
I'm curious--why are both changes needed? Shouldn't the setting of 
self._subslice in the first chunk be enough? If not, it suggests that 
there are more general design problems here. Can a line object have an 
associated axes when it is created (or when data are fed to it) and then 
lose that axes attribute by the time draw() is called? If so, then it 
seems like the second chunk is the right one to keep, and the first one 
is useless.
Eric

Hello,
The trunk is giving the following error while trying to build the
documentation via "python make.py all"
reading sources... [ 4%]
api/mlab_api
reST markup error:
/home/gsever/Desktop/python-repo/matplotlib/lib/matplotlib/mlab.py:docstring
of matplotlib.mlab.cohere_pairs:6: (SEVERE/4) Unexpected section title.
A similar error reported at:
http://groups.google.com/group/sphinx-dev/browse_thread/thread/300153957f2902f9?pli=1
however, couldn't figure to fix the issue.
Any comments?
-- 
Gökhan

Ah -- for some reason the benefit is immeasurable until about 1M 
points. I was using fewer points because you can't even get a baseline 
measurement with that many -- it exceeds the path length limit in Agg.
I see now why it's superior to the clipping I wrote -- it does a binary 
search to find the end points. Makes sense. We'll have to keep both 
approaches, since the clipping is also required to prevent rendering 
errors due to fixed-point number overflow in the Agg backend.
So, I'll just patch it up so subslicing is disabled when there is no 
associated axes (the root of my original problem) and leave it at that.
Cheers,
Mike
Eric Firing wrote:
> Michael Droettboom wrote:
>> According to svn blame, which only gives the most recent version a 
>> line was edited, not the first time a line appeared, obviously -- 
>> subslice support was added in r7100, and clipping was fixed in 
>> r6847. So, apparently at the time subslice was added the clipping 
>> was already there. So, if you were seeing a time improvement then, 
>> your data must be doing something my benchmark here doesn't.
>
> Mike,
>
> Try something like this:
>
> x = arange(1000000.0)
> y = sin(x/100)
> plot(x, y)
> xlim(0, 1000)
>
> Now select pan/zoom, hold down X, and pan. Notice the degree of speed 
> and smoothness (or not). Next, disable subslicing like this:
>
> clf()
> x[1] = -0.1 # no longer monotonic
> plot(x, y)
> xlim(0, 1000)
>
> And try panning again. At least on my machine, there is a big 
> difference.
>
> Eric
>
>
>
>>
>> The clipping support is probably more general than subslicing, since 
>> it doesn't require the data to be monotonic -- it clips as the line 
>> crosses any of the boundaries of the figure. Given that, I'm 
>> surprised it competes so favorably timewise -- I suspect the 
>> important thing is to just reduce the number of points passed to the 
>> renderer -- the actually speed at which those points are located is 
>> nothing compared to stroking points.
>>
>> Cheers,
>> Mike
>>
>> Eric Firing wrote:
>>> Michael Droettboom wrote:
>>>> We recently saw some breakage with our PyRAF plotting tool (which 
>>>> uses matplotlib as a "dumb" rendering backend) and matplotlib 
>>>> 0.99. It stops inside the subslice support that was added to 
>>>> Line2D, since subslicing requires that the Line2D object have an 
>>>> "axes" assigned to it. Since PyRAF doesn't use matplotlib's Axes 
>>>> objects, its lines don't have them.
>>>>
>>>> It's a simple fix to check for the existence of an axes member and 
>>>> skip the subslice support if it doesn't have one. However, I 
>>>> wonder if it couldn't just be removed, especially since it is the 
>>>> only dependency on an Axes from Line2D objects. I think it may 
>>>> have become redundant wrt the path simplification code which now 
>>>> handles clipping (at the figure boundary, not the axis boundary 
>>>> mind you) rather reliably. The simplification code now also works 
>>>> in all backends, which is fairly new.
>>>
>>> Mike,
>>>
>>> Was the simplification you are talking about added after I added the 
>>> subslice support (by which I assume you mean the slicing in the case 
>>> of monotonic x)? And is it as general? At least at the time I put 
>>> in the subslicing of sorted abcissas, I am pretty sure it made a big 
>>> difference when panning long timeseries--that is why I put it in. 
>>> Certainly I would be happy to see it go if it is not actually doing 
>>> anything useful now, but I would like to be sure. I can't look at 
>>> it right now.
>>>
>>> Eric
>>>
>>>>
>>>> I did a little benchmarking with the attached script. It generates 
>>>> a 10,000 point random array and then plots a 500 point subset in 
>>>> the middle.
>>>>
>>>> baseline: 3.94 fps (no clipping or subslicing)
>>>> subslice: 28.14 fps
>>>> clipping: 28.09 fps
>>>> clipping+subslice: 28.35 fps (i.e. current code)
>>>>
>>>> The last three are close enough to be considered equal.
>>>>
>>>> Of course, another benchmark may produce very different results, so 
>>>> I'm reluctant to just yank it. But it would be nice to remove 
>>>> nearly-identical optimizations.
>>>>
>>>> Cheers,
>>>> Mike
>>>>
>>>>
>>>> ------------------------------------------------------------------------ 
>>>>
>>>>
>>>> ------------------------------------------------------------------------------ 
>>>>
>>>> Let Crystal Reports handle the reporting - Free Crystal Reports 
>>>> 2008 30-Day trial. Simplify your report design, integration and 
>>>> deployment - and focus on what you do best, core application 
>>>> coding. Discover what's new with Crystal Reports now. 
>>>> http://p.sf.net/sfu/bobj-july
>>>>
>>>>
>>>> ------------------------------------------------------------------------ 
>>>>
>>>>
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Mat...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>
>>
>
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Michael Droettboom wrote:
> According to svn blame, which only gives the most recent version a line 
> was edited, not the first time a line appeared, obviously -- subslice 
> support was added in r7100, and clipping was fixed in r6847. So, 
> apparently at the time subslice was added the clipping was already 
> there. So, if you were seeing a time improvement then, your data must 
> be doing something my benchmark here doesn't.
Mike,
Try something like this:
x = arange(1000000.0)
y = sin(x/100)
plot(x, y)
xlim(0, 1000)
Now select pan/zoom, hold down X, and pan. Notice the degree of speed 
and smoothness (or not). Next, disable subslicing like this:
clf()
x[1] = -0.1 # no longer monotonic
plot(x, y)
xlim(0, 1000)
And try panning again. At least on my machine, there is a big difference.
Eric
> 
> The clipping support is probably more general than subslicing, since it 
> doesn't require the data to be monotonic -- it clips as the line crosses 
> any of the boundaries of the figure. Given that, I'm surprised it 
> competes so favorably timewise -- I suspect the important thing is to 
> just reduce the number of points passed to the renderer -- the actually 
> speed at which those points are located is nothing compared to stroking 
> points.
> 
> Cheers,
> Mike
> 
> Eric Firing wrote:
>> Michael Droettboom wrote:
>>> We recently saw some breakage with our PyRAF plotting tool (which 
>>> uses matplotlib as a "dumb" rendering backend) and matplotlib 0.99. 
>>> It stops inside the subslice support that was added to Line2D, since 
>>> subslicing requires that the Line2D object have an "axes" assigned to 
>>> it. Since PyRAF doesn't use matplotlib's Axes objects, its lines 
>>> don't have them.
>>>
>>> It's a simple fix to check for the existence of an axes member and 
>>> skip the subslice support if it doesn't have one. However, I wonder 
>>> if it couldn't just be removed, especially since it is the only 
>>> dependency on an Axes from Line2D objects. I think it may have 
>>> become redundant wrt the path simplification code which now handles 
>>> clipping (at the figure boundary, not the axis boundary mind you) 
>>> rather reliably. The simplification code now also works in all 
>>> backends, which is fairly new.
>>
>> Mike,
>>
>> Was the simplification you are talking about added after I added the 
>> subslice support (by which I assume you mean the slicing in the case 
>> of monotonic x)? And is it as general? At least at the time I put in 
>> the subslicing of sorted abcissas, I am pretty sure it made a big 
>> difference when panning long timeseries--that is why I put it in. 
>> Certainly I would be happy to see it go if it is not actually doing 
>> anything useful now, but I would like to be sure. I can't look at it 
>> right now.
>>
>> Eric
>>
>>>
>>> I did a little benchmarking with the attached script. It generates a 
>>> 10,000 point random array and then plots a 500 point subset in the 
>>> middle.
>>>
>>> baseline: 3.94 fps (no clipping or subslicing)
>>> subslice: 28.14 fps
>>> clipping: 28.09 fps
>>> clipping+subslice: 28.35 fps (i.e. current code)
>>>
>>> The last three are close enough to be considered equal.
>>>
>>> Of course, another benchmark may produce very different results, so 
>>> I'm reluctant to just yank it. But it would be nice to remove 
>>> nearly-identical optimizations.
>>>
>>> Cheers,
>>> Mike
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> ------------------------------------------------------------------------------ 
>>>
>>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 
>>> 30-Day trial. Simplify your report design, integration and deployment 
>>> - and focus on what you do best, core application coding. Discover 
>>> what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Matplotlib-devel mailing list
>>> Mat...@li...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>