matplotlib-devel Mailing List for matplotlib

Resending, the first try doesn't seem to make it to the list.
---------- Forwarded message ----------
From: Ondrej Certik <on...@ce...>
Date: Dec 30, 2007 7:33 PM
Subject: merging sympy plotting stuff with matplotlib
To: mat...@li...
Hi matplotlib developers,
we are developing a symbolic manipulation library in pure Python:
http://code.google.com/p/sympy/
and we wanted to do 3D plotting. To make a long story short, here is a
tutorial for our 3D plotting stuff:
http://code.google.com/p/sympy/wiki/PlottingModule
here is a reasoning behind all of this:
http://code.google.com/p/sympy/wiki/PlottingReport
especially this:
http://straightupcoding.blogspot.com/2007/05/case-for-dropping-matplotlib.html
We should have gotten involved more in matplotlib development earlier,
but at least now. I think there should be just one 3D plotting
library in Python and imho matplotlib should do it. However, we need:
* it should be pure python
* fast and interactive 3D stuff
* needs to work out of the box on linux, mac os x, windows (without compilation)
you can read all details above, but we simply use pyglet
(http://pyglet.org/), that is like pygame, but better (pure python,
works out of the box on all platforms etc).
Are you interested in getting our 3D stuff into matplotlib?
If yes, is there a way to make matplotlib pure python?
Looking at matplotlib sources, the only things done in C++ are agg
backend, and then the src/* directory, which I am not sure what
exactly it's doing, but I don't see a reason why the kernel of
matplotlib cannot be in pure Python (calling python gtk etc.).
Optionally, there can be the C++ modules.
If such a division is possible, then we could just include matplotlib
in sympy sources, the license of matplotlib seems quite permissive:
http://matplotlib.sourceforge.net/license.html
so it should allow this (sympy and pyglet is BSD). In distributions,
like Debian, sympy will just depend on matplotlib in debian. But the
sympy tarball should be self-contained.
The sympy plotting is quite nice, especially that you just download
the tarball and it works (even on windows, or macosx) without
compilation, however it's missing a lot of features that matplotlib
has, so the best thing to do is to integrate it in matplotlib.
Another cool stuff in matplotlib is the pure python latex renderer
(/matplotlib-0.91.1/lib/matplotlib/mathtext.py). See our issue for
more info:
http://code.google.com/p/sympy/issues/detail?id=506
we want to use it in our preview capability (currently only in our hg repo):
$ hg clone http://hg.sympy.org/sympy/
$ cd sympy
$ bin/isympy
[...]
In [1]: from sympy.abc import *
In [2]: pngview(Integral(exp(-(tau-mu)**2/2/sigma**2), (tau, -oo, oo))/
 ...: sigma/sqrt(2*pi))
And a window will popup (using pyglet) showing a nice latex printed
expression. Currently, you need to have latex installed (and
python-pexpect)
for it to be working. (if you use sympy hg right now, you need
python-pexpect, but we'll fix this before we release soon:
http://code.google.com/p/sympy/issues/detail?id=520, any sympy tarball
only needs pure Python 2.4 or newer and it will just work, otherwise
it's a bug)
So it'd be cool to you your pure python reimplementation of the tex
engine. Maybe let's create a new (standalone) project just for this
feature? I am sure
it will benefit to a lot of people. And sympy and matplotlib will just
include it in the sources.
If you have any other ideas regarding these issues, we are interested.
Looking for collaboration,
Ondrej

Hi matplotlib developers,
we are developing a symbolic manipulation library in pure Python:
http://code.google.com/p/sympy/
and we wanted to do 3D plotting. To make a long story short, here is a
tutorial for our 3D plotting stuff:
http://code.google.com/p/sympy/wiki/PlottingModule
here is a reasoning behind all of this:
http://code.google.com/p/sympy/wiki/PlottingReport
especially this:
http://straightupcoding.blogspot.com/2007/05/case-for-dropping-matplotlib.html
We should have gotten involved more in matplotlib development earlier,
but at least now. I think there should be just one 3D plotting
library in Python and imho matplotlib should do it. However, we need:
* it should be pure python
* fast and interactive 3D stuff
* needs to work out of the box on linux, mac os x, windows (without compilation)
you can read all details above, but we simply use pyglet
(http://pyglet.org/), that is like pygame, but better (pure python,
works out of the box on all platforms etc).
Are you interested in getting our 3D stuff into matplotlib?
If yes, is there a way to make matplotlib pure python?
Looking at matplotlib sources, the only things done in C++ are agg
backend, and then the src/* directory, which I am not sure what
exactly it's doing, but I don't see a reason why the kernel of
matplotlib cannot be in pure Python (calling python gtk etc.).
Optionally, there can be the C++ modules.
If such a division is possible, then we could just include matplotlib
in sympy sources, the license of matplotlib seems quite permissive:
http://matplotlib.sourceforge.net/license.html
so it should allow this (sympy and pyglet is BSD). In distributions,
like Debian, sympy will just depend on matplotlib in debian. But the
sympy tarball should be self-contained.
The sympy plotting is quite nice, especially that you just download
the tarball and it works (even on windows, or macosx) without
compilation, however it's missing a lot of features that matplotlib
has, so the best thing to do is to integrate it in matplotlib.
Another cool stuff in matplotlib is the pure python latex renderer
(/matplotlib-0.91.1/lib/matplotlib/mathtext.py). See our issue for
more info:
http://code.google.com/p/sympy/issues/detail?id=506
we want to use it in our preview capability (currently only in our hg repo):
$ hg clone http://hg.sympy.org/sympy/
$ cd sympy
$ bin/isympy
[...]
In [1]: from sympy.abc import *
In [2]: pngview(Integral(exp(-(tau-mu)**2/2/sigma**2), (tau, -oo, oo))/
 ...: sigma/sqrt(2*pi))
And a window will popup (using pyglet) showing a nice latex printed
expression. Currently, you need to have latex installed (and
python-pexpect)
for it to be working. (if you use sympy hg right now, you need
python-pexpect, but we'll fix this before we release soon:
http://code.google.com/p/sympy/issues/detail?id=520, any sympy tarball
only needs pure Python 2.4 or newer and it will just work, otherwise
it's a bug)
So it'd be cool to you your pure python reimplementation of the tex
engine. Maybe let's create a new (standalone) project just for this
feature? I am sure
it will benefit to a lot of people. And sympy and matplotlib will just
include it in the sources.
If you have any other ideas regarding these issues, we are interested.
Looking for collaboration,
Ondrej

On the what's new page,
http://matplotlib.sourceforge.net/whats_new.html
it says "expand it's coverage" but it should be "expand its coverage".
Happy holidays.
Dr. Tom
--
My brother, when you have a virtue, and it is your own virtue, you have
it in common with no one. Thus spoke Zarathustra.

Jouni K. Seppänen wrote:
> The matplotlib.use() function was always a no-op after importing pylab,
> and there were occasional questions about it on the mailing list. It was
> changed fairly recently to raise an error in this case. Bug #1855454
> points out that this is not backward compatible: an application that
> calls use() too late didn't do anything before the change, but now gets
> stopped with a traceback. I suggest changing the error to a warning,
> which I think would still deliver the message but would be compatible
> with (broken) legacy code. What do you think?
> 
> http://sourceforge.net/tracker/index.php?func=detail&aid=1855454&group_id=80706&atid=560720
> 
Good point. I probably went overboard when making the original change 
in October, so I have now changed it to a warning as you suggest.
Eric

On Dec 25, 2007 5:39 AM, Mark Bakker <ma...@gm...> wrote:
> Tedd, Michael -
>
> Sorry to join this discussion late, but wouldn't it be easier to use a
> representation of an ellipse in complex variables? That's what I have been
> using and it seems pretty quick to me.
The main point we are discussion is not how to represent the ellipse,
but what geometric primitives to use to render the ellipse. Earlier
versions of mpl used a polygon, then we added support for a 4 spline
approximation to reduce the error of the approximation. This too did
not have sufficient accuracy, and so Michael added an 8 spline
approximation (Ellipse) as well as an adaptive approximation (Arc).
JDH

The matplotlib.use() function was always a no-op after importing pylab,
and there were occasional questions about it on the mailing list. It was
changed fairly recently to raise an error in this case. Bug #1855454
points out that this is not backward compatible: an application that
calls use() too late didn't do anything before the change, but now gets
stopped with a traceback. I suggest changing the error to a warning,
which I think would still deliver the message but would be compatible
with (broken) legacy code. What do you think?
http://sourceforge.net/tracker/index.php?func=detail&aid=1855454&group_id=80706&atid=560720
-- 
Jouni

Tedd, Michael -
Sorry to join this discussion late, but wouldn't it be easier to use a
representation of an ellipse in complex variables? That's what I have been
using and it seems pretty quick to me.
Have you guys tried this? Any experience you want to share?
Mark

Ok -- this goes without saying, but... I've only tested this with the 
toy examples you guys have sent etc. Please verify the results on real 
data before relying on it.
Anyway, it's great to have contributed even such a small part to such a 
great undertaking!
Cheers,
Mike
Ted Drain wrote:
> Everyone,
> I just wanted to say thanks for tackling this problem so quickly. It's 
> great to see that problems can be worked out like this and we really 
> appreciate it.
> 
> Here is some background on what we're using this for: The current NASA 
> mission to Mars (Phoenix) will be landing at the end of May. They 
> normally view their current trajectory and associated uncertainty as a 
> plot of an ellipse projected on a plane that extends out from the center 
> of the body (with time to impact the plane being another parameter). 
> The spacecraft has to hit a very narrow entry flight path angle corridor 
> at the atmosphere - too steep and it burns up, too shallow and it skips 
> out. When this corridor is projected onto the plane mentioned above, it 
> forms a large circle around Mars. So those large circular plots we're 
> making are the safe entry corridor that the trajectory needs to hit and 
> the smaller ellipses are the possible trajectory entry points and 
> uncertainties. As you can imagine, it's fairly critical to get those 
> ellipses plotted accurately :)
> 
> Thanks again,
> Ted
> 
> At 08:23 AM 12/21/2007, John Hunter wrote:
>> On Dec 21, 2007 9:11 AM, Michael Droettboom <md...@st...> wrote:
>>
>> > > Porting this back to the trunk is non-trivial -- but I know John would
>> > > like to do it, so I think it will happen once one of us has time.
>>
>> > I have ported this arc stuff to the trunk. It is in r4783. Please let
>> > me know how that works, particularly wrt performance, for you. Some
>> > things that are in C on the branch were done in Python on the trunk for
>> > convenience.
>>
>> Awesome -- thanks again Michael. You keep stealing my holiday 
>> projects :-)
>>
>> I added a check to make sure the 'axes' instance was present at draw
>> time, since patches can be added directly to the figure also, and
>> noted in the docstring that this is an axes only object.
>>
>> I also added unit support, which is fairly trivial, and added Arc to
>> the unit/ellipse_large.py test case and
>> examples/unit/ellipse_with_units.py
>>
>> Great work!
>> JDH
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2005.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Everyone,
I just wanted to say thanks for tackling this problem so 
quickly. It's great to see that problems can be worked out like this 
and we really appreciate it.
Here is some background on what we're using this for: The current 
NASA mission to Mars (Phoenix) will be landing at the end of 
May. They normally view their current trajectory and associated 
uncertainty as a plot of an ellipse projected on a plane that extends 
out from the center of the body (with time to impact the plane being 
another parameter). The spacecraft has to hit a very narrow entry 
flight path angle corridor at the atmosphere - too steep and it burns 
up, too shallow and it skips out. When this corridor is projected 
onto the plane mentioned above, it forms a large circle around 
Mars. So those large circular plots we're making are the safe entry 
corridor that the trajectory needs to hit and the smaller ellipses 
are the possible trajectory entry points and uncertainties. As you 
can imagine, it's fairly critical to get those ellipses plotted accurately :)
Thanks again,
Ted
At 08:23 AM 12/21/2007, John Hunter wrote:
>On Dec 21, 2007 9:11 AM, Michael Droettboom <md...@st...> wrote:
>
> > > Porting this back to the trunk is non-trivial -- but I know John would
> > > like to do it, so I think it will happen once one of us has time.
>
> > I have ported this arc stuff to the trunk. It is in r4783. Please let
> > me know how that works, particularly wrt performance, for you. Some
> > things that are in C on the branch were done in Python on the trunk for
> > convenience.
>
>Awesome -- thanks again Michael. You keep stealing my holiday projects :-)
>
>I added a check to make sure the 'axes' instance was present at draw
>time, since patches can be added directly to the figure also, and
>noted in the docstring that this is an axes only object.
>
>I also added unit support, which is fairly trivial, and added Arc to
>the unit/ellipse_large.py test case and
>examples/unit/ellipse_with_units.py
>
>Great work!
>JDH
>
>-------------------------------------------------------------------------
>This SF.net email is sponsored by: Microsoft
>Defy all challenges. Microsoft(R) Visual Studio 2005.
>http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>_______________________________________________
>Matplotlib-devel mailing list
>Mat...@li...
>https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

On Dec 21, 2007 9:11 AM, Michael Droettboom <md...@st...> wrote:
> > Porting this back to the trunk is non-trivial -- but I know John would
> > like to do it, so I think it will happen once one of us has time.
> I have ported this arc stuff to the trunk. It is in r4783. Please let
> me know how that works, particularly wrt performance, for you. Some
> things that are in C on the branch were done in Python on the trunk for
> convenience.
Awesome -- thanks again Michael. You keep stealing my holiday projects :-)
I added a check to make sure the 'axes' instance was present at draw
time, since patches can be added directly to the figure also, and
noted in the docstring that this is an axes only object.
I also added unit support, which is fairly trivial, and added Arc to
the unit/ellipse_large.py test case and
examples/unit/ellipse_with_units.py
Great work!
JDH

I have ported this arc stuff to the trunk. It is in r4783. Please let 
me know how that works, particularly wrt performance, for you. Some 
things that are in C on the branch were done in Python on the trunk for 
convenience.
Cheers,
Mike
Michael Droettboom wrote:
> Porting this back to the trunk is non-trivial -- but I know John would 
> like to do it, so I think it will happen once one of us has time.
> 
> As an aside, the unit stuff *should* be working on the branch. Can you 
> send me a minimal example that shows it failing?
> 
> Cheers,
> Mike
> 
> James Evans wrote:
>> I have taken the transforms branch and played with it a bit and my super simple ellipse test cases appeared to be working great. I
>> couldn't run our more intensive tests since they use unitized data and I was getting errors that looked like the transforms branch
>> wasn't completely handling unitized data properly. It would be great if we could see this fix in the main branch, then we can make
>> use of it right away without having to wait for the transforms branch to be completed.
>>
>> --James Evans
>>
>>
>>> Date: 2007年12月11日 15:47:45 -0500
>>> From: Michael Droettboom <md...@st...>
>>> Subject: Re: [matplotlib-devel] Problem with Agg Ellipses
>>> To: Ted Drain <ted...@jp...>
>>> Cc: matplotlib development list
>>> 	<mat...@li...>
>>> Message-ID: <475...@st...>
>>> Content-Type: text/plain; charset="iso-8859-1"
>>>
>>> And an actually interesting part of the plot... ;)
>>>
>>> Michael Droettboom wrote:
>>>> Sorry -- correct attachment this time.
>>>>
>>>> Michael Droettboom wrote:
>>>>> I have a working draft of something that may work for this problem on
>>>>> the transforms branch. I am happy to backport this to the trunk, but
>>>>> that will require some effort, as the implementation relies on many of
>>>>> the new geometric utilities on the branch that would also have to be
>>>>> brought over. It may be some time until the branch is ready for
>>>>> production use, but if you are able to use it to experiment with this
>>>>> approach to this specific problem, that would certainly make my life
>>>>> easier, so I don't have to do the backporting work more than once.
>>>>>
>>>>> Attached is a plot comparing the new approach (above) vs. a 750-edge
>>>>> polygonal approximation for the ellipses (based directly on James
>>>>> Evans' example). Here's a description of what it does:
>>>>>
>>>>>
>>>>> Ellipses are normally drawn using an approximation that uses
>>>>> eight cubic bezier splines. The error of this approximation
>>>>> is 1.89818e-6, according to this unverified source:
>>>>>
>>>>> Lancaster, Don. Approximating a Circle or an Ellipse Using
>>>>> Four Bezier Cubic Splines.
>>>>>
>>>>> http://www.tinaja.com/glib/ellipse4.pdf
>>>>>
>>>>> There is a use case where very large ellipses must be drawn
>>>>> with very high accuracy, and it is too expensive to render the
>>>>> entire ellipse with enough segments (either splines or line
>>>>> segments). Therefore, in the case where either radius of the
>>>>> ellipse is large enough that the error of the spline
>>>>> approximation will be visible (greater than one pixel offset
>>>>> from the ideal), a different technique is used.
>>>>>
>>>>> In that case, only the visible parts of the ellipse are drawn,
>>>>> with each visible arc using a fixed number of spline segments
>>>>> (8). The algorithm proceeds as follows:
>>>>>
>>>>> 1. The points where the ellipse intersects the axes bounding
>>>>> box are located. (This is done be performing an inverse
>>>>> transformation on the axes bbox such that it is relative to
>>>>> the unit circle -- this makes the intersection calculation
>>>>> much easier than doing rotated ellipse intersection
>>>>> directly).
>>>>>
>>>>> This uses the "line intersecting a circle" algorithm from:
>>>>>
>>>>> Vince, John. Geometry for Computer Graphics: Formulae,
>>>>> Examples & Proofs. London: Springer-Verlag, 2005.
>>>>>
>>>>> 2. The angles of each of the intersection points are
>>>>> calculated.
>>>>>
>>>>> 3. Proceeding counterclockwise starting in the positive
>>>>> x-direction, each of the visible arc-segments between the
>>>>> pairs of vertices are drawn using the bezier arc
>>>>> approximation technique implemented in Path.arc().
>>>>>
>>>>>
>>>>> Cheers,
>>>>> Mike
>>>>>
>>>>>
>>>>> Ted Drain wrote:
>>>>>> All of these sound like good ideas to me. Just for some history: the
>>>>>> original reason we worked w/ John to get an Ellipse primitive in (vs
>>>>>> a normal line plot of sampled points) were to:
>>>>>> - improve ellipse plotting speeds (we plot a LOT of them at once)
>>>>>> - improve post script output
>>>>>>
>>>>>> Ted
>>>>>>
>>>>>> At 08:53 AM 12/10/2007, Michael Droettboom wrote:
>>>>>>> John Hunter wrote:
>>>>>>>> On Dec 10, 2007 10:25 AM, Ted Drain <ted...@jp...> wrote:
>>>>>>>>
>>>>>>>>> I don't know if the current MPL architecture can support this but it
>>>>>>>>> would be nice if it worked that way. We have people making decisions
>>>>>>>>> based on what these plots show that affect spacecraft worth hundreds
>>>>>>>>> of millions of dollars so it's important that we're plotting
>>>>>>> things accurately.
>>>>>>>> We can support this, but I think we would do this with an arc class
>>>>>>>> rather than an ellipse class, and write a special case class that is
>>>>>>>> viewlim aware.
>>>>>>> I agree -- I think there are two uses cases for ellipse that are in
>>>>>>> conflict here. One is these large ellipses, the other is for things
>>>>>>> like scatter plots, where speed and file size is more important than
>>>>>>> accuracy. My mind was probably stuck on the latter as I've worked
>>>>>>> along
>>>>>>> the transforms branch.
>>>>>>>
>>>>>>>> A simple example of a line that has analogous
>>>>>>>> behavior is examples/clippedline.py, which clips the points outside
>>>>>>>> the viewport and draws in a different style according to the
>>>>>>>> resolution of the viewlim. The reason I think it would be preferable
>>>>>>>> to use an arc here is because we won't have to worry about filling the
>>>>>>>> thing when we only approximate a section of it. You could feed in a
>>>>>>>> 360 degree elliptical arc and then zoom into a portion of it.
>>>>>>>>
>>>>>>>> With the 8 point ellipse as is, and the addition of an arc class that
>>>>>>>> does 4 or 8 point approximation within the zoom limits, should that
>>>>>>>> serve your requirements?
>>>>>>> As a possible starting point, the transforms branch already has
>>>>>>> arc-approximation-by-cubic-bezier-spline code. It determines the
>>>>>>> number
>>>>>>> of splines to use based on the radians included in the arc, which is
>>>>>>> clearly not what we want here. But it should be reasonably
>>>>>>> straightforward to make that some fixed number and draw the arc between
>>>>>>> the edges of the axes. Or, alternatively, (and maybe this is what John
>>>>>>> is suggesting), the arc could be approximated by line segments (with
>>>>>>> the
>>>>>>> number of segments something like the number of pixels across the
>>>>>>> axes).
>>>>>>> To my naive mind, that seems more verifiable -- or at least it puts
>>>>>>> the responsibility of getting this right all in one place.
>>>>>>>
>>>>>>> IMHO, these spline approximation tricks are all just with the aim of
>>>>>>> pushing curve rendering deeper into the backends for speed and file
>>>>>>> size
>>>>>>> improvements. But obviously there needs to be a way around it when it
>>>>>>> matters.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Mike
>>>>>>>
>>>>>>> --
>>>>>>> Michael Droettboom
>>>>>>> Science Software Branch
>>>>>>> Operations and Engineering Division
>>>>>>> Space Telescope Science Institute
>>>>>>> Operated by AURA for NASA
>>>>>>>
>>>>>>> -------------------------------------------------------------------------
>>>>>>>
>>>>>>> SF.Net email is sponsored by:
>>>>>>> Check out the new SourceForge.net Marketplace.
>>>>>>> It's the best place to buy or sell services for
>>>>>>> just about anything Open Source.
>>>>>>> http://sourceforge.net/services/buy/index.php
>>>>>>> _______________________________________________
>>>>>>> Matplotlib-devel mailing list
>>>>>>> Mat...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>>>> Ted Drain Jet Propulsion Laboratory ted...@jp...
>>>>>>
>>>>>> -------------------------------------------------------------------------
>>>>>>
>>>>>> SF.Net email is sponsored by: Check out the new SourceForge.net
>>>>>> Marketplace.
>>>>>> It's the best place to buy or sell services for
>>>>>> just about anything Open Source.
>>>>>> http://sourceforge.net/services/buy/index.php
>>>>>> _______________________________________________
>>>>>> Matplotlib-devel mailing list
>>>>>> Mat...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>> -------------------------------------------------------------------------
>>>>> SF.Net email is sponsored by: Check out the new SourceForge.net
>>>>> Marketplace.
>>>>> It's the best place to buy or sell services for
>>>>> just about anything Open Source.
>>>>> http://sourceforge.net/services/buy/index.php
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> Matplotlib-devel mailing list
>>>>> Mat...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> -------------------------------------------------------------------------
>>>> SF.Net email is sponsored by:
>>>> Check out the new SourceForge.net Marketplace.
>>>> It's the best place to buy or sell services for
>>>> just about anything Open Source.
>>>> http://sourceforge.net/services/buy/index.php
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Mat...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>> --
>>> Michael Droettboom
>>> Science Software Branch
>>> Operations and Engineering Division
>>> Space Telescope Science Institute
>>> Operated by AURA for NASA
>>> -------------- next part --------------
>>> A non-text attachment was scrubbed...
>>> Name: ellipse.png
>>> Type: image/png
>>> Size: 49913 bytes
>>> Desc: not available
>>>
>>> ------------------------------
>>>
>>> -------------------------------------------------------------------------
>>> SF.Net email is sponsored by:
>>> Check out the new SourceForge.net Marketplace.
>>> It's the best place to buy or sell services for
>>> just about anything Open Source.
>>> http://sourceforge.net/services/buy/index.php
>>>
>>> ------------------------------
>>>
>>> _______________________________________________
>>> Matplotlib-devel mailing list
>>> Mat...@li...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>
>>>
>>> End of Matplotlib-devel Digest, Vol 19, Issue 16
>>> ************************************************
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2005.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Porting this back to the trunk is non-trivial -- but I know John would 
like to do it, so I think it will happen once one of us has time.
As an aside, the unit stuff *should* be working on the branch. Can you 
send me a minimal example that shows it failing?
Cheers,
Mike
James Evans wrote:
> I have taken the transforms branch and played with it a bit and my super simple ellipse test cases appeared to be working great. I
> couldn't run our more intensive tests since they use unitized data and I was getting errors that looked like the transforms branch
> wasn't completely handling unitized data properly. It would be great if we could see this fix in the main branch, then we can make
> use of it right away without having to wait for the transforms branch to be completed.
> 
> --James Evans
> 
> 
>> Date: 2007年12月11日 15:47:45 -0500
>> From: Michael Droettboom <md...@st...>
>> Subject: Re: [matplotlib-devel] Problem with Agg Ellipses
>> To: Ted Drain <ted...@jp...>
>> Cc: matplotlib development list
>> 	<mat...@li...>
>> Message-ID: <475...@st...>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> And an actually interesting part of the plot... ;)
>>
>> Michael Droettboom wrote:
>>> Sorry -- correct attachment this time.
>>>
>>> Michael Droettboom wrote:
>>>> I have a working draft of something that may work for this problem on
>>>> the transforms branch. I am happy to backport this to the trunk, but
>>>> that will require some effort, as the implementation relies on many of
>>>> the new geometric utilities on the branch that would also have to be
>>>> brought over. It may be some time until the branch is ready for
>>>> production use, but if you are able to use it to experiment with this
>>>> approach to this specific problem, that would certainly make my life
>>>> easier, so I don't have to do the backporting work more than once.
>>>>
>>>> Attached is a plot comparing the new approach (above) vs. a 750-edge
>>>> polygonal approximation for the ellipses (based directly on James
>>>> Evans' example). Here's a description of what it does:
>>>>
>>>>
>>>> Ellipses are normally drawn using an approximation that uses
>>>> eight cubic bezier splines. The error of this approximation
>>>> is 1.89818e-6, according to this unverified source:
>>>>
>>>> Lancaster, Don. Approximating a Circle or an Ellipse Using
>>>> Four Bezier Cubic Splines.
>>>>
>>>> http://www.tinaja.com/glib/ellipse4.pdf
>>>>
>>>> There is a use case where very large ellipses must be drawn
>>>> with very high accuracy, and it is too expensive to render the
>>>> entire ellipse with enough segments (either splines or line
>>>> segments). Therefore, in the case where either radius of the
>>>> ellipse is large enough that the error of the spline
>>>> approximation will be visible (greater than one pixel offset
>>>> from the ideal), a different technique is used.
>>>>
>>>> In that case, only the visible parts of the ellipse are drawn,
>>>> with each visible arc using a fixed number of spline segments
>>>> (8). The algorithm proceeds as follows:
>>>>
>>>> 1. The points where the ellipse intersects the axes bounding
>>>> box are located. (This is done be performing an inverse
>>>> transformation on the axes bbox such that it is relative to
>>>> the unit circle -- this makes the intersection calculation
>>>> much easier than doing rotated ellipse intersection
>>>> directly).
>>>>
>>>> This uses the "line intersecting a circle" algorithm from:
>>>>
>>>> Vince, John. Geometry for Computer Graphics: Formulae,
>>>> Examples & Proofs. London: Springer-Verlag, 2005.
>>>>
>>>> 2. The angles of each of the intersection points are
>>>> calculated.
>>>>
>>>> 3. Proceeding counterclockwise starting in the positive
>>>> x-direction, each of the visible arc-segments between the
>>>> pairs of vertices are drawn using the bezier arc
>>>> approximation technique implemented in Path.arc().
>>>>
>>>>
>>>> Cheers,
>>>> Mike
>>>>
>>>>
>>>> Ted Drain wrote:
>>>>> All of these sound like good ideas to me. Just for some history: the
>>>>> original reason we worked w/ John to get an Ellipse primitive in (vs
>>>>> a normal line plot of sampled points) were to:
>>>>> - improve ellipse plotting speeds (we plot a LOT of them at once)
>>>>> - improve post script output
>>>>>
>>>>> Ted
>>>>>
>>>>> At 08:53 AM 12/10/2007, Michael Droettboom wrote:
>>>>>> John Hunter wrote:
>>>>>>> On Dec 10, 2007 10:25 AM, Ted Drain <ted...@jp...> wrote:
>>>>>>>
>>>>>>>> I don't know if the current MPL architecture can support this but it
>>>>>>>> would be nice if it worked that way. We have people making decisions
>>>>>>>> based on what these plots show that affect spacecraft worth hundreds
>>>>>>>> of millions of dollars so it's important that we're plotting
>>>>>> things accurately.
>>>>>>> We can support this, but I think we would do this with an arc class
>>>>>>> rather than an ellipse class, and write a special case class that is
>>>>>>> viewlim aware.
>>>>>> I agree -- I think there are two uses cases for ellipse that are in
>>>>>> conflict here. One is these large ellipses, the other is for things
>>>>>> like scatter plots, where speed and file size is more important than
>>>>>> accuracy. My mind was probably stuck on the latter as I've worked
>>>>>> along
>>>>>> the transforms branch.
>>>>>>
>>>>>>> A simple example of a line that has analogous
>>>>>>> behavior is examples/clippedline.py, which clips the points outside
>>>>>>> the viewport and draws in a different style according to the
>>>>>>> resolution of the viewlim. The reason I think it would be preferable
>>>>>>> to use an arc here is because we won't have to worry about filling the
>>>>>>> thing when we only approximate a section of it. You could feed in a
>>>>>>> 360 degree elliptical arc and then zoom into a portion of it.
>>>>>>>
>>>>>>> With the 8 point ellipse as is, and the addition of an arc class that
>>>>>>> does 4 or 8 point approximation within the zoom limits, should that
>>>>>>> serve your requirements?
>>>>>> As a possible starting point, the transforms branch already has
>>>>>> arc-approximation-by-cubic-bezier-spline code. It determines the
>>>>>> number
>>>>>> of splines to use based on the radians included in the arc, which is
>>>>>> clearly not what we want here. But it should be reasonably
>>>>>> straightforward to make that some fixed number and draw the arc between
>>>>>> the edges of the axes. Or, alternatively, (and maybe this is what John
>>>>>> is suggesting), the arc could be approximated by line segments (with
>>>>>> the
>>>>>> number of segments something like the number of pixels across the
>>>>>> axes).
>>>>>> To my naive mind, that seems more verifiable -- or at least it puts
>>>>>> the responsibility of getting this right all in one place.
>>>>>>
>>>>>> IMHO, these spline approximation tricks are all just with the aim of
>>>>>> pushing curve rendering deeper into the backends for speed and file
>>>>>> size
>>>>>> improvements. But obviously there needs to be a way around it when it
>>>>>> matters.
>>>>>>
>>>>>> Cheers,
>>>>>> Mike
>>>>>>
>>>>>> --
>>>>>> Michael Droettboom
>>>>>> Science Software Branch
>>>>>> Operations and Engineering Division
>>>>>> Space Telescope Science Institute
>>>>>> Operated by AURA for NASA
>>>>>>
>>>>>> -------------------------------------------------------------------------
>>>>>>
>>>>>> SF.Net email is sponsored by:
>>>>>> Check out the new SourceForge.net Marketplace.
>>>>>> It's the best place to buy or sell services for
>>>>>> just about anything Open Source.
>>>>>> http://sourceforge.net/services/buy/index.php
>>>>>> _______________________________________________
>>>>>> Matplotlib-devel mailing list
>>>>>> Mat...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>>> Ted Drain Jet Propulsion Laboratory ted...@jp...
>>>>>
>>>>> -------------------------------------------------------------------------
>>>>>
>>>>> SF.Net email is sponsored by: Check out the new SourceForge.net
>>>>> Marketplace.
>>>>> It's the best place to buy or sell services for
>>>>> just about anything Open Source.
>>>>> http://sourceforge.net/services/buy/index.php
>>>>> _______________________________________________
>>>>> Matplotlib-devel mailing list
>>>>> Mat...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> -------------------------------------------------------------------------
>>>> SF.Net email is sponsored by: Check out the new SourceForge.net
>>>> Marketplace.
>>>> It's the best place to buy or sell services for
>>>> just about anything Open Source.
>>>> http://sourceforge.net/services/buy/index.php
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Matplotlib-devel mailing list
>>>> Mat...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> -------------------------------------------------------------------------
>>> SF.Net email is sponsored by:
>>> Check out the new SourceForge.net Marketplace.
>>> It's the best place to buy or sell services for
>>> just about anything Open Source.
>>> http://sourceforge.net/services/buy/index.php
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Matplotlib-devel mailing list
>>> Mat...@li...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>> --
>> Michael Droettboom
>> Science Software Branch
>> Operations and Engineering Division
>> Space Telescope Science Institute
>> Operated by AURA for NASA
>> -------------- next part --------------
>> A non-text attachment was scrubbed...
>> Name: ellipse.png
>> Type: image/png
>> Size: 49913 bytes
>> Desc: not available
>>
>> ------------------------------
>>
>> -------------------------------------------------------------------------
>> SF.Net email is sponsored by:
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services for
>> just about anything Open Source.
>> http://sourceforge.net/services/buy/index.php
>>
>> ------------------------------
>>
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>>
>> End of Matplotlib-devel Digest, Vol 19, Issue 16
>> ************************************************
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

I have taken the transforms branch and played with it a bit and my super simple ellipse test cases appeared to be working great. I
couldn't run our more intensive tests since they use unitized data and I was getting errors that looked like the transforms branch
wasn't completely handling unitized data properly. It would be great if we could see this fix in the main branch, then we can make
use of it right away without having to wait for the transforms branch to be completed.
--James Evans
> Date: 2007年12月11日 15:47:45 -0500
> From: Michael Droettboom <md...@st...>
> Subject: Re: [matplotlib-devel] Problem with Agg Ellipses
> To: Ted Drain <ted...@jp...>
> Cc: matplotlib development list
> 	<mat...@li...>
> Message-ID: <475...@st...>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> And an actually interesting part of the plot... ;)
> 
> Michael Droettboom wrote:
> > Sorry -- correct attachment this time.
> >
> > Michael Droettboom wrote:
> >> I have a working draft of something that may work for this problem on
> >> the transforms branch. I am happy to backport this to the trunk, but
> >> that will require some effort, as the implementation relies on many of
> >> the new geometric utilities on the branch that would also have to be
> >> brought over. It may be some time until the branch is ready for
> >> production use, but if you are able to use it to experiment with this
> >> approach to this specific problem, that would certainly make my life
> >> easier, so I don't have to do the backporting work more than once.
> >>
> >> Attached is a plot comparing the new approach (above) vs. a 750-edge
> >> polygonal approximation for the ellipses (based directly on James
> >> Evans' example). Here's a description of what it does:
> >>
> >>
> >> Ellipses are normally drawn using an approximation that uses
> >> eight cubic bezier splines. The error of this approximation
> >> is 1.89818e-6, according to this unverified source:
> >>
> >> Lancaster, Don. Approximating a Circle or an Ellipse Using
> >> Four Bezier Cubic Splines.
> >>
> >> http://www.tinaja.com/glib/ellipse4.pdf
> >>
> >> There is a use case where very large ellipses must be drawn
> >> with very high accuracy, and it is too expensive to render the
> >> entire ellipse with enough segments (either splines or line
> >> segments). Therefore, in the case where either radius of the
> >> ellipse is large enough that the error of the spline
> >> approximation will be visible (greater than one pixel offset
> >> from the ideal), a different technique is used.
> >>
> >> In that case, only the visible parts of the ellipse are drawn,
> >> with each visible arc using a fixed number of spline segments
> >> (8). The algorithm proceeds as follows:
> >>
> >> 1. The points where the ellipse intersects the axes bounding
> >> box are located. (This is done be performing an inverse
> >> transformation on the axes bbox such that it is relative to
> >> the unit circle -- this makes the intersection calculation
> >> much easier than doing rotated ellipse intersection
> >> directly).
> >>
> >> This uses the "line intersecting a circle" algorithm from:
> >>
> >> Vince, John. Geometry for Computer Graphics: Formulae,
> >> Examples & Proofs. London: Springer-Verlag, 2005.
> >>
> >> 2. The angles of each of the intersection points are
> >> calculated.
> >>
> >> 3. Proceeding counterclockwise starting in the positive
> >> x-direction, each of the visible arc-segments between the
> >> pairs of vertices are drawn using the bezier arc
> >> approximation technique implemented in Path.arc().
> >>
> >>
> >> Cheers,
> >> Mike
> >>
> >>
> >> Ted Drain wrote:
> >>> All of these sound like good ideas to me. Just for some history: the
> >>> original reason we worked w/ John to get an Ellipse primitive in (vs
> >>> a normal line plot of sampled points) were to:
> >>> - improve ellipse plotting speeds (we plot a LOT of them at once)
> >>> - improve post script output
> >>>
> >>> Ted
> >>>
> >>> At 08:53 AM 12/10/2007, Michael Droettboom wrote:
> >>>> John Hunter wrote:
> >>>>> On Dec 10, 2007 10:25 AM, Ted Drain <ted...@jp...> wrote:
> >>>>>
> >>>>>> I don't know if the current MPL architecture can support this but it
> >>>>>> would be nice if it worked that way. We have people making decisions
> >>>>>> based on what these plots show that affect spacecraft worth hundreds
> >>>>>> of millions of dollars so it's important that we're plotting
> >>>> things accurately.
> >>>>> We can support this, but I think we would do this with an arc class
> >>>>> rather than an ellipse class, and write a special case class that is
> >>>>> viewlim aware.
> >>>> I agree -- I think there are two uses cases for ellipse that are in
> >>>> conflict here. One is these large ellipses, the other is for things
> >>>> like scatter plots, where speed and file size is more important than
> >>>> accuracy. My mind was probably stuck on the latter as I've worked
> >>>> along
> >>>> the transforms branch.
> >>>>
> >>>>> A simple example of a line that has analogous
> >>>>> behavior is examples/clippedline.py, which clips the points outside
> >>>>> the viewport and draws in a different style according to the
> >>>>> resolution of the viewlim. The reason I think it would be preferable
> >>>>> to use an arc here is because we won't have to worry about filling the
> >>>>> thing when we only approximate a section of it. You could feed in a
> >>>>> 360 degree elliptical arc and then zoom into a portion of it.
> >>>>>
> >>>>> With the 8 point ellipse as is, and the addition of an arc class that
> >>>>> does 4 or 8 point approximation within the zoom limits, should that
> >>>>> serve your requirements?
> >>>> As a possible starting point, the transforms branch already has
> >>>> arc-approximation-by-cubic-bezier-spline code. It determines the
> >>>> number
> >>>> of splines to use based on the radians included in the arc, which is
> >>>> clearly not what we want here. But it should be reasonably
> >>>> straightforward to make that some fixed number and draw the arc between
> >>>> the edges of the axes. Or, alternatively, (and maybe this is what John
> >>>> is suggesting), the arc could be approximated by line segments (with
> >>>> the
> >>>> number of segments something like the number of pixels across the
> >>>> axes).
> >>>> To my naive mind, that seems more verifiable -- or at least it puts
> >>>> the responsibility of getting this right all in one place.
> >>>>
> >>>> IMHO, these spline approximation tricks are all just with the aim of
> >>>> pushing curve rendering deeper into the backends for speed and file
> >>>> size
> >>>> improvements. But obviously there needs to be a way around it when it
> >>>> matters.
> >>>>
> >>>> Cheers,
> >>>> Mike
> >>>>
> >>>> --
> >>>> Michael Droettboom
> >>>> Science Software Branch
> >>>> Operations and Engineering Division
> >>>> Space Telescope Science Institute
> >>>> Operated by AURA for NASA
> >>>>
> >>>> -------------------------------------------------------------------------
> >>>>
> >>>> SF.Net email is sponsored by:
> >>>> Check out the new SourceForge.net Marketplace.
> >>>> It's the best place to buy or sell services for
> >>>> just about anything Open Source.
> >>>> http://sourceforge.net/services/buy/index.php
> >>>> _______________________________________________
> >>>> Matplotlib-devel mailing list
> >>>> Mat...@li...
> >>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> >>>
> >>> Ted Drain Jet Propulsion Laboratory ted...@jp...
> >>>
> >>> -------------------------------------------------------------------------
> >>>
> >>> SF.Net email is sponsored by: Check out the new SourceForge.net
> >>> Marketplace.
> >>> It's the best place to buy or sell services for
> >>> just about anything Open Source.
> >>> http://sourceforge.net/services/buy/index.php
> >>> _______________________________________________
> >>> Matplotlib-devel mailing list
> >>> Mat...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> >>
> >>
> >> ------------------------------------------------------------------------
> >>
> >>
> >> ------------------------------------------------------------------------
> >>
> >> -------------------------------------------------------------------------
> >> SF.Net email is sponsored by: Check out the new SourceForge.net
> >> Marketplace.
> >> It's the best place to buy or sell services for
> >> just about anything Open Source.
> >> http://sourceforge.net/services/buy/index.php
> >>
> >>
> >> ------------------------------------------------------------------------
> >>
> >> _______________________________________________
> >> Matplotlib-devel mailing list
> >> Mat...@li...
> >> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> >
> >
> > ------------------------------------------------------------------------
> >
> >
> > ------------------------------------------------------------------------
> >
> > -------------------------------------------------------------------------
> > SF.Net email is sponsored by:
> > Check out the new SourceForge.net Marketplace.
> > It's the best place to buy or sell services for
> > just about anything Open Source.
> > http://sourceforge.net/services/buy/index.php
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Matplotlib-devel mailing list
> > Mat...@li...
> > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
> --
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: ellipse.png
> Type: image/png
> Size: 49913 bytes
> Desc: not available
> 
> ------------------------------
> 
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> 
> ------------------------------
> 
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
> 
> End of Matplotlib-devel Digest, Vol 19, Issue 16
> ************************************************

Hi,
when matplotlib processes variables, it makes sure to handle masked
variables (numpy.ma) by converting them using "soft "
numpy.ma.asarray. So, when a subclass instance of numpy.ma is used as
a variable, it keeps properties and methods in this operation that can
be conflicting with future processing.
An example of problematic input variable can be cdms variable (from
CDAT - cdat.sourceforge.net/). Such a variable is like a numpy.ma with
axes (like time, longitude, latitude, etc) and attributes (like
missing value, name, units). A problem for example is that it cannot
handle "newaxis" slicings (var[:,newaxis]) that matplotlib uses to be
sure that a variable has a suitable rank. In addition, other
properties of cdms variable ares not interesting for matplotlib
processing.
Therefore, it may be useful to strictly convert input variables to
pure numpy.ma using something like numpy.ma.array(var,copy=0).
Is it feasible?
-- 
Stephane Raynaud

Thanks for looking at this. From the unittest that you checked in, it 
appears you've succeeded in getting dates and datetimes to work, modulo 
the apparent datetime precision buglet you mention below.
My understanding with repr() is that, in general, it should attempt to 
support lossless copying of data with the eval() function. It does 
appear that dateutils does this -- repr() produces a string that looks 
like what you'd use to call the constructor. (Numpy scalars don't follow 
this idiom, and it's doubtful they could change without breaking a lot 
of code.) I also agree that for a CSV file, something like the str() on 
date and datetime objects makes most sense. This is all just to say that 
the date and datetime stuff looks good to my eye, but you're using that 
stuff a lot more than me. Finally, from your example, it does look like 
there's a bug somewhere in the datetime code. Since I don't use them, I 
leave that to you...
Finally, as far as the checkrows -- I must admit that I wasn't even 
aware of its existence until you asked the question. (That's the problem 
with writing code that just works - no one notices it!) Now that you've 
posed the question, I suppose I'm in favor a default of 0 (which 
actually means infinity). I suppose no one in their right mind is going 
to use CSV files to store gigabytes of data, so parsing the whole thing 
for consistency seems like a worthwhile expenditure of cycles. (And if 
they're thinking about it, maybe the slowness with dissuade them, or at 
least cause them to read the docstrings! :)
John Hunter wrote:
> On Dec 16, 2007 5:26 PM, Andrew Straw <str...@as...> wrote:
>
> 
>> As a followup to the work on floats, I fixed rec2csv to deal with funky
>> strings (strings with commas and quotes). I've checked this is as r4749,
>> but I thought I'd announce here in case someone (John, in particular)
>> was depending on some peculiar aspect of the old implementation. If
>> there is something you depend on, can you update the unittests in
>> unit/mlab_unit.py to check for the behavior you need?
>> 
>
> OK, I found one problem with repr. When adding support for date vs
> datetime to the csv2rec roundtrip, and adding this to the unit test, I
> notice that repr is probably not what we want for datetime; str makes
> more sense here:
>
> In [1]: import datetime
>
> In [2]: d = datetime.date.today()
>
> In [3]: repr(d)
> Out[3]: 'datetime.date(2007, 12, 16)'
>
> In [4]: str(d)
> Out[4]: '2007-12-16'
>
> The latter is more natural in the CSV file, and the repr version is
> not supported by dateutil, at least not the one we are shipping:
>
> In [5]: import dateutil.parser
>
> In [6]: dateutil.parser.parse(repr(d))
> ---------------------------------------------------------------------------
> ValueError Traceback (most recent call last)
>
> /Users/jdhunter/python/svn/matplotlib/examples/<ipython console> in <module>()
>
> ...snip the rest of the traceback...
>
> In [7]: dateutil.parser.parse(str(d))
> Out[7]: datetime.datetime(2007, 12, 16, 0, 0)
>
>
> So I changed FormatObject to use str, pending further discussion. At
> least for my common use cases, the only obj types I have in my record
> arrays are dates and datetimes, and I find this to be a pretty
> compelling use case since it is the type least likely to be supported
> by other persistence methods (tostring and pickle both fail or do not
> behave as expected with datetimes in the recarray).
>
> But there is an oddity in the parsing of milliseconds which is causing
> the updated unit test to fail; the code below illustrates the problem:
>
> In [3]: import dateutil.parser
>
> In [4]: import datetime
>
> In [5]: s = '2007-12-18 22:29:34.924122'
>
> In [6]: dateutil.parser.parse(s)
> Out[6]: datetime.datetime(2007, 12, 18, 22, 29, 34, 924121)
>
> Thoughts?
>
> JDH
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 

Darren Dale wrote:
> On Thursday 13 December 2007 2:22:13 pm Robert Kern wrote:
>> John Hunter wrote:
>>> Do we need namespace packages in toolkits? I recently added gtktools
>>> and exceltools to toolkits, and got a very hard to debug error:
>>>
>>> In [1]: import matplotlib
>>>
>>> In [2]: matplotlib.__file__
>>> Out[2]:
>>> '/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/__init__.py
>>> c'
>>>
>>> In [3]: matplotlib.rcParams['axes.axisbelow']
>>> Out[3]: False
>>>
>>> In [4]: import matplotlib.toolkits.gtktools
>>>
>>> In [5]: matplotlib.__file__
>>> Out[5]:
>>> '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/__ini
>>> t__.pyc'
>>>
>>> In [6]: matplotlib.rcParams['axes.axisbelow']
>>> ------------------------------------------------------------
>>> Traceback (most recent call last):
>>> File "<ipython console>", line 1, in ?
>>> KeyError: 'axes.axisbelow'
>>>
>>> Notice that the matplotlib module which was previously imported got
>>> changed by the import of the toolkit which declared itself a namespace
>>> package. Now this may be the result of mpl not using namespace
>>> packages correctly (eg matplotlib/__init__.py should be empty save for
>>> the namespace package or something like that)
>> Probably. In setuptools 0.7, they can have stuff in them, but all of them
>> must be identical.
> 
> How would one learn about this new feature? According to setuptools.txt from 
> the 0.7 svn checkout, the namespace package's __init__.py must still be empty 
> save the namespace declaration.
It's a logical consequence of how namespace packages are loaded in 0.7 as has
been explained on the mailing list whenever the subject of namespace packages in
0.7 is brought up. It's still not recommended, but it can be made to work. The
remaining objection in the documentation is about installing the package using
--single-version-externally-managed for distro packagers. Using --svem, the
__init__.py files are not installed. Since a distro package would have a
mynamespace-common package (or something similar) that *just* provides this
__init__.py file for all of the mynamespace.foo packages, it can provide the
single copy of the meaningful code.
But it's still not a good idea.
>> If you are using an unmodified matplotlib/__init__.py in 
>> the main matplotlib package, then the "if 0:" is preventing the declaration
>> of the namespace package, and that may be contributing.
>>
>> Also, neither matplotlib nor matplotlib.toolkits is named as a
>> namespace_package in matplotlib's setup.py, and both have to be. Similarly,
>> basemap (for example) only lists matplotlib.toolkits but not matplotlib.
> 
> If I understand correctly, even after fixing these two problems, namespace 
> packages would not work since matplotlib/__init__.py contains lots of 
> meaningful code. Based on the remarks at 
> http://mail.python.org/pipermail/distutils-sig/2007-October/008346.html and 
> the setuptools svn commit log, it looks like development on version 0.7 is 
> not a priority and a release in the next year is unlikely. So is it fair to 
> say that for the foreseeable future, matplotlib is not compatible with 
> namespace packages unless we cleaned out __init__.py? 
Yes. Phillip explained this in February and suggested a way around the issue:
 http://www.mail-archive.com/dis...@py.../msg02908.html
-- 
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
 -- Umberto Eco

On Dec 16, 2007 9:10 PM, John Hunter <jd...@gm...> wrote:
> On Dec 16, 2007 5:26 PM, Andrew Straw <str...@as...> wrote:
>
> > As a followup to the work on floats, I fixed rec2csv to deal with funky
> > strings (strings with commas and quotes). I've checked this is as r4749,
> > but I thought I'd announce here in case someone (John, in particular)
> > was depending on some peculiar aspect of the old implementation. If
> > there is something you depend on, can you update the unittests in
> > unit/mlab_unit.py to check for the behavior you need?
> >
> > Also, John, I didn't see mlab.FormatString being used anywhere, but I
> > didn't want to change it, either. So I made FormatString2 and used it.
> > But if there's no known use of FormatString, lets kill the original and
> > more FormatString2 into its place.
>
> I am happy to leave it as is, and fix it if I bump into anything.
> There was a use case at work that made me quote all the strings with "
> but I am not sure what it is right now so I will test with your
> version tomorrow and see if I can find any problems.
>
> There is one more thing I'd like to clean up in this code, and that is
> the handling of date and datetime. The current implementation will
> return datetime regardless of whether the string looks like a date or
> a datetime. Thus it would fail a roundtrip tests if you had a date
> field. I may make some changes tomorrow to support date or datetime.
I'm CCing Chris Burns, from Berkeley here, but it would be great if
you post a note of these changes on the numpy list. One of the things
to come out of the sprint was a lot of work on numpy i/o, and that
includes looking into integrating much of the MPL facilities. So it
would be great if you keep the numpy team posted, to make sure they
get your latest code.
Cheers.
>
> What do you think of making checkrows=0 the default for csv2rec? It
> is slower, but mostly guaranteed to do the right thing, whereas
> checkrows=5 which is the current default can often fail if the first 5
> rows match some type but later rows do not, and may lead to confusion
> among new users.

On Dec 16, 2007 5:26 PM, Andrew Straw <str...@as...> wrote:
> As a followup to the work on floats, I fixed rec2csv to deal with funky
> strings (strings with commas and quotes). I've checked this is as r4749,
> but I thought I'd announce here in case someone (John, in particular)
> was depending on some peculiar aspect of the old implementation. If
> there is something you depend on, can you update the unittests in
> unit/mlab_unit.py to check for the behavior you need?
OK, I found one problem with repr. When adding support for date vs
datetime to the csv2rec roundtrip, and adding this to the unit test, I
notice that repr is probably not what we want for datetime; str makes
more sense here:
In [1]: import datetime
In [2]: d = datetime.date.today()
In [3]: repr(d)
Out[3]: 'datetime.date(2007, 12, 16)'
In [4]: str(d)
Out[4]: '2007-12-16'
The latter is more natural in the CSV file, and the repr version is
not supported by dateutil, at least not the one we are shipping:
In [5]: import dateutil.parser
In [6]: dateutil.parser.parse(repr(d))
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/Users/jdhunter/python/svn/matplotlib/examples/<ipython console> in <module>()
...snip the rest of the traceback...
In [7]: dateutil.parser.parse(str(d))
Out[7]: datetime.datetime(2007, 12, 16, 0, 0)
So I changed FormatObject to use str, pending further discussion. At
least for my common use cases, the only obj types I have in my record
arrays are dates and datetimes, and I find this to be a pretty
compelling use case since it is the type least likely to be supported
by other persistence methods (tostring and pickle both fail or do not
behave as expected with datetimes in the recarray).
But there is an oddity in the parsing of milliseconds which is causing
the updated unit test to fail; the code below illustrates the problem:
In [3]: import dateutil.parser
In [4]: import datetime
In [5]: s = '2007-12-18 22:29:34.924122'
In [6]: dateutil.parser.parse(s)
Out[6]: datetime.datetime(2007, 12, 18, 22, 29, 34, 924121)
Thoughts?
JDH

On Dec 16, 2007 5:26 PM, Andrew Straw <str...@as...> wrote:
> As a followup to the work on floats, I fixed rec2csv to deal with funky
> strings (strings with commas and quotes). I've checked this is as r4749,
> but I thought I'd announce here in case someone (John, in particular)
> was depending on some peculiar aspect of the old implementation. If
> there is something you depend on, can you update the unittests in
> unit/mlab_unit.py to check for the behavior you need?
>
> Also, John, I didn't see mlab.FormatString being used anywhere, but I
> didn't want to change it, either. So I made FormatString2 and used it.
> But if there's no known use of FormatString, lets kill the original and
> more FormatString2 into its place.
I am happy to leave it as is, and fix it if I bump into anything.
There was a use case at work that made me quote all the strings with "
but I am not sure what it is right now so I will test with your
version tomorrow and see if I can find any problems.
There is one more thing I'd like to clean up in this code, and that is
the handling of date and datetime. The current implementation will
return datetime regardless of whether the string looks like a date or
a datetime. Thus it would fail a roundtrip tests if you had a date
field. I may make some changes tomorrow to support date or datetime.
What do you think of making checkrows=0 the default for csv2rec? It
is slower, but mostly guaranteed to do the right thing, whereas
checkrows=5 which is the current default can often fail if the first 5
rows match some type but later rows do not, and may lead to confusion
among new users.
JDH

As a followup to the work on floats, I fixed rec2csv to deal with funky 
strings (strings with commas and quotes). I've checked this is as r4749, 
but I thought I'd announce here in case someone (John, in particular) 
was depending on some peculiar aspect of the old implementation. If 
there is something you depend on, can you update the unittests in 
unit/mlab_unit.py to check for the behavior you need?
Also, John, I didn't see mlab.FormatString being used anywhere, but I 
didn't want to change it, either. So I made FormatString2 and used it. 
But if there's no known use of FormatString, lets kill the original and 
more FormatString2 into its place.
-Andrew

On Wednesday 28 November 2007 1:12:00 pm John Hunter wrote:
> On Nov 28, 2007 11:46 AM, Eric Firing <ef...@ha...> wrote:
> > I assume there is also some sort of C and/or C++ mode, and similar hook
> > additions are needed for them--correct? Or is there a hierarchy of
> > modes, in which case a single hook could go at a higher level?
>
> I don't know about a hierarchy, but there is a c++-mode-hook and a
> c-mode-hook. I've updated the CODING_GUIDE to advise emacs users to
> set these hooks. Soon your nightmare of trailing whitespaces will be
> nothing more than a nasty memory ....
For those who use the eric4 IDE, the version 4.1 snapshot that was released 
today includes support for stripping whitespace when saving, the same as the 
emacs hook does.
Darren

Done in r4748. I added the kwarg "return_opened" to cbook.to_filehandle().
John Hunter wrote:
> On Dec 16, 2007 1:33 PM, Andrew Straw <str...@as...> wrote:
>> OK, I added unit/mlab_unit.py to svn. This checks that double precision
>> floats can round-trip through rec2csv and csv2rec. To pass requires not
>> only svn matplotlib (due to John's change), but also svn numpy (to be
>> included with 1.0.5).
>>
>> Also, it doesn't seem to me that rec2csv should close a file handle
>> passed to it. This prevents use with StringIO, for example. So, I added
>> a test for that, too. John, if it's not going to break anything for you,
>> I'll go ahead and fix that.
> 
> Yes, please do. You might want to modify cbook.to_filehandle to
> optionally return a tuple specifying whether the returned fh is owned
> by the caller. Something like:
> 
> fh, owner = cbook.to_filehandle(fname, returnowner=True)
> 
> and then later
> 
> if owner:
> fh.close()
> 
> JDH
> 
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

On Thursday 13 December 2007 2:22:13 pm Robert Kern wrote:
> John Hunter wrote:
> > Do we need namespace packages in toolkits? I recently added gtktools
> > and exceltools to toolkits, and got a very hard to debug error:
> >
> > In [1]: import matplotlib
> >
> > In [2]: matplotlib.__file__
> > Out[2]:
> > '/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/__init__.py
> >c'
> >
> > In [3]: matplotlib.rcParams['axes.axisbelow']
> > Out[3]: False
> >
> > In [4]: import matplotlib.toolkits.gtktools
> >
> > In [5]: matplotlib.__file__
> > Out[5]:
> > '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/__ini
> >t__.pyc'
> >
> > In [6]: matplotlib.rcParams['axes.axisbelow']
> > ------------------------------------------------------------
> > Traceback (most recent call last):
> > File "<ipython console>", line 1, in ?
> > KeyError: 'axes.axisbelow'
> >
> > Notice that the matplotlib module which was previously imported got
> > changed by the import of the toolkit which declared itself a namespace
> > package. Now this may be the result of mpl not using namespace
> > packages correctly (eg matplotlib/__init__.py should be empty save for
> > the namespace package or something like that)
>
> Probably. In setuptools 0.7, they can have stuff in them, but all of them
> must be identical.
How would one learn about this new feature? According to setuptools.txt from 
the 0.7 svn checkout, the namespace package's __init__.py must still be empty 
save the namespace declaration.
> If you are using an unmodified matplotlib/__init__.py in 
> the main matplotlib package, then the "if 0:" is preventing the declaration
> of the namespace package, and that may be contributing.
>
> Also, neither matplotlib nor matplotlib.toolkits is named as a
> namespace_package in matplotlib's setup.py, and both have to be. Similarly,
> basemap (for example) only lists matplotlib.toolkits but not matplotlib.
If I understand correctly, even after fixing these two problems, namespace 
packages would not work since matplotlib/__init__.py contains lots of 
meaningful code. Based on the remarks at 
http://mail.python.org/pipermail/distutils-sig/2007-October/008346.html and 
the setuptools svn commit log, it looks like development on version 0.7 is 
not a priority and a release in the next year is unlikely. So is it fair to 
say that for the foreseeable future, matplotlib is not compatible with 
namespace packages unless we cleaned out __init__.py? 
Darren

On Dec 16, 2007 1:33 PM, Andrew Straw <str...@as...> wrote:
> OK, I added unit/mlab_unit.py to svn. This checks that double precision
> floats can round-trip through rec2csv and csv2rec. To pass requires not
> only svn matplotlib (due to John's change), but also svn numpy (to be
> included with 1.0.5).
>
> Also, it doesn't seem to me that rec2csv should close a file handle
> passed to it. This prevents use with StringIO, for example. So, I added
> a test for that, too. John, if it's not going to break anything for you,
> I'll go ahead and fix that.
Yes, please do. You might want to modify cbook.to_filehandle to
optionally return a tuple specifying whether the returned fh is owned
by the caller. Something like:
 fh, owner = cbook.to_filehandle(fname, returnowner=True)
and then later
 if owner:
 fh.close()
JDH

OK, I added unit/mlab_unit.py to svn. This checks that double precision 
floats can round-trip through rec2csv and csv2rec. To pass requires not 
only svn matplotlib (due to John's change), but also svn numpy (to be 
included with 1.0.5).
Also, it doesn't seem to me that rec2csv should close a file handle 
passed to it. This prevents use with StringIO, for example. So, I added 
a test for that, too. John, if it's not going to break anything for you, 
I'll go ahead and fix that.
John Hunter wrote:
> On Dec 15, 2007 3:13 PM, Andrew Straw <str...@as...> wrote:
> 
>> mlab.defaultformatd sets the default float formatter for rec2csv() to be
>> something that doesn't keep the full representation of a floating point
>> number. Obviously, I can pass in my own formatd argument to rec2csv(),
>> but I wonder if there's any reason why defaultformatd shouldn't use
>> repr() for numpy.float32 and numpy.float64?
>> 
>
> I'll be happy to make repr the default. I was using %g because I
> mistakenly though this provided the appropriate number of significant
> digits. I changed this to %r in the csvformat_factory in svn.
>
> JDJ
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>