matplotlib-devel Mailing List for matplotlib

On Thursday 19 July 2007 7:31:11 pm John Hunter wrote:
> On 7/19/07, Christopher Barker <Chr...@no...> wrote:
> > How many back-ends does the future hold? It seems if the GUI toolkits
> > all use *Agg, then that's only one render for all of them. Then we need:
> >
> > SVG
> > PDF
> > PS
> > Cairo would be nice, as it gives us almost all of them at once, but I
> > guess licensing keeps that a non-starter. Oh well.
>
> Not at all, we want to fully support Cairo. We just want to have some
> fully BSD compliant backends as well.
Is there much demand for BSD-compliant svg, pdf, and ps backends?

On Thursday 19 July 2007 6:31:26 pm Christopher Barker wrote:
> > There is also the question of whether
> > we want to pay up and use 4x4 from the ground up and just ignore the
> > 3rd dimension to open the door for 3D support.
>
> I say yes! 3-d really is a very often needed and requested feature.
> Sure, we can go to VTK or something for really sophisticated 3-d work,
> but being able to do the basic stuff with MPL would be wonderful.
>
> If the framework supports it cleanly internally, it's much more likely
> that the 3-d stuff will get written.
I also think we should use 4x4 affines, and ignore the third dimension. 10 
years down the line, we might look back and regret not taking advantage of 
this opportunity.

On 7/19/07, Christopher Barker <Chr...@no...> wrote:
> > This is potentially a major win, because we currently
> > move the data around on every draw.
>
> Is it that expensive to push data around? In any case, it does sound
> cleaner and more efficient not to.
It can be very expensive. Imagine you are smoothly panning or zooming
a line object with 100,000 x,y points. All you are really doing is
changing the affine. Although we've done some things to help this
case, in matplotlib we still have to create a new path object every
time in the agg backend, and then transform it. It's much cheaper
just to push the affine to the backend in this case. So interaction
with large data sets should get better.
> > Do we want to use 3x3 or 4x4 to leave the door open for 3D developers?
>
> 4X4 -- is there much cost?
The potential cost is not in the 3x3 vs 4x4, but in the extra row of
junk data you would store in the data matrix, which is N extra values
for plotting N points . The matrix multiplication would be 3x3 * 3xN
vs 4x4 * 4xN , so there would be a cost in memory and performance.
> > This approach requires the backends to be smarter, but they have to
> > handle fewer entities.
>
> How many back-ends does the future hold? It seems if the GUI toolkits
> all use *Agg, then that's only one render for all of them. Then we need:
>
> SVG
> PDF
> PS
> Cairo would be nice, as it gives us almost all of them at once, but I
> guess licensing keeps that a non-starter. Oh well.
Not at all, we want to fully support Cairo. We just want to have some
fully BSD compliant backends as well. agg 2.4 will remain BSD and I
don't have too much of a problem relying on it. We are not alone in
needing a BSD agg. I think the 4 you mentioned plus *Agg are the
ones we should target. The goal is to get all the GUIs to work with a
python buffer object or a numpy pixel buffer array -- if Agg and Cairo
can provide the same buffer or numpy format, then we would
automagically get *Agg and *Cairo across the GUIs.
JDH

Lots of god stuff John!
> There is also the question of whether
> we want to pay up and use 4x4 from the ground up and just ignore the
> 3rd dimension to open the door for 3D support. 
I say yes! 3-d really is a very often needed and requested feature. 
Sure, we can go to VTK or something for really sophisticated 3-d work, 
but being able to do the basic stuff with MPL would be wonderful.
If the framework supports it cleanly internally, it's much more likely 
that the 3-d stuff will get written.
> This is potentially a major win, because we currently
> move the data around on every draw.
Is it that expensive to push data around? In any case, it does sound 
cleaner and more efficient not to.
> Do we want to use 3x3 or 4x4 to leave the door open for 3D developers?
4X4 -- is there much cost?
> This approach requires the backends to be smarter, but they have to
> handle fewer entities.
How many back-ends does the future hold? It seems if the GUI toolkits 
all use *Agg, then that's only one render for all of them. Then we need:
SVG
PDF
PS
???
Cairo would be nice, as it gives us almost all of them at once, but I 
guess licensing keeps that a non-starter. Oh well.
> In matplotlib, the plot functions are matplotlib.axes.Axes methods and
> I think there is consensus that this is a poor design.
Well, the OO interface has always felt a bit clunky to me, but I'm not 
sure where else plot functions could go -- I'd love to hear ideas, though.
> Do we want to create high level objects like Circle, Rectangle and
> Line, each of which manage a Path object under the hood?
I like that idea -- working with Paths should be saved for the gurus.
> Just having the right Path object
> will reduce the need for many of these, eg LineCollection,
> PolygonCollection, etc... 
sounds good.
> Also, everything should be numpy enabled,
> and the sequence-of-python-tuples approach that many of the
> collections take should be dropped.
who hoo!
However, numpy doesn't handle "ragged" arrays well. I wonder if there's 
a good way to implement those, so that transforms can be done 
numpy-efficient.
> = Extension code =
> 
> If we can enhance the
> SWIG agg wrapper, we can also do images through there, getting rid of
> _image.cpp. Having a fully featured, python-exposed agg wrapper will
> be a plus in mpl and beyond.
Very nice.
> But with the agg license change, I'm
> open to discussion of other approaches.
hmm GPL now. Well, maybe Cairo's LGPL isn't so bad after all!
> I want to do away with *all* GUI extension code.
yeah!
> = Traits =
> I think we should make a major committment to traits and use them from
> the ground up.
Good plan.
> = Breakage =
> 
> I think we need to be prepared to break the hell out of matplotlib.
> The API will basically be a significant rewrite. 
Well worth it.
 > pylab will still
> mostly work unchanged -- that is the beauty of pylab 
As a rule for the future though, a stable OO interface would be nice.
> Or we could forget all this wild speculation and resume our normally
> scheduled lives.
no!!
> = Chaco and Kiva =
> 
> It is a good idea for an enterprising developer to take a careful look
> at the current Chaco and Kiva
OK. I have to ask -- why aren't we all just using Chaco? I know I'm not 
because ??years ago, Enthought was not really supporting anything but 
Windows -- is that still true? Would it be a whole lot less work to 
support GTK, OS-X, ??? in Chaco than keep developing a separate lib?
Great conversation starters!
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

On Thursday 19 July 2007 04:05:11 pm ah...@cs... wrote:
> Somehow I accidentally deleted a line in a part I thought I hadn't touched.
>
> It's a two line change, so I'll just tell you what to change:
>
> Find the line:
> set_clipbox_rasterizer(gc.cliprect);
>
> in src/_backend_agg.cpp in the draw_lines function. (around line 1500)
>
> Right after it, add the following two lines:
> //path_t transpath(path, xytrans);
> _process_alpha_mask(gc);
It's done, svn 3579. Thank you Allan.
Darren

On Jul 19, 2007, at 3:05 PM, Bill Baxter wrote:
> Chaco may be formidable and complex, but so is the list of features
> and requirements you just posted. What about just focusing on a Pylab
> wrapper for Chaco? And working with Peter to make Chaco everything
> you envison. Or does Chaco have the same needs-a-rewrite architecture
> issues as the mpl?
There are certainly directions I'd like to take the architecture, but 
I'm not planning a rewrite anytime soon. One rewrite every 4 years 
is more than enough for me. ;)
> Just to be clear, I don't have any first hand experience with Chaco,
> other than running the demos once. The main problems with Chaco I'm
> aware of are 1) entanglement with the rest of ETS, which they're
> working on, 2) no pylab like easy-to-use interface.
(1): Other than traits (and a teensy bit of traits UI), Chaco 
requires only Kiva and Enable. Its setup.py reflects this. This has 
been the case for a while, but historically the issue has been that 
all the interdependencies at the traits UI level sucked in basically 
the rest of ETS.
(2): Chaco2.shell has some rudimentary pylab-like features, but 
obviously is nowhere near complete. There are some examples of the 
sorts of things it can do: https://svn.enthought.com/enthought/ 
browser/branches/enthought.chaco2_2.0/examples/shell. One thing to 
note about the shell is that its commands are just convenience 
functions that wrap existing Chaco containers and components, so the 
structure of the live plot that is built with, say, an imshow() 
command is similar to one that you could build by hand. This means 
that you can dynamically extend its behavior by adding new tools that 
the command-line interface doesn't know about. It also means that 
you can use the command-line interface to construct a plot (or grid 
of plots) and trivially embed that into an external application, no 
differently than if you had hand-coded to the lower, object-oriented 
layer.
-Peter

On 7/20/07, John Hunter <jd...@gm...> wrote:
> = Chaco and Kiva =
>
> It is a good idea for an enterprising developer to take a careful look
> at the current Chaco and Kiva to see if we can further integrate with
> them. I am gun shy because they seem formiddable and complex, and one
> of my major goals here is to streamline and simplify, but they are
> incredible pieces of work and we need to carefully consider them,
> especially as we integrate other parts of the enthought suite into our
> core, eg traits, increasing the possibility of synergies.
Chaco may be formidable and complex, but so is the list of features
and requirements you just posted. What about just focusing on a Pylab
wrapper for Chaco? And working with Peter to make Chaco everything
you envison. Or does Chaco have the same needs-a-rewrite architecture
issues as the mpl?
Just to be clear, I don't have any first hand experience with Chaco,
other than running the demos once. The main problems with Chaco I'm
aware of are 1) entanglement with the rest of ETS, which they're
working on, 2) no pylab like easy-to-use interface.
--bb

Somehow I accidentally deleted a line in a part I thought I hadn't touched.
It's a two line change, so I'll just tell you what to change:
Find the line:
set_clipbox_rasterizer(gc.cliprect);
in src/_backend_agg.cpp in the draw_lines function. (around line 1500)
Right after it, add the following two lines:
//path_t transpath(path, xytrans);
_process_alpha_mask(gc);
I don't know how that happened....
Allan
On Thu, July 19, 2007 3:33 pm, ah...@cs... wrote:
> That seems to have to do with the line culling agg patch I sent. I never
> thought to check with polar plots. I'll look into it.
>
> Allan
>
>
> On Thu, July 19, 2007 12:17 pm, Paul Kienzle wrote:
>
>> The polar demo in examples/polar_demo.py no longer displays the spiral
>> and axes. It worked a couple of weeks ago when I was testing the
>> contains() method.
>>
>> I downloaded a fresh build of matplotlib pulled from svn today. Tested
>> on python 2.5 OS X. Should be on the wxAgg backend, though I don't
>> know how to confirm that.
>>
>> - Paul
>>

On Jul 19, 2007, at 12:28 PM, Fernando Perez wrote:
> Is Peter Wang on this list? If not, perhaps you should CC him and tip
> him to come over. I know Robert monitors this, but we shouldn't make
> him the single point of responsibility for keeping tabs on the bridges
> with Chaco/ETS.
Actually I am subscribed to the list, but thanks to Robert for 
pointing out this thread to me. :)
John, much of what you have written is very interesting, and I will 
have a more detailed response later. I just want to say real quick, 
though, that I have been trying to work out Chaco's next 
architectural steps, and there is definitely some overlap with what 
you've outlined, but coming from a different direction.
-Peter

That seems to have to do with the line culling agg patch I sent. I never
thought to check with polar plots. I'll look into it.
Allan
On Thu, July 19, 2007 12:17 pm, Paul Kienzle wrote:
> The polar demo in examples/polar_demo.py no longer displays the spiral
> and axes. It worked a couple of weeks ago when I was testing the
> contains() method.
>
> I downloaded a fresh build of matplotlib pulled from svn today. Tested
> on python 2.5 OS X. Should be on the wxAgg backend, though I don't know
> how to confirm that.
>
> - Paul

Probably a better question for the help list, but has anybody written
an artist that can display a semi-infinite or infinite line?
axvline and axhline can fake it for vertical and horizontal infinite lines,
but they cannot handle slopes or semi-infinite lines.
Thanks,
	- Paul

John Hunter wrote:
> On 7/19/07, Eric Firing <ef...@ha...> wrote:
> 
>> The instructions still say to check out traits 2.0, but Robert is
>> recommending that we go with traits 3. Do you really want to stick with
>> version 2 now?
> 
> No, I'm happy to move over. But I spent way more time getting traits
> working and installed than I wanted to, and I wanted to spend most of
> my time coding the sketch, so once I had it working I did not want to
> break it. If someone wants to take the lead getting a working traits3
> install with instructions and then migrate mpl1 (probably not much to
> do there) I'm happy to switch over. I think Robert was recommending
> the first release of Traits3 for us, which hasn't happened yet. But
> if the svn version is working and installable, I'm happy to make the
> switch now if advised.
> 
> JDH
John,
I thought initially that a simple svn checkout of 3 was working with a 
slight tweak (editing api.py), but it looks like there are still some 
dependencies that don't show up immediately but that do show up when 
trying to run mpl1; it is again the ui code trying to pull in things 
from outside traits. So I don't have an immediate solution. It looks 
like the effort to make traits independently installable still has a 
ways to go.
Eric

On 7/19/07, Eric Firing <ef...@ha...> wrote:
> The instructions still say to check out traits 2.0, but Robert is
> recommending that we go with traits 3. Do you really want to stick with
> version 2 now?
No, I'm happy to move over. But I spent way more time getting traits
working and installed than I wanted to, and I wanted to spend most of
my time coding the sketch, so once I had it working I did not want to
break it. If someone wants to take the lead getting a working traits3
install with instructions and then migrate mpl1 (probably not much to
do there) I'm happy to switch over. I think Robert was recommending
the first release of Traits3 for us, which hasn't happened yet. But
if the svn version is working and installable, I'm happy to make the
switch now if advised.
JDH

Darren Dale wrote:
> On Thursday 19 July 2007 02:26:05 pm John Hunter wrote:
>> On 7/19/07, Darren Dale <dd...@co...> wrote
>>
>>> On Thursday 19 July 2007 01:18:21 pm John Hunter wrote:
>>>
>>> I have not been able to install traits by following the instructions in
>>> mtraits.py. 
> [...]
>> I encountered a similar problem at home last night, and Dave
>> recommended on the enthought list. I haven't had a chance to test
>> this yet. If this works, please update the install instructions in
>> mtraits if you get a minute.
>>
>> Here is Dave's answer::
> [...]
>> sudo easy_install -f
>> http://code.enthought.com/enstaller/eggs/source/unstable/ \
>> "enthought.etsconfig < 3.0a" "enthought.util <3.0a" "enthought.debug
>> <3.0a"
> 
> That worked. The instructions in mtraits have been updated.
The instructions still say to check out traits 2.0, but Robert is 
recommending that we go with traits 3. Do you really want to stick with 
version 2 now?
Eric

Darren Dale wrote:
[...]
>> The point is that although users will have to *have* numpy, they will
>> not yet have to convert all their other packages to numpy; if they have
>> extension packages built on numarray, for example, and accessed via code
>> using matplotlib.numerix, everything will still work.
> 
> Thanks for the clarification. Has there been any discussion about deprecating 
> numerix at some point in the future? 
I don't think so, apart from what I stated above, which was discussed 
quite some time ago. I think there is really no strong motivation to 
completely remove numerix from the present generation of matplotlib, 
although it does seem like it would be nice to nudge users toward pure 
and current numpy usage.
I imagine that mpl1, however, might never have a numerix at all; and 
maybe by the time mpl1 is ready for use, maskedarray will have replaced 
ma in numpy, so we won't need anything like numerix.npyma either.
Eric

On Thursday 19 July 2007 02:26:05 pm John Hunter wrote:
> On 7/19/07, Darren Dale <dd...@co...> wrote
>
> > On Thursday 19 July 2007 01:18:21 pm John Hunter wrote:
> >
> > I have not been able to install traits by following the instructions in
> > mtraits.py. 
[...]
> I encountered a similar problem at home last night, and Dave
> recommended on the enthought list. I haven't had a chance to test
> this yet. If this works, please update the install instructions in
> mtraits if you get a minute.
>
> Here is Dave's answer::
[...]
> sudo easy_install -f
> http://code.enthought.com/enstaller/eggs/source/unstable/ \
> "enthought.etsconfig < 3.0a" "enthought.util <3.0a" "enthought.debug
> <3.0a"
That worked. The instructions in mtraits have been updated.

Darren Dale wrote:
> Hi Eric,
> 
> On Thursday 19 July 2007 02:10:03 pm Eric Firing wrote:
>> unless John or someone else
>> contradicts me I request that you restore the original numerix, or
>> something like it, so that users' external code can still use numerix to
>> deal with Numeric and/or numarray code and arrays.
> 
> I thought we were dropping support for numeric and numarray in numerix, 
> starting with 0.91?
Not quite. I think the idea was to use numpy internally, but to let the 
numerix layer stay as it is so that users' code built on numerix can 
still work with other array packages. When that code calls mpl, 
everything should still work; the other arrays will internally be 
converted to numpy.
The point is that although users will have to *have* numpy, they will 
not yet have to convert all their other packages to numpy; if they have 
extension packages built on numarray, for example, and accessed via code 
using matplotlib.numerix, everything will still work.
Eric

On Thu, Jul 19, 2007 at 01:26:05PM -0500, John Hunter wrote:
> On 7/19/07, Darren Dale <dd...@co...> wrote
> > On Thursday 19 July 2007 01:18:21 pm John Hunter wrote:
> > I have not been able to install traits by following the instructions
> > in
> > mtraits.py. easy_install is pulling in enthought.util-3.0a1, which
> > conflicts
> > with enthought.resource-2.0b1. Why do you pull etsconfig, util and
> > debug
from
> > one place, and traits 2 from another? I would have thought it easier
> > to do:
> I think the answer is because the install is broken and you have to
> get some combination of packages that work together through a little
> hackery. They're working on it ...
You replied faster than I could grep my mailbox :->.
The problem is that the traits 2. egg that you want to install depends on
ets2 components, but the dependance has not been well coded in the
package (ets2 is in general api incompatible with ets3, and the
dependance should specify a version number below 3.a) and some ets3
components get pulled in. So its a packaging bug that will be addressed.
In the mean time John solution is the good answer.
Gaël
PS: sorry for the dup, John, I "miss-mutted"

Hi Eric,
On Thursday 19 July 2007 02:10:03 pm Eric Firing wrote:
> unless John or someone else
> contradicts me I request that you restore the original numerix, or
> something like it, so that users' external code can still use numerix to
> deal with Numeric and/or numarray code and arrays.
I thought we were dropping support for numeric and numarray in numerix, 
starting with 0.91?

On 7/19/07, Darren Dale <dd...@co...> wrote
> On Thursday 19 July 2007 01:18:21 pm John Hunter wrote:
> I have not been able to install traits by following the instructions in
> mtraits.py. easy_install is pulling in enthought.util-3.0a1, which conflicts
> with enthought.resource-2.0b1. Why do you pull etsconfig, util and debug from
> one place, and traits 2 from another? I would have thought it easier to do:
I think the answer is because the install is broken and you have to
get some combination of packages that work together through a little
hackery. They're working on it ...
I encountered a similar problem at home last night, and Dave
recommended on the enthought list. I haven't had a chance to test
this yet. If this works, please update the install instructions in
mtraits if you get a minute.
Here is Dave's answer::
The problem is that this command, without any versions specified, is
mixing versions from the ETS 2.5 release and the still nascent ETS
3.0. And, as your seeing, they don't mix because most of the 2.x stuff
declares that it doesn't work with anything later than a 2.x version --
i.e. enthought.resource 2.0b2 requires an enthought.util version less
than 3.0a but you've already installed enthought.util version 3.0a1.
Easy_install is just doing its job here by giving you an error.
Clearly we need to get the uninstallable ETS 3.0 components out of the
various repos -- at least until they install smoothly. (See below
discussion.) In the meantime, to resolve this for yourself, back out
any 3.0a enthought components, and then do the following command:
 sudo easy_install -f
http://code.enthought.com/enstaller/eggs/source/unstable/ \
 "enthought.etsconfig < 3.0a" "enthought.util <3.0a" "enthought.debug
<3.0a"
For the rest of the world, I think we have to pull out the ETS v3.x
components from the repo until we get them to where they can be
installed. Bryce, you'll have to get them from somewhere else for your
project -- perhaps the customer-specific repo. Let's talk tomorrow and
then I can get them out of the repo and people can stop running into
this problem.
-- Dave

Hi John,
On Thursday 19 July 2007 01:18:21 pm John Hunter wrote:
> I've been working on a laboratory in which we can fruitfully discuss,
> test, implement mpl1 design issues.
[...]
> You will need the latest svn matplotlib and 
> the latest svn enthought traits 2 -- see the header of mpl1/mtraits.py
> for install instructions for the latter.
I have not been able to install traits by following the instructions in 
mtraits.py. easy_install is pulling in enthought.util-3.0a1, which conflicts 
with enthought.resource-2.0b1. Why do you pull etsconfig, util and debug from 
one place, and traits 2 from another? I would have thought it easier to do:
easy_install -f http://code.enthought.com/enstaller/eggs/source/unstable/ 
enthought.traits-2.0b2.dev-r12847
but that doesnt work either, it doesnt download any of traits dependencies:
Searching for enthought.traits-2.0b2.dev-r12847
Reading http://code.enthought.com/enstaller/eggs/source/unstable/
Best match: enthought.traits-2.0b2.dev-r12847 [unknown version]
Downloading 
http://code.enthought.com/enstaller/eggs/source/unstable/enthought.traits-2.0b2.dev-r12847.zip
Processing enthought.traits-2.0b2.dev-r12847.zip
Running enthought.traits-2.0b2.dev-r12847/setup.py -q 
bdist_egg --dist-dir /tmp/easy_install-iDf8BC/enthought.traits-2.0b2.dev-r12847/egg-dist-tmp-oozCl0
install_requires:
 enthought.etsconfig >=2.0b1.dev, <3.a
 enthought.util >=2.0b1.dev, <3.a
test_requires:
 nose >= 0.9,
ui_requires:
 enthought.pyface >=2.0b1.dev, <3.a
 enthought.resource >=2.0b1.dev, <3.a
wx_requires:
 enthought.traits.ui.wx >=2.0b1.dev, <3.a
Adding enthought.traits 2.0b2.dev-r12847 to easy-install.pth file
Installed /usr/lib64/python2.5/site-packages/enthought.traits-2.0b2.dev_r12847-py2.5-linux-x86_64.egg
Skipping dependencies for enthought.traits 2.0b2.dev-r12847

Norbert,
Cancel my last message. I panicked. It looks like the problem with my 
attempt to run backend_driver.py was that your changes required deletion 
of the build directory and/or previous matplotlib installation; after 
doing that, backend_driver.py runs.
The comment about numerix is still valid, so unless John or someone else 
contradicts me I request that you restore the original numerix, or 
something like it, so that users' external code can still use numerix to 
deal with Numeric and/or numarray code and arrays. This is not urgent, 
but should be done within a day or two to prevent confusion and surprises.
Thanks, and I'm sorry for my excessive and erroneous earlier reaction.
Eric
Eric Firing wrote:
> Norbert,
> 
> In addition to the problem that numerix was supposed to retain support 
> for Numeric and numarray for the time being, for external use, there is 
> the problem that all examples are currently broken.
> 
> Please let me know if you are still online (I know it is very late in 
> Germany); otherwise I may have to revert your changes until the problems 
> are fixed.
> 
> Thanks.
> 
> Eric
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Norbert,
In addition to the problem that numerix was supposed to retain support 
for Numeric and numarray for the time being, for external use, there is 
the problem that all examples are currently broken.
Please let me know if you are still online (I know it is very late in 
Germany); otherwise I may have to revert your changes until the problems 
are fixed.
Thanks.
Eric

On 7/19/07, John Hunter <jd...@gm...> wrote:
> = Chaco and Kiva =
>
> It is a good idea for an enterprising developer to take a careful look
> at the current Chaco and Kiva to see if we can further integrate with
> them. I am gun shy because they seem formiddable and complex, and one
> of my major goals here is to streamline and simplify, but they are
> incredible pieces of work and we need to carefully consider them,
> especially as we integrate other parts of the enthought suite into our
> core, eg traits, increasing the possibility of synergies.
Is Peter Wang on this list? If not, perhaps you should CC him and tip
him to come over. I know Robert monitors this, but we shouldn't make
him the single point of responsibility for keeping tabs on the bridges
with Chaco/ETS.
Just a minor logistical comment. Otherwise, go John!!!
In related news, I'll be posting the traits/configuration work I've
been playing with soon. I'm starting to like the ConfigObj/Traits
combo a LOT. Stay tuned.
Cheers,
f

I've been working on a laboratory in which we can fruitfully discuss,
test, implement mpl1 design issues. I am a big fan of
python-as-modeling-language approach to design. I have tried to solve
from the ground up some of the design flaws in matplotlib -- the
transformation architecture and the data model, in which transformed
data is pushed to the backend with every draw. The goal was to get a
single file of pure python so people can get their heads around the
code in one place, and experiment w/o having to go through a
compile/install cycle. You will need the latest svn matplotlib and
the latest svn enthought traits 2 -- see the header of mpl1/mtraits.py
for install instructions for the latter.
The sketch is in mpl1/mpl1.py in matplotlib svn, and it does produce a
graph (see attached). Right now only path drawing is implemented. It
is now time to think about how to handle the Axis. We want to figure
out the right way to bundle and xaxis and a yaxis with an artist so
that we can support multiple y-axis etc on one Axes. Drawing axis
ticks also brings up another problem I have not figured out -- how to
draw markers in points at data locations in the figure. matplotlib
uses some trickery in the transforms (transoffset and friends)
designed to handle this. An alternative that I am considering is
making a first class primitive called Markers, which have a list of
x,y locations, a marker path, an affine and some path properties. The
renderer can then cache the path and then draw markers in points in
the right place. I am open to other ideas, but this is my current
thinking.
Most of the effort here has been trying to get the transformations
right, so please give me feedback and or make corrections and
suggestions -- I'm not wild about the naming either, so feel free to
come up with something better. There is also the question of whether
we want to pay up and use 4x4 from the ground up and just ignore the
3rd dimension to open the door for 3D support. My inclination is
probably not, but I am open to ideas.
Included below is the "DESIGN_GOALS" document, also in mpl1 svn::
Here are some of the things I would like to accomplish with mpl1. Any
and all of this is open to discussion. What I present below is pretty
ambitious, so if there is support, we will need significant
contributions from several developers for several months. Ideally, we
would get a good sketch working, and then organize a spint (3-4 days?)
for late August, where we try get as far as possible to making this
viable.
= Data copying =
Push the data to the backend only once, or only when required. Update
the transforms in the backend, but do not push transformed data on
every draw. This is potentially a major win, because we currently
move the data around on every draw. Eg, see how mpl1.py handles pusing
the paths when the renderer is set (Figure.set_renderer) but on draw
commands (Figure.draw) only pushes the current affine.
= Transformations =
Support a normal transformation architecture. The current draft
implementation assumes one nonlinear transformation, which happens at
a high layer, and all transformations after that are affines. In the
mpl1 draft, there are three affines: the transformation from view
limits -> axes units (AxesCoords.affineview), the transformation from
axes units to normalized figure units (AxesCoords.affineaxes), and the
transformation from normalized figure units to display
(Renderer.affinerenderer)
Do we want to use 3x3 or 4x4 to leave the door open for 3D developers?
How do transformations (linear and nonlinear) play with Axis features
(ticking and gridding). The ideal is a framework in which ticking,
gridding and labeling work intelligently with arbitrary, user
supplied, transformations. What is the proper transformation API?
= Objects that talk to the backend "primitives" =
Have just a few, fairly rich obects, that the backends need to
understand. Clear candidates are a Path, Text and Image, but despite
their names, don't confuse these with the eponymous matplotlib
matplotlib Artists, which are higher level than what I'm thinking of
here (eg matplotlib.text.Text does *a lot* of layout, and this would
be offloaded ot the backend in this conception of the Text primitive).
Each of these will carry their metadata, eg a path will carry its
stroke color, facecolor, linewidth, etc..., and Text will carry its
font size, color, etc.... We may need some optimizations down the
road, but we should start small. For now, let's call these objects
"primitives".
This approach requires the backends to be smarter, but they have to
handle fewer entities.
= Where do the plot functions live? =
In matplotlib, the plot functions are matplotlib.axes.Axes methods and
I think there is consensus that this is a poor design. Where should
these live, what should they create, etc?
= How much of an intermediate artist layer do we need? =
Do we want to create high level objects like Circle, Rectangle and
Line, each of which manage a Path object under the hood? Probably,
for user convenience and general compability with matplotlib. By
using traits properly here, many current matplotlib Arists will be
thin interfaces around one or more primitives.
I think the whole matplotlib.collections module is poorly designed,
and should be chucked wholesale, in favor of faster, more elegant,
optimizations and special cases. Just having the right Path object
will reduce the need for many of these, eg LineCollection,
PolygonCollection, etc... Also, everything should be numpy enabled,
and the sequence-of-python-tuples approach that many of the
collections take should be dropped. Obviously some of the more useful
things there, like quad meshes, need to be ported and retained.
= Z-ordering, containers, etc =
Peter has been doing a lot of nice work on z-order and layers for
chaco, stuff that looks really useful for picking, interaction, etc...
We should look at this approach, and think carefully about how this
should be handled. Paul may be a good candidate for this, since he
has been working recently on the picking API.
= Extension code =
I would like to shed all of the CXX extension code -- it is just too
small a nitch in the python world to base our project on. SWIG is
pretty clearly the right choice. mpl1 will use numpy for
transformations with some carefully chosen extension code where
necessary, to get rid of _transforms.cpp. I also plan to use the SWIG
agg wrapper, so this gets rid of _backend_agg. If we can enhance the
SWIG agg wrapper, we can also do images through there, getting rid of
_image.cpp. Having a fully featured, python-exposed agg wrapper will
be a plus in mpl and beyond. But with the agg license change, I'm
open to discussion of other approaches.
The major missing piece in ft2font, which is a pretty elaborate CXX
module. Michael may want to consider alternatives, including looking
at the agg support for freetype, and the kiva/chaco approach.
I want to do away with *all* GUI extension code. This should live
outside MPL if at all, eg in a toolkit if we need it. This means
someone needs to figure out how to get TkInter talking to a python
buffer object or a numpy array. Maintaining the GUI extension code
across platforms is an unending headache.
= Traits =
I think we should make a major committment to traits and use them from
the ground up. Even without the UI stuff, they add plenty to make
them worthwhile, especially the validation and notification features.
With the UI (wx only) , they are a major win for many GUI developers.
Compare the logic for sharing an x-axis using matplotlib transforms
with Axes.sharex with the approach used in mpl1.py with sync_trait-ed
affines.
= Axis handling =
The whole concept of the Axes object needs to be rethought, in light
of the fact that we need to support multiple axis objects on one Axes.
The matplotlib implementation assumes 1 xaxis and 1 yaxis per Axes,
and we hack two y-axis support (examples/two_scales.py) with some
transform shenanigans via twinx and multiple Axes where one is hidden,
but the approach is not scalable and is unwieldy.
This will require a fair amount of thought, but we should aim for
supporting an arbitrary number of axis obects, presumably associated
with individual artists or primitives. They also need to be *much*
faster. matplotlib uses Artists for each tick, tickline, gridline,
ticklabel, etc, and this is mind-numbingly slow. I have a prototype
axis implementations that draws the ticks with a single path using
repeated MOVETO and LINETO, for example, which will be incomparably
faster than using a separate object for each tick.
The other important featiure for axis support is that, for the most
part, they should be arbitrarily placeable (eg a "detached" axis).
= Breakage =
I think we need to be prepared to break the hell out of matplotlib.
The API will basically be a significant rewrite. pylab will still
mostly work unchanged -- that is the beauty of pylab -- though API
calls on return objects may be badly broken. We can mitigate this pain
if we desire with clever wrapper objects, but once you start calling
methods on return objects, you join the community of power users, and
this is the community I'm most willing to inconvenience with breakage.
We'll probably want to install into a new namespace, eg "mpl", and
envision both matplotlib and mpl co-existing for some time. In fact,
mpl might depend on matplotlib initially, eg until a CXX-free ft2font
is available.
We should expect to be supporting and using matplotlib for a long
time, since the proposals discussed here imply that it will be a long
wait until mpl1 is feature complete with matplotlib. In fact, we could
rightly consider this to be the mpl2 proposal, and keep releasing
matplotlib ehancements to 1.0 and beyond w/o signfificant breakage.
It's a nominal difference so I don't really have a preference.
Or we could forget all this wild speculation and resume our normally
scheduled lives.
= Chaco and Kiva =
It is a good idea for an enterprising developer to take a careful look
at the current Chaco and Kiva to see if we can further integrate with
them. I am gun shy because they seem formiddable and complex, and one
of my major goals here is to streamline and simplify, but they are
incredible pieces of work and we need to carefully consider them,
especially as we integrate other parts of the enthought suite into our
core, eg traits, increasing the possibility of synergies.
= Unit handling, custom object types =
There is a legitimate need to be able to feed custom objects into
matplotlib. Recent versions of matplotlib support this with a unit
registry in the "units" module. A clear use case is plotting with
native python datetime objects, which is supported in 0.90 via the
unit handling, which should probably be called "custom object handling
and conversion". This is a deep and complicated subject, involving
questions of where the original data live, how they are converted to
useful types (arrays of floats) etc. It's worth thinking this about
as we discuss redesign issues.