matplotlib-devel Mailing List for matplotlib (Page 5)

Christopher Barker wrote:
[...]
> 
> Does internal MPL code rely on mlab? if so, it needs to be part of some 
> core lib that is stable enough to be a MPL dependency.
> 
Good question. With a couple of minor exceptions (which are easily 
handled), the only dependency is in axes methods that I don't think 
should be axes methods: xcorr, psd, etc. These are not the sort of core 
plot types that belong at the heart of mpl; they should be out on the 
periphery, in something like an optional toolkit. They are more 
analysis than plotting. I wanted to do a reorganization along these 
lines a long time ago, but never got around to it. The question of what 
should be an axes method has been raised in the context of John's mpl1 
major rewrite ideas, but an incremental change might be feasible and 
worthwhile also. How much user code is relying on Axes.psd() etc? 
Probably not too much, and code modification required if these things 
were made functions that take an axes instance as the first argument 
would not be large. The methods could be moved out of Axes and converted 
to functions without affecting the corresponding pylab API.
Eric

Fernando Perez wrote:
> On 8/12/07, Eric Firing <ef...@ha...> wrote:
>> mlab.py in svn has now been changed quite a bit:
> But if I may make a small comment, in my view MPL should simply not
> contain any numerics, period (beyond that needed for plotting
> functions). 
For the most part,m I agree with Fernando on this. However, the numpy 
team is making a concerted effort to have numpy be pretty straight core 
functionality. All the stuff in mlab will not and should not be put into 
numpy.
However, it's a lot of useful stuff. What to do?
I think it probably belongs in SciPy -- after all, SciPy is a lot of 
useful stuff (some domain-specific, some not) built on top of numpy.
However, mlab is now in MPL, and I think Eric is doing a great job of 
cleaning things up, so that the mlab module it self could easily be 
moved somewhere else in the future.
A couple points to keep in mind:
Does internal MPL code rely on mlab? if so, it needs to be part of some 
core lib that is stable enough to be a MPL dependency.
Despite what Fernando and I think, a lot of people DO want MPL to be an 
all-in-one plotting AND computing environment -- like Matlab. I think 
Scipy is what should be the complete environment, and I think it's 
getting close, but there are all still issues, so MPL does fill a need.
-Chris

On 8/12/07, Eric Firing <ef...@ha...> wrote:
> mlab.py in svn has now been changed quite a bit:
>
> 1) It uses straight numpy.
> 2) In many but not all cases, I tried to simplify algorithms by using
> numpy capabilities. I did some testing, but beware: there may be new bugs.
> 3) For functions with numpy equivalents I put in deprecation warnings,
> with the intention that mlab be stripped down to things that are useful
> and are not readily available in numpy.
> 4) I added some comments to docstrings.
> 5) I added a "demean" function that is like detrend_mean but takes an
> axis argument. This ability to operate on a specified axis might be
> added to other functions in mlab.
>
> Please test the new version. Especially if there are functions that you
> contributed to mlab (Fernando, I think you provided many of them),
Unfortunately I'm getting ready to leave for scipy'07, so I won't be
able to look at this.
But if I may make a small comment, in my view MPL should simply not
contain any numerics, period (beyond that needed for plotting
functions). I realize that it's convenient to have a layer of
numerical functionality directly in there, since you control the
library and that lets you have in there what you want, when you want
it, rather than waiting for numpy to catch up. IPython used to do the
same (IPython.numutils, which is where that stuff in mlab came from),
but I've completely stopped. IPython also had plotting
(IPython.Gnuplot*), and I also stopped that, now all IPython has is
the bare minimum needed to support MPL (-pylab), but no actual
plotting facilities of its own.
In the days of Numeric/numarray/numerix, I think mlab made a lot of
sense. But I would hope that now, any truly useful numerical
contributions should go into numpy (there's no confusion anymore), so
that we have a clear separation of responsibilities:
- numpy: base numerics
- scipy: bigger algorithms, some domain-specific functionality
- scikits: extensions to scipy, most of the domain-specific
functionality, staging area for scipy candidates.
- mpl: plotting
- ipython: interactive work (and distributed computing)
It used to be that scipy had plotting and interactive stuff, mpl had
numerics and interactive code (the little standalone GTK-threaded
interpreter), ipython had numerics and plotting, etc. We all had a
bit of each other's domain.
Moving forward, I think we should all make an effort to avoid that.
Scipy by and large has shed all of its old plotting and interactive
support, IPython carries it but I haven't touched that code in years
(and it will just stay there, it won't move to the new branches).
In that regard, I think in the long run having MPL carry a numerical
layer confuses its purpose. Unless you (a collective you, the MPL
developers) decide that MPL is really meant to be a complete numerical
system, but then you should change the description from the website,
which currently reads:
"matplotlib is a python 2D plotting library which produces publication
quality figures in a variety of hardcopy formats and interactive
environments across platforms."
I realize that this requires more work, since it means asking the
numpy team for inclusion of things you might like to have and waiting
for that to happen, rather than just doing it yourselves. That
coordination takes time, obviously, and that's a scarce resource for
all of us. But I honestly think that now that the
Numeric/Numarray/numerix mess is finally behind us, this should be an
opportunity for completing the 'separation of responsibilities'
effort. In the long run, I think it will help new users coming to
Python to better understand what is what, it will make for a
stronger/better Numpy, etc.
I know this isn't an easy problem, because you are balancing
matlab-compatibility, backwards compatibility, ease of development
workflow, etc. In the end, I trust your judgment, I just wanted to
mention this point of view in case you find it feasible.
Cheers,
f

mlab.py in svn has now been changed quite a bit:
1) It uses straight numpy.
2) In many but not all cases, I tried to simplify algorithms by using 
numpy capabilities. I did some testing, but beware: there may be new bugs.
3) For functions with numpy equivalents I put in deprecation warnings, 
with the intention that mlab be stripped down to things that are useful 
and are not readily available in numpy.
4) I added some comments to docstrings.
5) I added a "demean" function that is like detrend_mean but takes an 
axis argument. This ability to operate on a specified axis might be 
added to other functions in mlab.
Please test the new version. Especially if there are functions that you 
contributed to mlab (Fernando, I think you provided many of them), 
please check to see whether I have fouled them up, or deprecated them 
when this should not have been done, or whether you think they should 
still be included at all.
This version is not intended as an endpoint but as a step in the 
evolution of mpl and pylab.
We should come up with a clear policy as to what belongs in mlab.py and 
what does not. As it settles down, the module docstring can be made 
more complete and useful.
Eric

On Aug 6, 2007, at 4:03 PM, Michael Droettboom wrote:
> There is now experimental support for custom fonts in math mode.
> Try the above, and let me know how it goes...
I finally had time to try out your new code a bit, and I like it. It 
works well for the very simple cases, like superscripts. One thing 
that I noticed is that the height of the superscript depends on the 
font size -- is this a hardwired distance, or relative to font size. 
Also, it seems that when using the regular unicode fonts, more 
difficult math expressions, e.g., integrals, look terrible. I guess 
this is the point when CM is unavoidable...
I have been thinking that it might be possible to use some of the CM 
sans fonts, like CM bright. Also, there is a Arev font set (vera, 
backwards -- basically Bitstream Vera with math extensions) that 
looks like it might be promising, but it seems that mathtext does not 
see these extented characters.
I found an excellent writeup on math/text font combinations in LaTeX 
here:
	http://ctan.tug.org/tex-archive/info/Free_Math_Font_Survey/survey.html
from this informative page:
	http://cg.scs.carleton.ca/~luc/greek.html
There are, of course, many others.
-Rob
----
Rob Hetland, Associate Professor
Dept. of Oceanography, Texas A&M University
http://pong.tamu.edu/~rob
phone: 979-458-0096, fax: 979-845-6331

Eric Firing wrote:
> I do not think mlab 
> should "from numpy import *"; if that is what someone wants, they can do 
> it explicitly themselves, very easily. Now, pylab should do something 
> like "from mlab import *; from numpy import *; from pyplot import *", 
> but mlab should be its own nice, clean set of functions.
+1
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

Christopher Barker wrote:
> Ryan May wrote:
>>> I for one will be happy to change my code; numerical stuff in
>>> numpy, plotting stuff in pylab (or pyplot?), though some things
>>> like linspace() may be hard to loose; that's really an mlab
>>> function and I can import mlab.
> 
> I'm a big namespace fan. I'd much rather see us all do something like:
> 
> import pyplot
> import numpy as N
> import mlab
> 
> etc.
> 
> In this case, there should be little name overlap between these modules, 
> and you could "import *" all of them if you really wanted.
> 
> However, they are not all maintained by the same folks, so some overlap 
> may occur, and that's a good reason for keeping them in separate namespaces.
> 
> In any case, it needs to be decided what the point of the mlab module 
> is. There was a discussion about this a few (quite a few!) years ago 
> when there was just Numeric, and it came with an mlab module (at least I 
> think it was called mlab). At that point, we decided that Matlab 
> compatibility wasn't really the point -- the point was that there were a 
> number of nifty utilities in Matlab that people missed, and that the 
> stuff in mlab wasn't completely Matlab compatible anyway (numpy and 
> Matlab are just plain different, after all). We more or less decided to 
> what we really needed was a "Utilities" module that could hold some of 
> those nifty useful functions (like linspace, etc), but not much was done 
> at the time. Since then, numpy as gained quite a bit of those features 
> anyway.
> 
> So for this mlab module -- what is the goal?
> 
> 1) A Matlab-like set of tools? In that case, it way want to default to 
> matrices, rather than arrays, etc. If that's what's wanted, I'd make it 
> a stand-alone module, that imports what it needs from numpy, but 
> provides a complete API.
It looks like it started out something like this, but I don't think it 
should stay that way. If someone wants something like this, I think 
they will have to do it themselves.
> 
> 2) A bunch of extra utilities that numpy doesn't have, but are handy. In 
> that case, I'd have it include only the extras, and one would need to 
> import numpy and mlab both. It could also "import *" numpy , then add 
> the extra stuff, but I'm too much of a namespace fan to advocate for 
> that. What happens when numpy adds a name that clashes with an mlab 
> name, for instance?
I think this is the direction mlab has evolved, and my intention is to 
take it the rest of the way. As with pylab, some degree of 
matlab-similarity can be retained, but the goal is not a clone--that 
would be neither attainable nor desirable. If someone wants something 
more clone-like, they can use octave or scilab. I do not think mlab 
should "from numpy import *"; if that is what someone wants, they can do 
it explicitly themselves, very easily. Now, pylab should do something 
like "from mlab import *; from numpy import *; from pyplot import *", 
but mlab should be its own nice, clean set of functions.
The strategy here is to make a clear distinction between modules that 
provide unique functionality, and modules that serve as namespace 
aggregators.
Eric
> 
> -Chris
> 
> 

Ryan May wrote:
>> I for one will be happy to change my code; numerical stuff in
>> numpy, plotting stuff in pylab (or pyplot?), though some things
>> like linspace() may be hard to loose; that's really an mlab
>> function and I can import mlab.
I'm a big namespace fan. I'd much rather see us all do something like:
import pyplot
import numpy as N
import mlab
etc.
In this case, there should be little name overlap between these modules, 
and you could "import *" all of them if you really wanted.
However, they are not all maintained by the same folks, so some overlap 
may occur, and that's a good reason for keeping them in separate namespaces.
In any case, it needs to be decided what the point of the mlab module 
is. There was a discussion about this a few (quite a few!) years ago 
when there was just Numeric, and it came with an mlab module (at least I 
think it was called mlab). At that point, we decided that Matlab 
compatibility wasn't really the point -- the point was that there were a 
number of nifty utilities in Matlab that people missed, and that the 
stuff in mlab wasn't completely Matlab compatible anyway (numpy and 
Matlab are just plain different, after all). We more or less decided to 
what we really needed was a "Utilities" module that could hold some of 
those nifty useful functions (like linspace, etc), but not much was done 
at the time. Since then, numpy as gained quite a bit of those features 
anyway.
So for this mlab module -- what is the goal?
1) A Matlab-like set of tools? In that case, it way want to default to 
matrices, rather than arrays, etc. If that's what's wanted, I'd make it 
a stand-alone module, that imports what it needs from numpy, but 
provides a complete API.
2) A bunch of extra utilities that numpy doesn't have, but are handy. In 
that case, I'd have it include only the extras, and one would need to 
import numpy and mlab both. It could also "import *" numpy , then add 
the extra stuff, but I'm too much of a namespace fan to advocate for 
that. What happens when numpy adds a name that clashes with an mlab 
name, for instance?
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

Michael Droettboom wrote:
[...]
> All this raises a much larger question of whether we want to replace
> font_manager.py with something like fontconfig. There don't appear to
> be Python wrappers for it, but it would eliminate a number of issues
> with font_manager.py : a) it would look up fonts in the same way as many
> other apps in the OS (rather than from hard-coded folders), and b) seems
> a little more correct in how it looks up fonts and substitutes for
> missing fonts etc. For instance, if font_manager.py doesn't find an 
> exact match, it always returns Vera Sans, rather than returning 
> something with the same slant and weight as what you requested. There 
> would be no need to manually flush the matplotlib font cache, since 
> fontconfig would deal with all of that.
> 
> [The Python wrappers are probably a non-issue on Unix-like systems, 
> since you can call out to the "fc-match" command for lookup.]
> 
> This may be more effort than it's worth -- but I wanted to get opinions 
> from others on this list.
I find the fontconfig web site inscrutable; I don't understand how the 
library would be used by mpl in practice. I get the impression that it 
needs to be installed and configured at a system level--is this correct? 
 How would it work on OSX and Win?
Eric
> 
> Cheers,
> Mike

Tom Holroyd (NIH/NIMH) [E] wrote:
[...]
> 
> For the C library you say "man strcpy" or "man erf" and it tells you 
> what include file to use to get the definition (function prototype). In 
> python you only get the help after you've already imported. But it was 
> and still is reasonable to have an import statement like an #include to 
> define stuff you need. If I'm looking at somebody else's code and I see 
> "linspace" or something, it would be nice to have a "man linspace" or 
> so, that tells me which module I need to import. Is there a way to 
> convert doc strings to man pages?
I don't know of any such tool, but it could be done. I don't think it 
would solve the problem you refer to, though, which I understand as 
wanting to know where a given function actually comes from. One of the 
problems with creating a huge number of man pages for this is that 
function names may be duplicated; linspace, for example, could be coming 
from numpy or from mlab (until we get everything cleaned up...).
So, I think you are stuck with two approaches: actually tracing imports 
back through the code, or running ipython, importing a module, and then 
using the ? facility:
In [1]:import pylab
In [2]:pylab.linspace?
Type: function
Base Class: <type 'function'>
String Form: <function linspace at 0x8541a04>
Namespace: Interactive
File: 
/usr/local/lib/python2.5/site-packages/numpy/lib/function_base.py
Definition: pylab.linspace(start, stop, num=50, endpoint=True, 
retstep=False)
Docstring:
 Return evenly spaced numbers.
 Return num evenly spaced samples from start to stop. If
 endpoint is True, the last sample is stop. If retstep is
 True then return the step value used.
Which shows that at present, the linspace supplied by pylab is actually 
the numeric version, not the mlab version. This is not at all obvious 
from looking at pylab.py!
Eric

I just finished writing code to support scalable mathtext with the Cairo
backend. (The old version rendered the mathtext to bitmaps first).
Mostly straightforward, but I ran into one small snag. It *seems* that
pycairo requires that the fonts it uses are installed and accessible
through fontconfig, i.e. you can not load a font by filename. Not a big
deal, but there is also no way to tell if a particular font family
matched exactly, and thus no way to warn a user that they need to
install the Bakoma fonts.
There are some ways out of this. It appears possible to load a font
from a file for Cairo in C (though apparently different for each
platform). I would hate to have to support another extension in
matplotlib, but maybe the pycairo folks would be amenable to adding
this. Probably worth dropping a note to the pycairo list either way.
Or, we could find some way to install fonts automatically. That's
fundamentally a packaging/distribution issue.
All this raises a much larger question of whether we want to replace
font_manager.py with something like fontconfig. There don't appear to
be Python wrappers for it, but it would eliminate a number of issues
with font_manager.py : a) it would look up fonts in the same way as many
other apps in the OS (rather than from hard-coded folders), and b) seems
a little more correct in how it looks up fonts and substitutes for
missing fonts etc. For instance, if font_manager.py doesn't find an 
exact match, it always returns Vera Sans, rather than returning 
something with the same slant and weight as what you requested. There 
would be no need to manually flush the matplotlib font cache, since 
fontconfig would deal with all of that.
[The Python wrappers are probably a non-issue on Unix-like systems, 
since you can call out to the "fc-match" command for lookup.]
This may be more effort than it's worth -- but I wanted to get opinions 
from others on this list.
Cheers,
Mike

Just a bad example. I know there are functions that were originally defined in Matlab and then got rewritten in python, but I've never been clear on which modules they really live in. 
Ryan May wrote:
> Tom Holroyd (NIH/NIMH) [E] wrote:
>> Eric Firing wrote:
>>> Similarly, after dealing with mlab, I would like to simplify pylab. 
>>> Right now, we have a horrible tangle of namespaces in pylab. Cleaning 
>>> this up will potentially break user code; if a numpy function formerly 
>>> could be referenced with three different names and we knock that down to 
>>> one, code using the other two will not work. My guess is that in 
>>> practice the amount of breakage will be *very* small and easy for users 
>>> to deal with.
>> I for one will be happy to change my code; numerical stuff in numpy, plotting stuff in pylab (or pyplot?), though some things like linspace() may be hard to loose; that's really an mlab function and I can import mlab. It probably makes more sense.
> 
> I'm not sure if linspace was just a bad example, but it does exist
> within numpy itself.
> 
> Ryan
> 
-- 
I say to you: good and evil which would be everlasting- it does not
exist! Of its own accord must it ever overcome itself anew.
	-- thus spoke Zarathustra

Tom Holroyd (NIH/NIMH) [E] wrote:
> 
> Eric Firing wrote:
>> Similarly, after dealing with mlab, I would like to simplify pylab. 
>> Right now, we have a horrible tangle of namespaces in pylab. Cleaning 
>> this up will potentially break user code; if a numpy function formerly 
>> could be referenced with three different names and we knock that down to 
>> one, code using the other two will not work. My guess is that in 
>> practice the amount of breakage will be *very* small and easy for users 
>> to deal with.
> 
> I for one will be happy to change my code; numerical stuff in numpy, plotting stuff in pylab (or pyplot?), though some things like linspace() may be hard to loose; that's really an mlab function and I can import mlab. It probably makes more sense.
I'm not sure if linspace was just a bad example, but it does exist
within numpy itself.
Ryan
-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Eric Firing wrote:
> Similarly, after dealing with mlab, I would like to simplify pylab. 
> Right now, we have a horrible tangle of namespaces in pylab. Cleaning 
> this up will potentially break user code; if a numpy function formerly 
> could be referenced with three different names and we knock that down to 
> one, code using the other two will not work. My guess is that in 
> practice the amount of breakage will be *very* small and easy for users 
> to deal with.
I for one will be happy to change my code; numerical stuff in numpy, plotting stuff in pylab (or pyplot?), though some things like linspace() may be hard to loose; that's really an mlab function and I can import mlab. It probably makes more sense.
For the C library you say "man strcpy" or "man erf" and it tells you what include file to use to get the definition (function prototype). In python you only get the help after you've already imported. But it was and still is reasonable to have an import statement like an #include to define stuff you need. If I'm looking at somebody else's code and I see "linspace" or something, it would be nice to have a "man linspace" or so, that tells me which module I need to import. Is there a way to convert doc strings to man pages?
Speaking as somebody who has bascially switched from "write it in C and use assembly for speed" to "write it in Python and use C for speed".
-- 
I say to you: good and evil which would be everlasting- it does not
exist! Of its own accord must it ever overcome itself anew.
	-- thus spoke Zarathustra

Christopher Barker wrote:
>> http://www.artima.com/weblogs/viewpost.jsp?thread=95863
> 
> Thanks
>> (It was in Mike's original message.)
> 
> I must have missed that. Guido says:
> 
> """
> I've yet to see an extension module using SWIG that doesn't make me 
> think it was a mistake to use SWIG instead of manually written wrappers. 
> The extra time paid upfront to create hand-crafted wrappers is gained 
> back hundredfold by time saved debugging the SWIG-generated code later.
> """
> 
> hmm. wxPython is my prime example. I can't imagine that ever being done 
> enough to be useful without auto code generation. Period. Of course, 
> that doesn't apply to far smaller libraries.
I agree with you that some form of automation is absolutely necessary 
for these large libraries. However, wxPython also proves, IMHO, that 
using SWIG specifically is no walk in the park. wxPython has put a lot 
of effort into patching bugs in SWIG (which never seem to make it into 
upstream SWIG), and writing complicated extensions and workarounds. It 
is not a silver bullet for reference counting -- wxPython has had 
reference counting bugs as recently as 2.8.3. In fact, since SWIG 
typemaps are required so often (which is just raw Python/C API code 
anyway), there are still opportunities to make reference counting bugs, 
and they are harder to track down since you're one step removed from the 
real code.
> And many of the hand-written wrappers I've seen are nightmares of 
> incorrect reference counting. I think hand-wrapping is a just plain bad 
> idea, not when you have ctypes and pyrex and Boost (and CXX?) as options 
> instead.
I'll have to respectfully disagree with your assessment of 
hand-wrapping. It is the least likely to become unsupported of any of 
these options, and the Python/C API has been reasonably stable over a 
number of revisions (certainly since new-style classes were introduced, 
at least), and is extremely well-documented.
Cheers,
Mike

Eric Firing wrote:
> Michael Droettboom wrote:
>> I appreciate the way it fits so nicely into C++ ideas about RAII and 
>> exceptions -- but if you're not a C++ guy, that pro is probably a con.
> What is RAII?
"Resource Aquisition is Initialization" -- It's a C++ memory management 
technique where all resources are allocated in constructors and 
destroyed in destructors (I'm grossly oversimplifying). It allows 
memory management to be mostly hidden from the users of classes, and 
exceptions to work as they were intended (without lots of try/catch 
blocks everywhere.) RAII is not the solution to all memory management 
problems, of course, but it's a pretty common and important rule of 
thumb for C++. See here for more info:
http://www.parashift.com/c++-faq-lite/big-picture.html#faq-6.18
PyCxx uses RAII to manage the lifetime of Python objects without 
requiring explicit reference counting. For instance, when you get a 
Py::Int from an argument, the reference count of its "owned" Python 
object is increased in the constructor and decreased in the destructor. 
 So when the Py::Int goes out of scope, it automatically destroys its 
reference to the underlying Python object.
In this way, I see pycxx less as a wrapper mechanism (like SWIG or even 
Boost), and more like C++ convenience and safety extensions to the 
regular Python/C API. Given my familiarity with the Python/C API, that 
could be why I like it.
> I don't know how well pyrex will be maintained and updated, long-term.
Which, of course, is one of the impetuses (impeti?) for moving away from 
CXX. We'll have to tread carefully.
>> I suppose my initial disappointment in SWIG is that I like its 
>> fundamental idea -- of automating the tedious boilerplate -- but the 
>> execution of it just seems so heavyweight. But maybe that doesn't 
>> really matter. This is Python after all ;)
> 
> I think it is a perfectly valid concern and consideration.
Some benchmarking may be in order. I think it would be useful to know 
the difference in function call overhead between the different 
approaches, for instance. And overall memory usage is probably a 
secondary concern.
Cheers,
Mike

> http://www.artima.com/weblogs/viewpost.jsp?thread=95863
Thanks
> (It was in Mike's original message.)
I must have missed that. Guido says:
"""
I've yet to see an extension module using SWIG that doesn't make me 
think it was a mistake to use SWIG instead of manually written wrappers. 
The extra time paid upfront to create hand-crafted wrappers is gained 
back hundredfold by time saved debugging the SWIG-generated code later.
"""
hmm. wxPython is my prime example. I can't imagine that ever being done 
enough to be useful without auto code generation. Period. Of course, 
that doesn't apply to far smaller libraries.
And many of the hand-written wrappers I've seen are nightmares of 
incorrect reference counting. I think hand-wrapping is a just plain bad 
idea, not when you have ctypes and pyrex and Boost (and CXX?) as options 
instead.
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

Christopher Barker wrote:
> 
> Eric Firing wrote:
>> Swig: 
>> GvR's comments are interesting.
> 
> Do you have a pointer to those? I'd love to see them.
http://www.artima.com/weblogs/viewpost.jsp?thread=95863
(It was in Mike's original message.)
Eric

Michael Droettboom wrote:
[...]
> 
> I think the small amount of support is a valid concern, particularly as 
> we move toward Py3k, if no one steps up to the plate to help with the 
> migration.
Do you have a sense of how difficult that migration would be?
> 
> I appreciate the way it fits so nicely into C++ ideas about RAII and 
> exceptions -- but if you're not a C++ guy, that pro is probably a con.
What is RAII?
Actually, to the limited extent that I understand it, I *like* what I 
take to be the pycxx approach, transparently translating C++ constructs 
into corresponding Python constructs (e.g., exceptions). And I 
appreciate some of the ways that C++ advances beyond C. I just get all 
tangled up in the complicated declarations and the need to track code in 
.cpp and .h files in parallel. C++ feels an order of magnitude more 
complex to me than C. My discomfort level is very gradually decreasing, 
though.
> Pyrex is "fun to use" and it allows code to be more gradually migrated 
> from Python to C. We would need to take care to not add another 
> run-time dependency for users. (As an aside, I can't find the license 
> of Pyrex. Does anyone know what it is?)
Not a problem, as Chris has noted in another reply. Pyrex is already 
being used in Jeff Whitaker's basemap module, which is a critical 
contribution to mpl for many users although it is not presently in the 
main tree.
I don't know how well pyrex will be maintained and updated, long-term.
[...]
> I suppose my initial disappointment in SWIG is that I like its 
> fundamental idea -- of automating the tedious boilerplate -- but the 
> execution of it just seems so heavyweight. But maybe that doesn't 
> really matter. This is Python after all ;)
I think it is a perfectly valid concern and consideration.
Eric

Michael Droettboom wrote:
> Pyrex is "fun to use" and it allows code to be more gradually migrated 
> from Python to C. We would need to take care to not add another 
> run-time dependency for users.
There are no run-time dependencies with Pyrex -- it generates C code, 
which is then compiled.
 > (As an aside, I can't find the license
> of Pyrex. Does anyone know what it is?)
No, but it doesn't matter as it won't get included in the project, only 
it's output would -- so it's like a compiler, not a library. Unless we 
want to write our own fork of it, which I doubt!
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

Eric Firing wrote:
> Swig: 
> GvR's comments are interesting.
Do you have a pointer to those? I'd love to see them.
My thoughts on SWIG:
It's real strength is that, being automated, it can be used to wrap 
large libraries, particularly ones that are constantly evolving 
(wxPython). Another one is that it is used for a lot of projects, so 
it's a handy tool to know. I'd never use it to wrap code written just to 
extend python.
By the way, it looks like there is now a code generator for boost:
http://www.language-binding.net/pyplusplus/pyplusplus.html
so that's another option -- but talk about dependencies!
And there's SIP, but I've never heard of it being used by anything other 
than PyQT
-Chris
-- 
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...

I don't mind this going on the list with the warning that I was not 
trying to start any sort of anti-SWIG movement -- These are just my 
first impressions, and at the end of the day I may feel differently. I 
think you're *dead on* that personal experience and preferences have a 
lot to do with how these options are evaluated. Although, ideally, it 
would be nice to find some common ground so we aren't all just hacking 
on our own things, obviously ;) And it would be nice to find a single 
tool that is suitable for both C and C++. And it would be nice if it 
made my dinner ;)
Eric Firing wrote:
> 2) Given my discomfort with C++, I am even more uncomfortable relying on 
> what seems to be an almost unmaintained component. All the warnings 
> were worrying me, for example. Your recent fixups have reduced that 
> source of discomfort!
My recent fixup was exactly 6 characters -- glad to see they had so much 
impact! ;)
I think the small amount of support is a valid concern, particularly as 
we move toward Py3k, if no one steps up to the plate to help with the 
migration.
I appreciate the way it fits so nicely into C++ ideas about RAII and 
exceptions -- but if you're not a C++ guy, that pro is probably a con.
> Swig: I agree with you on that one. I have never taken to swig either, 
> although I recognize that it has many fans. I'm afraid that it may be 
> the only automated game in town for C++ apart from PyCxx, though.
Boost.Python is another option -- probably more like SWIG than pycxx in 
that the user specifies *how* something should be called rather than the 
steps to do it. But if you're C++ adverse, that's a huge step in the 
wrong direction -- it's pretty simple to use when things go well, but 
when things go wrong, the error messages can be completely inscrutable.
ctypes is also an option for calling C code, though I have no experience 
with it.
> Pyrex: now, there is something I like! Not so great for C++, but it 
> surely is nice for wrapping C, and for speeding some things up while 
> writing "almost Python".
Pyrex is "fun to use" and it allows code to be more gradually migrated 
from Python to C. We would need to take care to not add another 
run-time dependency for users. (As an aside, I can't find the license 
of Pyrex. Does anyone know what it is?)
> Native API: I've done a little of that--the cntr.c interface code. The 
> good things are that it eliminates another layer of dependency, and what 
> you see is what you get. It is pretty tedious, though, and there are 
> lots of gotchas. For me it is a reasonable choice for C code, although 
> I would still gravitate towards Pyrex in most cases. For C++, I would 
> not know where to start. Again, that's my personal handicap.
Agreed, it can be tricky to mix C++ exceptions with raw Python/C API 
wrappers. It's doable by following certain patterns, but it'd be nice 
to automate those somehow.
When I worked on Gamera, there was nothing terribly suitable for the 
sort of generic programming C++ stuff we had (the kind of C++ that makes 
you cringe, I'm sure), so we ended up writing our own wrapper mechanism 
that was very highly specialized. I don't think that approach is 
necessary or desirable here.
I suppose my initial disappointment in SWIG is that I like its 
fundamental idea -- of automating the tedious boilerplate -- but the 
execution of it just seems so heavyweight. But maybe that doesn't 
really matter. This is Python after all ;)
Cheers,
Mike
> Michael Droettboom wrote:
>> Eric Firing wrote:
>>> Mike,
>>>
>>> Thanks for fixing pycxx to remove all those warnings!
>>>
>>> I am curious: have you looked at the most recent upstream pycxx? My 
>>> impression is that there were supposedly some changes to support 
>>> python 2.5, and a couple months ago I made a feeble and unsuccessful 
>>> attempt to use the updated version in place of the one included in mpl.
>>
>> I looked at it a while ago because I suspected it might be the cause 
>> of a segfault. (Turned out it wasn't). I diffed it against the mpl 
>> one and didn't see much of significance, but I was really only looking 
>> for potential memory usage mistakes.
>>
>> It looks as if some Python2.5 changes have already been made in mpl. 
>> All the references to Py_ssize_t are a Python2.5-ism, for instance.
>>
>>> In any case, perhaps your improvements can be incorporated upstream.
>>
>> It actually looks like someone else has already arrived at the 
>> identical fix!
>>
>>> Longer term, I will be pleased if our dependence on pycxx can be 
>>> removed.
>>
>> It's funny you bring this up. I know that's on John's wish list. I 
>> just spent a little time looking at SWIG (trying to convert my simple 
>> ttconv extension to use SWIG), and I have some real concerns about it.
>>
>> For one, the syntax is weird, and it seems like any real-world 
>> non-trivial extension requires all kinds of magic goo.
>>
>> Secondly, it generates very bloated and suboptimal wrapper code, even 
>> after experimenting with the myriad of options to cut down on what 
>> gets generated. With my ttconv extension the object file doubled in 
>> size, since it appears to be throwing in all kinds of stuff we don't 
>> need.
>>
>> GvR said:
>>
>> "I've yet to see an extension module using SWIG that doesn't make me 
>> think it was a mistake to use SWIG instead of manually written 
>> wrappers. The extra time paid upfront to create hand-crafted wrappers 
>> is gained back hundredfold by time saved debugging the SWIG-generated 
>> code later."
>>
>> http://www.artima.com/weblogs/viewpost.jsp?thread=95863
>>
>> In my own experience, writing to the Python/C API directly is not 
>> horrible, but there are lots of pits to fall down if you're not 
>> careful. GvR wrote a lot of that API, so I'm not sure I agree with 
>> the full strength of his statement ;)
>>
>> However, SWIG rubs me the wrong way, including its history of needing 
>> a specific version to get a specific result. (I don't know if they're 
>> still breaking backward compatibility at the same rate that they once 
>> were.)
>>
>> I would hate to move away from CXX because it is less popular (unless 
>> the lower popularity translates into a real loss of utility, like 
>> updates following Python releases). There are a lot of things in this 
>> world that less popular but still superior -- think of beers for 
>> instance ;) I would just advocate looking at all the alternatives to 
>> SWIG, including raw Python/C API.
>>
>> (Again, I'm late to the party, so forgive me if my assumptions are 
>> wrong or these things have already been worked through.)
>>
>> Cheers,
>> Mike
> 

Mike,
I hope you don't mind my including the devel list on this; I think it is 
a very important topic (although it may be one that ends up coming down 
to each individual's personal experience and preferences).
The reasons pycxx worries me are:
0) C++ worries me in general--I know lots of wonderful code like Agg is 
written in C++, but personally I find C++ hard to deal with. This is 
mainly my problem. (But maybe not entirely. From the SWIG history:
"February 11, 2000. SWIG1.3 alpha released. This is the first in a 
series of releases that slowly migrate most of SWIG's implementation 
back to ANSI C.")
1) Transform module classes wrapped with pycxx have some strange 
characteristics, and don't behave like nice python classes. Sorry this 
is vague, but I stumbled on it, and John confirmed it, months ago, and 
then I blissfully forgot all the details. I can probably dredge them up 
from the mailing list if it would be of interest.
2) Given my discomfort with C++, I am even more uncomfortable relying on 
what seems to be an almost unmaintained component. All the warnings 
were worrying me, for example. Your recent fixups have reduced that 
source of discomfort!
Swig: I agree with you on that one. I have never taken to swig either, 
although I recognize that it has many fans. I'm afraid that it may be 
the only automated game in town for C++ apart from PyCxx, though. GvR's 
comments are interesting.
Pyrex: now, there is something I like! Not so great for C++, but it 
surely is nice for wrapping C, and for speeding some things up while 
writing "almost Python".
Native API: I've done a little of that--the cntr.c interface code. The 
good things are that it eliminates another layer of dependency, and what 
you see is what you get. It is pretty tedious, though, and there are 
lots of gotchas. For me it is a reasonable choice for C code, although 
I would still gravitate towards Pyrex in most cases. For C++, I would 
not know where to start. Again, that's my personal handicap.
Eric
Michael Droettboom wrote:
> Eric Firing wrote:
>> Mike,
>>
>> Thanks for fixing pycxx to remove all those warnings!
>>
>> I am curious: have you looked at the most recent upstream pycxx? My 
>> impression is that there were supposedly some changes to support 
>> python 2.5, and a couple months ago I made a feeble and unsuccessful 
>> attempt to use the updated version in place of the one included in mpl.
> 
> I looked at it a while ago because I suspected it might be the cause of 
> a segfault. (Turned out it wasn't). I diffed it against the mpl one 
> and didn't see much of significance, but I was really only looking for 
> potential memory usage mistakes.
> 
> It looks as if some Python2.5 changes have already been made in mpl. All 
> the references to Py_ssize_t are a Python2.5-ism, for instance.
> 
>> In any case, perhaps your improvements can be incorporated upstream.
> 
> It actually looks like someone else has already arrived at the identical 
> fix!
> 
>> Longer term, I will be pleased if our dependence on pycxx can be removed.
> 
> It's funny you bring this up. I know that's on John's wish list. I 
> just spent a little time looking at SWIG (trying to convert my simple 
> ttconv extension to use SWIG), and I have some real concerns about it.
> 
> For one, the syntax is weird, and it seems like any real-world 
> non-trivial extension requires all kinds of magic goo.
> 
> Secondly, it generates very bloated and suboptimal wrapper code, even 
> after experimenting with the myriad of options to cut down on what gets 
> generated. With my ttconv extension the object file doubled in size, 
> since it appears to be throwing in all kinds of stuff we don't need.
> 
> GvR said:
> 
> "I've yet to see an extension module using SWIG that doesn't make me 
> think it was a mistake to use SWIG instead of manually written wrappers. 
> The extra time paid upfront to create hand-crafted wrappers is gained 
> back hundredfold by time saved debugging the SWIG-generated code later."
> 
> http://www.artima.com/weblogs/viewpost.jsp?thread=95863
> 
> In my own experience, writing to the Python/C API directly is not 
> horrible, but there are lots of pits to fall down if you're not careful. 
> GvR wrote a lot of that API, so I'm not sure I agree with the full 
> strength of his statement ;)
> 
> However, SWIG rubs me the wrong way, including its history of needing a 
> specific version to get a specific result. (I don't know if they're 
> still breaking backward compatibility at the same rate that they once 
> were.)
> 
> I would hate to move away from CXX because it is less popular (unless 
> the lower popularity translates into a real loss of utility, like 
> updates following Python releases). There are a lot of things in this 
> world that less popular but still superior -- think of beers for 
> instance ;) I would just advocate looking at all the alternatives to 
> SWIG, including raw Python/C API.
> 
> (Again, I'm late to the party, so forgive me if my assumptions are wrong 
> or these things have already been worked through.)
> 
> Cheers,
> Mike

Hi Eric,
thanks for the warning. I had looked into the issue but never got around
to start working on it.
While you are at it, I would suggest that you split the graphics stuff
out of pylab into a separate
module (pyplot?) to be used for "interactive plotting".
I believe I'm not the only user who likes the stateful plotting
functionality (i.e. based on gca(), gcf() etc.) but does not want to
import all the matlab-compatibility numerics-functions along. Typically,
I prefer the nice and clean numpy stuff to do numerics.
I would suggest to create an additional module (e.g. pyplot?) that only
contains the stateful plotting routines and a module pylab that simply
imports everything from pyplot and mlab.
Greetings,
Norbert
Eric Firing wrote:
> I have been gradually numpifying mlab.py; I hope this is not a 
> duplication of effort with someone else. I have not finished yet (and 
> have made no commits), and it may still take a while--there are a lot of 
> things to change, and I am trying to do some cleaning up and 
> rationalization, not just mechanical numpification. I will probably 
> commit a mlab1.py version so as to facilitate some extra testing before 
> making the switch.
>
> I am trying to simplify and shrink the mlab namespace, with the idea 
> that if a function exists in modern numpy it should simply be imported 
> >from there.
>
> Similarly, after dealing with mlab, I would like to simplify pylab. 
> Right now, we have a horrible tangle of namespaces in pylab. Cleaning 
> this up will potentially break user code; if a numpy function formerly 
> could be referenced with three different names and we knock that down to 
> one, code using the other two will not work. My guess is that in 
> practice the amount of breakage will be *very* small and easy for users 
> to deal with.
>
> Eric
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems? Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
> 

Mike,
It is a good question.
Right now I am doing some of each. If a function was defined in mlab.py 
and it has a replacement in numpy I am issuing a deprecation warning 
and passing the arguments to the numpy function. If a function was not 
defined in mlab but was imported from numerix, then, with a couple of 
exceptions, I am simply not leaving it in the mlab namespace.
This is in the present (old) mlab.py:
from numerix import array, asarray, arange, divide, exp, arctan2, \
 multiply, transpose, ravel, repeat, resize, reshape, floor, ceil,\
 absolute, matrixmultiply, power, take, where, Float, Int, asum,\
 dot, convolve, pi, Complex, ones, zeros, diagonal, Matrix, nonzero, \
 log, searchsorted, concatenate, sort, ArrayType, clip, size, indices,\
 conjugate, typecode, iscontiguous
For compatibility, access to some or all of these things may need to 
remain in the *pylab* namespace for a while, but I don't think it should 
stay in the *mlab* namespace as well. I really don't want to add 
wrappers with warnings for all these functions.
Eric
Michael Droettboom wrote:
> Eric Firing wrote:
>> Similarly, after dealing with mlab, I would like to simplify pylab. 
>> Right now, we have a horrible tangle of namespaces in pylab. Cleaning 
>> this up will potentially break user code; if a numpy function formerly 
>> could be referenced with three different names and we knock that down 
>> to one, code using the other two will not work. My guess is that in 
>> practice the amount of breakage will be *very* small and easy for 
>> users to deal with.
> 
> Speaking off the top of my head, without looking at the specifics -- 
> would it be useful to generate deprecation warnings for these functions 
> before we ultimately remove them? The deprecated functions could just 
> be decorated versions of the "correct" functions that raise a warning 
> and then delegate to the correct version.
> 
> May be overkill if these functions really are in low usage, as you suspect.
> 
> Cheers,
> Mike