matplotlib-devel Mailing List for matplotlib

On 3/20/07, Eric Firing <ef...@ha...> wrote:
> You accidentally whacked out the new Axes.matshow, so I put it back.
Oops, sorry.
> I also noticed a few decorators--gasp!--in axes.py. I presume you will
> want them replaced by old-style syntax to preserve 2.3 compatibility,
> but I will leave that to you. (After about the 10th or so time of
> reading a bit about decorators, I think I understand them enough for
> simple use cases; apart from that ugly and utterly unpythonic @ symbol,
> maybe they are not as bad as I thought.)
>
> The curmudgeon in me has to wonder whether the snazzy unit support is
> really a good thing; this is partly a question of where the boundary of
> a plotting library should be. The simpler view (classic mpl) is that
> the role of mpl is to do a good job plotting numbers and labeling
> things, and the role of the user or application programmer is to supply
> the numbers and labels. I am not sure that enough is gained by enabling
> unit conversion and automatic axis labeling inside a plot command to
> compensate for the added complexity. My hesitation probably reflects
> the facts (1) that I don't see any *compelling* use cases in the sort
> of work I do, (2) I am not familiar with whatever use cases motivated
> this, (3) I haven't thought about it much yet, and (4) I may be a bit
> unimaginative.
>
> I will try to take a closer look, both at the changes and at the
> questions you raise in your message, tomorrow.
I too have been concerned by the complexity of this implementation --
I think it is trying to support too many paradigms, for example,
sequences of hetergeneous units. I have dramatically simplified the
code, and moved almost everything out of Axes. I have also made
"units" an rc param so that if units is not enabled, there is a dummy
do nothing units manager so you'll only pay for a few extra do nothing
function calls. Take a look at the code again when you get a minute,
I think you'll be more satisfied at the reduced complexity.
I've also cleaned up the examples to hopefully make clearer the
potential use cases. Eg, for radian plotting
 from basic_units import radians, degrees
 from pylab import figure, show, nx
 x = nx.arange(0, 15, 0.01) * radians
 fig = figure()
 ax = fig.add_subplot(211)
 ax.plot(x, nx.cos(x), xunits=radians)
 ax.set_xlabel('radians')
 ax = fig.add_subplot(212)
 ax.plot(x, nx.cos(x), xunits=degrees)
 ax.set_xlabel('degrees')
 show()
and see attached screenshot. One of the things this implementation
buys you is the units writer can provide a mapping from types to
locators and formatters -- notice in the attached screenshot how you
get the fancy tick locating and formatting. This enables a matplotlib
application developer to alter the default ticking and formatting
outside of the code base.
Here is another use case, working with native datetimes -- note that
we get to use native dates in plot and set_xlim
 import date_support # set up the date converters
 import datetime
 from pylab import figure, show, nx
 xmin = datetime.date(2007,1,1)
 xmax = datetime.date.today()
 xdates = [xmin]
 while 1:
 thisdate = xdates[-1] + datetime.timedelta(days=1)
 xdates.append(thisdate)
 if thisdate>=xmax: break
 fig = figure()
 fig.subplots_adjust(bottom=0.2)
 ax = fig.add_subplot(111)
 ax.plot(xdates, nx.mlab.rand(len(xdates)), 'o')
 ax.set_xlim(datetime.date(2007,2,1), datetime.date(2007,3,1))
 for label in ax.get_xticklabels():
 label.set_rotation(30)
 label.set_ha('right')
 show()
Some of the features were inspired by some real use cases that the JPL
has encountered in developing their monty application for ground
tracking of orbiting spacecraft. The basic problem is this. Imagine
you are a large C++ shop with a lot of legacy code and a python
interface, and you've decided to jettison your internal plotting
library for matplotlib. Your users work at a enhanced python shell
and have all of the legacy functionality and objects and want to so
something like
>>> plot(o)
where o is one of these legacy objects. They may not know a lot of
matplotlib, but can do plot. Asking them to learn about tickers and
formatters and conversion to arrays etc may be a non-starter for many
of these users. You could wrap all of the bits of matplotlib that you
need and do the conversion under the hood for your users, but then you
will always be trying to keep up with the mpl changes. You can't
really change the objects to suit mpl because too much legacy code
depends on them. What the proposed changes allow the developer to do
is write a converter class and register their types with a unit
manager and mpl will do the conversions in the right place. In the
current implementation (heavily revised) this happens when the Axes
adds the artist to itself, which provides a finite number of points of
entry.
Here is what the converter code in date_support in the units example
directory (used in the demo above). Note that this pretty much
replaces all of the current plot_date functionality, but happens
outside mpl and doesn't require the user to think about date2num
 import matplotlib
 matplotlib.rcParams['units'] = True
 import matplotlib.units as units
 import matplotlib.dates as dates
 import matplotlib.ticker as ticker
 import basic_units
 import datetime
 class DateConverter(units.ConversionInterface):
 def tickers(x, unit=None):
 'return major and minor tick locators and formatters'
 majloc = dates.AutoDateLocator()
 minloc = ticker.NullLocator()
 majfmt = dates.AutoDateFormatter(majloc)
 minfmt = ticker.NullFormatter()
 return majloc, minloc, majfmt, minfmt
 tickers = staticmethod(tickers)
 def convert_to_value(value, unit):
 return dates.date2num(value)
 convert_to_value = staticmethod(convert_to_value)
 units.manager.converters[datetime.date] = DateConverter()
And I've gotten the units.py module down to a digestable 105 lines of
code!
I haven't finished porting all of the artists yet, eg there is work
left to do in collections and text, but lines, patches and regular
polygon collections are working, and there are more examples.
See if you find the new interface less onerous. There is still work
to do if we want to support this kind of thing -- one of the hard
parts is to modify the various plotting functions to try and get the
original data into the primitive objects, which the current approach
is building around.
I've also gotten rid of all the decorators and properties. The code
is not python2.3 compatible.
JDH

FYI The unit system John is working on will be a huge improvement for 
the way we use MPL. Our users do a ton of plotting that involves 
unitized numbers vs time. We have our own unit class and time class 
and right now users have to convert the unitized numbers into floats 
in the correct units and convert the times to the correct MPL format 
in the correct reference frame. Being able to seamlessly pass these 
objects to MPL is going to make all of our plotting scripts much 
simpler to use, easier to understand, and much safer (by eliminating 
different unit/time frame problems).
It's not a big deal to convert values when the plot is first created 
- where it makes the biggest difference is when you want to 
manipulate the plot after it's created (xlim for example). Being 
able to pass unitized numbers to the various manipulation methods is 
what makes everything much easier to use (especially when dates are 
being plotted).
Ted
At 02:15 PM 3/20/2007, Norbert Nemec wrote:
>Actually, I like the idea of unit support quite a bit and could well
>imagine that it makes sense
>to support it explicitely in matplotlib.
>
>I am using physical units very frequently in my computations. Lacking a
>robust units package,
>I simply define the units as numerical constants without checks but at
>least with comfortable
>conversion. If there were a good units package, support in matplotlib
>would mean that the axis
>labels could automatically be completed with appropriate units without
>need for explicit conversion.
>
>I agree, though, that the units package itself should not be part of
>matplotlib. But this is exactly
>how I understand the idea by John Hunter: describe an interface to allow
>the use of any third-party
>unit package.
>
>Of course, the whole thing only makes sense is there is a units package
>that is fit for production use.
>
>
>
>Darren Dale wrote:
> >
> > My first impression is similar to Eric's. I don't know if there 
> is a robust
> > units package for python, but I imagine it should be a part of 
> scipy. I think
> > it would be better to get an array and if you wanted to plot it 
> in different
> > units, you call a method on the array at plot time. Maybe I dont 
> understand
> > all the intended uses.
> >
> > Darren
> >
> >
>
>
>-------------------------------------------------------------------------
>Take Surveys. Earn Cash. Influence the Future of IT
>Join SourceForge.net's Techsay panel and you'll get the chance to share your
>opinions on IT & business topics through brief surveys-and earn cash
>http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>_______________________________________________
>Matplotlib-devel mailing list
>Mat...@li...
>https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Actually, I like the idea of unit support quite a bit and could well 
imagine that it makes sense
to support it explicitely in matplotlib.
I am using physical units very frequently in my computations. Lacking a 
robust units package,
I simply define the units as numerical constants without checks but at 
least with comfortable
conversion. If there were a good units package, support in matplotlib 
would mean that the axis
labels could automatically be completed with appropriate units without 
need for explicit conversion.
I agree, though, that the units package itself should not be part of 
matplotlib. But this is exactly
how I understand the idea by John Hunter: describe an interface to allow 
the use of any third-party
unit package.
Of course, the whole thing only makes sense is there is a units package 
that is fit for production use.
Darren Dale wrote:
>
> My first impression is similar to Eric's. I don't know if there is a robust 
> units package for python, but I imagine it should be a part of scipy. I think 
> it would be better to get an array and if you wanted to plot it in different 
> units, you call a method on the array at plot time. Maybe I dont understand 
> all the intended uses.
>
> Darren
>
> 

On Tuesday 20 March 2007 3:50:07 am Eric Firing wrote:
> John Hunter wrote:
> > If you are using mpl svn, please read this as it describes
> > some fairly major changes.
> >
> > Mike Lusignan has been working on adding units support, and as a
> > consequence, partial support for working with arbitrary types in mpl.
> > The support is not complete yet, but it is basically working and
> > compatible with the rest of mpl, so I thought now would be a good time
> > to integrate it into the svn HEAD (he's been working in a branch)
> > and get some more eyeballs on it.
>
> John,
>
> You accidentally whacked out the new Axes.matshow, so I put it back.
>
> I also noticed a few decorators--gasp!--in axes.py. I presume you will
> want them replaced by old-style syntax to preserve 2.3 compatibility,
> but I will leave that to you. (After about the 10th or so time of
> reading a bit about decorators, I think I understand them enough for
> simple use cases; apart from that ugly and utterly unpythonic @ symbol,
> maybe they are not as bad as I thought.)
>
> The curmudgeon in me has to wonder whether the snazzy unit support is
> really a good thing; this is partly a question of where the boundary of
> a plotting library should be. The simpler view (classic mpl) is that
> the role of mpl is to do a good job plotting numbers and labeling
> things, and the role of the user or application programmer is to supply
> the numbers and labels. I am not sure that enough is gained by enabling
> unit conversion and automatic axis labeling inside a plot command to
> compensate for the added complexity. My hesitation probably reflects
> the facts (1) that I don't see any *compelling* use cases in the sort
> of work I do, (2) I am not familiar with whatever use cases motivated
> this, (3) I haven't thought about it much yet, and (4) I may be a bit
> unimaginative.
My first impression is similar to Eric's. I don't know if there is a robust 
units package for python, but I imagine it should be a part of scipy. I think 
it would be better to get an array and if you wanted to plot it in different 
units, you call a method on the array at plot time. Maybe I dont understand 
all the intended uses.
Darren
-- 
Darren S. Dale, Ph.D.
dd...@co...

John Hunter wrote:
> If you are using mpl svn, please read this as it describes
> some fairly major changes.
> 
> Mike Lusignan has been working on adding units support, and as a
> consequence, partial support for working with arbitrary types in mpl.
> The support is not complete yet, but it is basically working and
> compatible with the rest of mpl, so I thought now would be a good time
> to integrate it into the svn HEAD (he's been working in a branch)
> and get some more eyeballs on it.
John,
You accidentally whacked out the new Axes.matshow, so I put it back.
I also noticed a few decorators--gasp!--in axes.py. I presume you will 
want them replaced by old-style syntax to preserve 2.3 compatibility, 
but I will leave that to you. (After about the 10th or so time of 
reading a bit about decorators, I think I understand them enough for 
simple use cases; apart from that ugly and utterly unpythonic @ symbol, 
maybe they are not as bad as I thought.)
The curmudgeon in me has to wonder whether the snazzy unit support is 
really a good thing; this is partly a question of where the boundary of 
a plotting library should be. The simpler view (classic mpl) is that 
the role of mpl is to do a good job plotting numbers and labeling 
things, and the role of the user or application programmer is to supply 
the numbers and labels. I am not sure that enough is gained by enabling 
unit conversion and automatic axis labeling inside a plot command to 
compensate for the added complexity. My hesitation probably reflects 
the facts (1) that I don't see any *compelling* use cases in the sort 
of work I do, (2) I am not familiar with whatever use cases motivated 
this, (3) I haven't thought about it much yet, and (4) I may be a bit 
unimaginative.
I will try to take a closer look, both at the changes and at the 
questions you raise in your message, tomorrow.
Eric

If you are using mpl svn, please read this as it describes
some fairly major changes.
Mike Lusignan has been working on adding units support, and as a
consequence, partial support for working with arbitrary types in mpl.
The support is not complete yet, but it is basically working and
compatible with the rest of mpl, so I thought now would be a good time
to integrate it into the svn HEAD (he's been working in a branch)
and get some more eyeballs on it.
The code base is a little complicated and daunting at first, but we
are working to try and simplify it and refactor it so the main
functionality is minimally invasive into the rest of the code base.
Right now it is somewhat distributed among units, figure, axes,
artist, lines, patched, etc, but will be consolidated in the upcoming
week(s). Not all of the plotting functions support units, but the
examples show some with scatter and plot.
The documentation is in matplotlib.units. We do not assume any
particular units package, we only require that package to provide
a certain interface. Alternatively, one can use a units type that doesn't
have the required interface as long as you register some adaptors with
the figure. More on this later. Mike also provided a mockup units package
in the examples/units dir called basic_unit.py to test and demo the support.
I ran into a little problem today in trying to reconcile Eric's work
supporting multicolumn y data with the unit support for arbitrary
types. The basic tension is in _process_plot_var_args._xy_from_xy and
friends which simplfies the array column logic by forcing all inputs
to have a array.shape==2 using some conversion functions. The problem
is this strips out the units tagging Mike is relying on (in the
current implementation he needs _xorig in Line2D to be the original
data type). This is a fairly important question that
requires some thought: do we want mpl objects to store original data
objects as long as they know how to convert themselves under the hood
to something useful when requested (eg Text now supports any object
that supports '%s'%o. I think if we could support this generally,
that is the ideal, because it let's users use mpl with custom objects,
possibly from 3rd party closed src vendors, as long as the objects
expose the right interface. It's also useful for picking, where you
might want to store your custom objects and arrays in mpl and query
them later. If we lose access to the orginal data when constructing
our objects, we lose this ability. That said, we are fairly far from
achieving this goal globally.
I did a quick-and-dirty hack in process_plot_var_args for the time
being to get something everyone can chew on, which is to use the
existing approach if the data are indeed multicolumn, but use the
original x and y data otherwise. We'll come up with something more
general and elegant shortly.
backend_driver is passing in my local repository, and the units
examples are passing as well, and I thought this is sufficient
progress that it merits getting the merge done now and getting more
testers on this. I expect there will be some breakage
and performance hits, and we can fix these as they arise.
The new examples are in examples/units -- a couple of screenshot
of the example below is attached.
Thanks Mike!
JDH
import matplotlib
matplotlib.rcParams['numerix'] = 'numpy'
import basic_units as bu
import numpy as N
from pylab import figure, show
from matplotlib.cbook import iterable
cm = bu.BasicUnit('cm', 'centimeters')
inch = bu.BasicUnit('inch', 'inches')
inch.add_conversion_factor(cm, 2.54)
cm.add_conversion_factor(inch, 1/2.54)
lengths_cm = cm*N.arange(0, 10, 0.5)
# iterator test
print 'Testing iterators...'
for length in lengths_cm:
 print length
print 'Iterable() = ' + `iterable(lengths_cm)`
print 'cm', lengths_cm
print 'toinch', lengths_cm.convert_to(inch)
print 'toval', lengths_cm.convert_to(inch).get_value()
fig = figure()
ax1 = fig.add_subplot(2,2,1)
ax1.plot(lengths_cm, 2.0*lengths_cm, xunits=cm, yunits=cm)
ax1.set_xlabel('in centimeters')
ax1.set_ylabel('in centimeters')
ax2 = fig.add_subplot(2,2,2)
ax2.plot(lengths_cm, lengths_cm, xunits=cm, yunits=inch)
ax2.set_xlabel('in centimeters')
ax2.set_ylabel('in inches')
ax3 = fig.add_subplot(2,2,3)
ax3.plot(lengths_cm, 2.0*lengths_cm, xunits=inch, yunits=cm)
ax3.set_xlabel('in inches')
ax3.set_ylabel('in centimeters')
ax4 = fig.add_subplot(2,2,4)
ax4.plot(lengths_cm, 2.0*lengths_cm, xunits=inch, yunits=inch)
ax4.set_xlabel('in inches')
ax4.set_ylabel('in inches')
fig.savefig('simple_conversion_plot.png')
show()