matplotlib-devel Mailing List for matplotlib

On Mon, Sep 21, 2009 at 7:04 AM, Michael Droettboom <md...@st...> wrote:
> It's caused by the files having multiple locations (relative to the root of
> the source tree) in different branches, or existing in some branches but not
> in others. In other words, files that have been moved, deleted or added
> since branch creation. I think if we retire old branches (0.91, 0.98),
> which I'm in favor of, we might alleviate some of that (I'm not 100% sure,
> but it seems plausible).
>
I think it is safe to delete these branches
JDH

The relevant bugs on the tracker have been addressed and I just did a
round of testing on the release branch and it looks like we are good
to go for 0.99.1. Unless something critical comes up in the next 24
hours, I'm tagging r7813 on the release branch for 0.99.1 and will
build the OS X binaries tonight. Christoph, unless you hear otherwise
in the interim, if you could build the win32 binaries from this
revision, I'll shoot for a release and announcement tomorrow.
Thanks,
JDH

Wow - thanks for the detailed update. I feel bad for making you type that much :)
Thanks for fixing that problem. 
Ted
> -----Original Message-----
> From: John Hunter [mailto:jd...@gm...]
> Sent: Sunday, September 20, 2009 7:35 PM
> To: Drain, Theodore R (343P)
> Cc: Andrew Straw; matplotlib development list
> Subject: Re: [matplotlib-devel] empty date formatter unit tests
> 
> On Sun, Sep 20, 2009 at 6:50 PM, Drain, Theodore R (343P)
> <the...@jp...> wrote:
> 
> > I've run into this problem quite a few times and I'd love to figure
> out some way to fix it. As an example, here's the kind of scenario
> this occurs in:
> >
> > I embed MPL in a few different GUI's that plot data either in real-
> time or via the user selecting things. There is a saved state which
> contains preferences like auto-scaling, legend on/off, axis formatting,
> etc. When the app starts up, I need to create a plot to put on the
> screen and configure it. What I'd like to do is this:
> >
> > - create widget
> > - apply format (date formatter, etc)
> > - apply settings (autoscale, etc)
> > - wait for data (either via real time feed or user clicking on
> things)
> >
> > But this is impossible because of this kind of bug. Instead, I have
> to create a plot with a fake date range and test every operation to see
> if it's actually setting data before applying the settings like
> autoscale. In addition, if the user removes data from the plot (via
> menu or selectable lists), I have to either start over or "unset" the
> settings back to something safe so this error won't occur. It really
> makes coding something like this a royal pain.
> >
> > I don't have a suggestion as of yet... Perhaps it could just return
> "N/A" or something like that.
> >
> > I think part of the problem might be the default ranges used by the
> autoscaling algorithm when there is no data are invalid for certain
> formatters and locators. That suggests that possible solutions might be
> one of:
> >
> > 1) require autoscaling or scaling algorithms to return ranges that
> will be OK for known scalers/formatters. Perhaps some system that
> allows different autoscaling algorithms to be set which can configure
> the default?
> > 2) require scalers/formatters to be robust for any range or engineer
> the system to allow them to report "errors" in a way that allows the
> plot do something reasonable and not trigger an exception (perhaps some
> changeable behavior w/ the default as an exception?).
> >
> > I'll think about this a little this week and see if any other ideas
> come to mind.
> 
> I think we have this problem mostly licked. The problem I was writing
> about in my email is a 2nd tier problem. For example, in svn HEAD,
> you can specify an "empty" date plot as long as you inform mpl of you
> intentions.. From the test_date_empty unit test::
> 
> fig = plt.figure()
> ax = fig.add_subplot(1,1,1)
> ax.xaxis_date()
> fig.savefig('date_empty')
> 
> Here we are fine, because we call ax.axaxis_date, which informs mpl
> that you intend to pass in datetime instances. The key piece which
> makes this possible, which you allude to in your post, is the default
> xlimits, which is new in svn HEAD. In particular, the default
> converter provides an AxisInfo which now supports an optional
> attribute
> 
> default_limits: the default min, max of the axis if no data is
> present
> 
> which is overridden in the DateConverter:
> 
> def axisinfo(unit, axis):
> 'return the unit AxisInfo'
> # make sure that the axis does not start at 0
> 
> majloc = AutoDateLocator(tz=unit)
> majfmt = AutoDateFormatter(majloc, tz=unit)
> datemin = datetime.date(2000, 1, 1)
> datemax = datetime.date(2010, 1, 1)
> 
> return units.AxisInfo( majloc=majloc, majfmt=majfmt, label='',
> default_limits=(datemin, datemax))
> 
> 
> while the min/max are arbitrary, the important thing is that custom
> types can now handle the default min/max limits, so when you present a
> new type to mpl, the type can request a certain default view/data lim
> if no data are presented. Additionally, because of the
> "ignore"setting on the limts argument, we can detect whether the
> limits we are applying are defaults or actively set by the user.
> 
> The complication that motivated the sf bug
> http://sourceforge.net/tracker/?func=detail&aid=2861426&group_id=80706&
> atid=560720
> is a bit more subtle. Here no data type is presented to mpl -- either
> through "plot" or "fill" or "set_xlim" or whatever. If the user had
> passed any data in, or manually expressed their intent through
> "ax.xaxis_date" we would be fine. The difficulty is that they passed
> no data in but declared their intention to use a "YearFormatter". My
> original inclination, and the one that failed the unit tests, was to
> trigger a call to Axis.axis_date (a new method) on a call to
> ax.xaxis.set_major_formatter (or locator) where the argument was a
> DateLocator or DateFormatter. This seemed to be an imminently
> reasonable and helpful thing to do -- if they want a date locator or
> formatter presumably they will be passing in dates -- but the unit
> tests told me this was wrong.
> 
> The locators and formatters work on *converted* units. The
> EpochConverter and DateConverter both convert their native types to
> floating point days since 0000-00-00. So here are two custom
> converter interfaces which both end up with the same floating point
> representation. The conclusion is: mpl cannot use the
> locator/formatter type to infer what the basic type that users will be
> passing in. Just because two classes end up with the same floating
> point representation does not indicate that they want the same
> conversion pipeline from type -> float.
> 
> Nonetheless, we can, and already do in svn HEAD, handle the cases that
> I think you are worried about in this email. As long as you know what
> type you will be passing into mpl (regardless of whether you have any
> data available right now) you can inform the units interface with
> 
> ax.xaxis.update_units(someval)
> 
> where someval is an instance of the type you plan to pass in. Doing
> so will not affect your current data or view limits, but will trigger
> the conversion interface and importantly will trigger the
> units.AxisInfo.default_limits scaling which was recently added to
> avoid the kinds of problems we have been seeing with date conversion
> when no data is passed in.
> 
> 
> So despite this long winded email, the current infrastructure should
> support
> 
> * create axes, etc
> 
> * set your current formatter/locator
> 
> * ax.xaxis.update_units(myInstance)
> 
> where myInstance is an object of the type you expect to pass in. As
> long as you have registered a converter from type(myInstance) ->
> ConversionInterface, you can now specify the default limits through
> the ConversionInterface.default_limits method::
> 
> @staticmethod
> def default_units(x, axis):
> 'return the default unit for x or None for the given axis'
> return None
> 
> 
> As an example in matplotlib.dates, we choose an arbitrary interval,
> which while arbitrary avoids the 0..1 problem we have been having::
> 
> class DateConverter(units.ConversionInterface):
> """The units are equivalent to the timezone."""
> 
> @staticmethod
> def axisinfo(unit, axis):
> 'return the unit AxisInfo'
> # make sure that the axis does not start at 0
> 
> majloc = AutoDateLocator(tz=unit)
> majfmt = AutoDateFormatter(majloc, tz=unit)
> datemin = datetime.date(2000, 1, 1)
> datemax = datetime.date(2010, 1, 1)
> 
> return units.AxisInfo( majloc=majloc, majfmt=majfmt, label='',
> default_limits=(datemin, datemax))
> 
> JDH
> 
> No virus found in this incoming message.
> Checked by AVG - www.avg.com
> Version: 8.5.409 / Virus Database: 270.13.110/2385 - Release Date:
> 09/20/09 17:51:00

On Sun, Sep 20, 2009 at 9:35 PM, John Hunter <jd...@gm...> wrote:
> where myInstance is an object of the type you expect to pass in. As
> long as you have registered a converter from type(myInstance) ->
> ConversionInterface, you can now specify the default limits through
> the ConversionInterface.default_limits method::
>
>  @staticmethod
>  def default_units(x, axis):
>    'return the default unit for x or None for the given axis'
>    return None
This is a typo: I should be referring to the default_limits attribute
of AxisInfo. Instead, I pasted in the default_units method of the
conversion interface
> As an example in matplotlib.dates, we choose an arbitrary interval,
> which while arbitrary avoids the 0..1 problem we have been having::
>
> class DateConverter(units.ConversionInterface):
>  """The units are equivalent to the timezone."""
>
>  @staticmethod
>  def axisinfo(unit, axis):
>    'return the unit AxisInfo'
>    # make sure that the axis does not start at 0
>
>    majloc = AutoDateLocator(tz=unit)
>    majfmt = AutoDateFormatter(majloc, tz=unit)
>    datemin = datetime.date(2000, 1, 1)
>    datemax = datetime.date(2010, 1, 1)
>
>    return units.AxisInfo( majloc=majloc, majfmt=majfmt, label='',
>                default_limits=(datemin, datemax))
This is the correct way to specify default_limits
JDH

It's caused by the files having multiple locations (relative to the root 
of the source tree) in different branches, or existing in some branches 
but not in others. In other words, files that have been moved, deleted 
or added since branch creation. I think if we retire old branches 
(0.91, 0.98), which I'm in favor of, we might alleviate some of that 
(I'm not 100% sure, but it seems plausible).
Mike
On 09/20/2009 01:21 PM, John Hunter wrote:
> Some files seem to get a properties modification on them every time
> someone does an svn merge, eg
>
> examples/misc/multiprocess.py
>
> which is rarely changed, but frequently has its svnmerge properties change
>
> home:~/mpl> svn diff -rPREV:HEAD examples/misc/multiprocess.py
>
> Property changes on: examples/misc/multiprocess.py
> ___________________________________________________________________
> Name: svn:mergeinfo
> - /branches/v0_91_maint/examples/misc/log.py:5753-5771
> /branches/v0_98_5_maint/examples/misc/log.py:6581,6585,6587,6589-6609,6614,6616,6625,6652,6660-6662,6672-6673,6714-6715,6717-6732,6752-6754,6761-6773,6781,6792,6800,6802,6805,6809,6811,6822,6827,6850,6854,6856,6859,6861-6873,6883-6884,6886,6890-6891,6906-6909,6911-6912,6915-6916,6918,6920-6925,6927-6928,6934,6941,6946,6948,6950,6952,6960,6972,6984-6985,6990,6995,6997-7001,7014,7016,7018,7024-7025,7033,7035,7042,7072,7080
> /branches/v0_99_maint/examples/misc/multiprocess.py:7338,7393,7395-7404,7407-7424,7428-7433,7442-7444,7446,7475-7482,7484,7486,7489-7523,7567,7569,7582-7584,7616-7618,7633,7638,7703,7727-7734,7740-7741,7745,7751,7756,7762,7770,7772,7774,7776-7778,7780,7784,7788,7790,7792,7794
> + /branches/v0_91_maint/examples/misc/log.py:5753-5771
> /branches/v0_98_5_maint/examples/misc/log.py:6581,6585,6587,6589-6609,6614,6616,6625,6652,6660-6662,6672-6673,6714-6715,6717-6732,6752-6754,6761-6773,6781,6792,6800,6802,6805,6809,6811,6822,6827,6850,6854,6856,6859,6861-6873,6883-6884,6886,6890-6891,6906-6909,6911-6912,6915-6916,6918,6920-6925,6927-6928,6934,6941,6946,6948,6950,6952,6960,6972,6984-6985,6990,6995,6997-7001,7014,7016,7018,7024-7025,7033,7035,7042,7072,7080
> /branches/v0_99_maint/examples/misc/multiprocess.py:7338,7393,7395-7404,7407-7424,7428-7433,7442-7444,7446,7475-7482,7484,7486,7489-7523,7567,7569,7582-7584,7616-7618,7633,7638,7703,7727-7734,7740-7741,7745,7751,7756,7762,7770,7772,7774,7776-7778,7780,7784,7788,7790,7792,7794,7796
>
> home:~/mpl> examples/misc/multiprocess.py
>
> Other examples that are frequently subject to properties changes are
>
> U doc/sphinxext/gen_gallery.py
> U doc/sphinxext/gen_rst.py
> U examples/mplot3d/contourf3d_demo.py
> U examples/mplot3d/scatter3d_demo.py
> U examples/mplot3d/polys3d_demo.py
> U examples/mplot3d/wire3d_demo.py
> U examples/mplot3d/surface3d_demo.py
> U examples/mplot3d/contour3d_demo.py
>
> Any idea what is special about these files that is causing them to get
> frequent property changes even when they are not modified apparently?
>
> It is a minor nuisance, but it makes it harder to see what has
> actually changed after a merge.
>
> JDH
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry® Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9-12, 2009. Register now!
> http://p.sf.net/sfu/devconf
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 

On Sun, Sep 20, 2009 at 6:50 PM, Drain, Theodore R (343P)
<the...@jp...> wrote:
> I've run into this problem quite a few times and I'd love to figure out some way to fix it. As an example, here's the kind of scenario this occurs in:
>
> I embed MPL in a few different GUI's that plot data either in real-time or via the user selecting things. There is a saved state which contains preferences like auto-scaling, legend on/off, axis formatting, etc. When the app starts up, I need to create a plot to put on the screen and configure it. What I'd like to do is this:
>
> - create widget
> - apply format (date formatter, etc)
> - apply settings (autoscale, etc)
> - wait for data (either via real time feed or user clicking on things)
>
> But this is impossible because of this kind of bug. Instead, I have to create a plot with a fake date range and test every operation to see if it's actually setting data before applying the settings like autoscale. In addition, if the user removes data from the plot (via menu or selectable lists), I have to either start over or "unset" the settings back to something safe so this error won't occur. It really makes coding something like this a royal pain.
>
> I don't have a suggestion as of yet... Perhaps it could just return "N/A" or something like that.
>
> I think part of the problem might be the default ranges used by the autoscaling algorithm when there is no data are invalid for certain formatters and locators. That suggests that possible solutions might be one of:
>
> 1) require autoscaling or scaling algorithms to return ranges that will be OK for known scalers/formatters. Perhaps some system that allows different autoscaling algorithms to be set which can configure the default?
> 2) require scalers/formatters to be robust for any range or engineer the system to allow them to report "errors" in a way that allows the plot do something reasonable and not trigger an exception (perhaps some changeable behavior w/ the default as an exception?).
>
> I'll think about this a little this week and see if any other ideas come to mind.
I think we have this problem mostly licked. The problem I was writing
about in my email is a 2nd tier problem. For example, in svn HEAD,
you can specify an "empty" date plot as long as you inform mpl of you
intentions.. From the test_date_empty unit test::
 fig = plt.figure()
 ax = fig.add_subplot(1,1,1)
 ax.xaxis_date()
 fig.savefig('date_empty')
Here we are fine, because we call ax.axaxis_date, which informs mpl
that you intend to pass in datetime instances. The key piece which
makes this possible, which you allude to in your post, is the default
xlimits, which is new in svn HEAD. In particular, the default
converter provides an AxisInfo which now supports an optional
attribute
 default_limits: the default min, max of the axis if no data is present
which is overridden in the DateConverter:
 def axisinfo(unit, axis):
 'return the unit AxisInfo'
 # make sure that the axis does not start at 0
 majloc = AutoDateLocator(tz=unit)
 majfmt = AutoDateFormatter(majloc, tz=unit)
 datemin = datetime.date(2000, 1, 1)
 datemax = datetime.date(2010, 1, 1)
 return units.AxisInfo( majloc=majloc, majfmt=majfmt, label='',
 default_limits=(datemin, datemax))
while the min/max are arbitrary, the important thing is that custom
types can now handle the default min/max limits, so when you present a
new type to mpl, the type can request a certain default view/data lim
if no data are presented. Additionally, because of the
"ignore"setting on the limts argument, we can detect whether the
limits we are applying are defaults or actively set by the user.
The complication that motivated the sf bug
http://sourceforge.net/tracker/?func=detail&aid=2861426&group_id=80706&atid=560720
is a bit more subtle. Here no data type is presented to mpl -- either
through "plot" or "fill" or "set_xlim" or whatever. If the user had
passed any data in, or manually expressed their intent through
"ax.xaxis_date" we would be fine. The difficulty is that they passed
no data in but declared their intention to use a "YearFormatter". My
original inclination, and the one that failed the unit tests, was to
trigger a call to Axis.axis_date (a new method) on a call to
ax.xaxis.set_major_formatter (or locator) where the argument was a
DateLocator or DateFormatter. This seemed to be an imminently
reasonable and helpful thing to do -- if they want a date locator or
formatter presumably they will be passing in dates -- but the unit
tests told me this was wrong.
The locators and formatters work on *converted* units. The
EpochConverter and DateConverter both convert their native types to
floating point days since 0000-00-00. So here are two custom
converter interfaces which both end up with the same floating point
representation. The conclusion is: mpl cannot use the
locator/formatter type to infer what the basic type that users will be
passing in. Just because two classes end up with the same floating
point representation does not indicate that they want the same
conversion pipeline from type -> float.
Nonetheless, we can, and already do in svn HEAD, handle the cases that
I think you are worried about in this email. As long as you know what
type you will be passing into mpl (regardless of whether you have any
data available right now) you can inform the units interface with
 ax.xaxis.update_units(someval)
where someval is an instance of the type you plan to pass in. Doing
so will not affect your current data or view limits, but will trigger
the conversion interface and importantly will trigger the
units.AxisInfo.default_limits scaling which was recently added to
avoid the kinds of problems we have been seeing with date conversion
when no data is passed in.
So despite this long winded email, the current infrastructure should support
 * create axes, etc
 * set your current formatter/locator
 * ax.xaxis.update_units(myInstance)
where myInstance is an object of the type you expect to pass in. As
long as you have registered a converter from type(myInstance) ->
ConversionInterface, you can now specify the default limits through
the ConversionInterface.default_limits method::
 @staticmethod
 def default_units(x, axis):
 'return the default unit for x or None for the given axis'
 return None
As an example in matplotlib.dates, we choose an arbitrary interval,
which while arbitrary avoids the 0..1 problem we have been having::
class DateConverter(units.ConversionInterface):
 """The units are equivalent to the timezone."""
 @staticmethod
 def axisinfo(unit, axis):
 'return the unit AxisInfo'
 # make sure that the axis does not start at 0
 majloc = AutoDateLocator(tz=unit)
 majfmt = AutoDateFormatter(majloc, tz=unit)
 datemin = datetime.date(2000, 1, 1)
 datemax = datetime.date(2010, 1, 1)
 return units.AxisInfo( majloc=majloc, majfmt=majfmt, label='',
 default_limits=(datemin, datemax))
JDH