matplotlib-devel Mailing List for matplotlib

Fernando Perez wrote:
> On Mon, Jul 7, 2008 at 10:28 AM, Michael Droettboom <md...@st...> wrote:
> 
>> Thank you for finding this! That is indeed the case.
>>
>> I think we need to rework the plot generation code to avoid side effects
>> by forcibly resetting state between plots. Maybe we should just fork
>> another Python process for each plot. Darren, since (I believe) you
>> wrote the initial documentation plotting code, do you have any thoughts?
>> 
>
> Isn't that unnecessarily slow? Why not have each plot simply make a
> call to load a clean state? rc_defaults() or somesuch? This little
> call could even be auto-run by the doc generation code, without
> requiring a full process restart.
> 
Agreed, and that does seem to fix this particular issue.
> Basically I think MPL should be stateless enough to tolerate working
> in a long-lived process with repeatable results. Anything else should
> be considered a bug IMHO ;) (I know rcParams is precisely a stateful
> system, but it should be trivial to obtain a known clean state always,
> with a single call).
> 
For the most part, I think that's correct. There are still some 
lingering uses where certain objects and values are cached at startup 
time based on rcParams. (For example, in backend_pdf, and some things 
mathtext used to do). But that's sort of a different issue, because 
there the problem is with some values being immutable after startup, not 
being unable to reset back to the startup state.
Cheers,
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Mon, Jul 7, 2008 at 10:28 AM, Michael Droettboom <md...@st...> wrote:
> Thank you for finding this! That is indeed the case.
>
> I think we need to rework the plot generation code to avoid side effects
> by forcibly resetting state between plots. Maybe we should just fork
> another Python process for each plot. Darren, since (I believe) you
> wrote the initial documentation plotting code, do you have any thoughts?
Isn't that unnecessarily slow? Why not have each plot simply make a
call to load a clean state? rc_defaults() or somesuch? This little
call could even be auto-run by the doc generation code, without
requiring a full process restart.
Basically I think MPL should be stateless enough to tolerate working
in a long-lived process with repeatable results. Anything else should
be considered a bug IMHO ;) (I know rcParams is precisely a stateful
system, but it should be trivial to obtain a known clean state always,
with a single call).
Cheers,
f

Ah - that makes sense. I guess I didn't catch that change in the release notes. Thanks again!
-----Original Message-----
From: Manuel Metz [mailto:mm...@as...]
Sent: 7 Jul 2008 11:49 AM
To: mat...@li...
Cc: Andrew Hawryluk
Subject: Re: [matplotlib-devel] hist doesn't work with 2D arrays
Andrew Hawryluk wrote:
> Hopefully this isn't old news for you. Since the 0.98 release, the histogram plot doesn't work properly with 2D arrays: it is quite slow and the output is wrong. Passing a flattened array produces the quick, correct output that we are accustomed to. Here is the test code I ran, and the attached image shows the output compared with the previous version.
> 
> import numpy as n
> import matplotlib.pyplot as p
> 
> a = n.random.normal(size=10000)
> a = a.reshape((100,100)) # make a 2D array of normally-distributed random numbers
> p.hist(a)
> 
> 
> Thanks for your work on matplotlib!
Hi Andrew,
 2D arrays are now treated differently. An (N,M) 2D array is 
interpreted as M data-sets with N elements each, e.g.
a = n.random.normal(size=10000)
a = a.reshape((1000,10))
is interpreted as 10 data-sets with 1000 elements each. See 
histogram_demo_extended.py in examples/pylab_examples.
To reproduce the old behaviour you should use pylab.hist(a.flat).
Manuel
> Andrew Hawryluk
> Calgary, Canada
> <<hist-comparison.png>> 
> 
> 
> ------------------------------------------------------------------------
> 
> 
> ------------------------------------------------------------------------
> 
> -------------------------------------------------------------------------
> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
> Studies have shown that voting for your favorite open source project,
> along with a healthy diet, reduces your potential for chronic lameness
> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Andrew Hawryluk wrote:
> Hopefully this isn't old news for you. Since the 0.98 release, the histogram plot doesn't work properly with 2D arrays: it is quite slow and the output is wrong. Passing a flattened array produces the quick, correct output that we are accustomed to. Here is the test code I ran, and the attached image shows the output compared with the previous version.
> 
> import numpy as n
> import matplotlib.pyplot as p
> 
> a = n.random.normal(size=10000)
> a = a.reshape((100,100)) # make a 2D array of normally-distributed random numbers
> p.hist(a)
> 
> 
> Thanks for your work on matplotlib!
Hi Andrew,
 2D arrays are now treated differently. An (N,M) 2D array is 
interpreted as M data-sets with N elements each, e.g.
a = n.random.normal(size=10000)
a = a.reshape((1000,10))
is interpreted as 10 data-sets with 1000 elements each. See 
histogram_demo_extended.py in examples/pylab_examples.
To reproduce the old behaviour you should use pylab.hist(a.flat).
Manuel
> Andrew Hawryluk
> Calgary, Canada
> <<hist-comparison.png>> 
> 
> 
> ------------------------------------------------------------------------
> 
> 
> ------------------------------------------------------------------------
> 
> -------------------------------------------------------------------------
> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
> Studies have shown that voting for your favorite open source project,
> along with a healthy diet, reduces your potential for chronic lameness
> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Thank you for finding this! That is indeed the case.
I think we need to rework the plot generation code to avoid side effects 
by forcibly resetting state between plots. Maybe we should just fork 
another Python process for each plot. Darren, since (I believe) you 
wrote the initial documentation plotting code, do you have any thoughts?
Cheers,
Mike
Tony Yu wrote:
>
> On Jul 7, 2008, at 12:19 PM, Michael Droettboom wrote:
>
>> I noticed today that the examples in the docs have the ticks on the
>> outside of the axes. You can see it here:
>>
>> http://matplotlib.sourceforge.net/doc/html/users/pyplot_tutorial.html
>>
>> I'm having a terrible time tracking down the cause -- maybe I'm just not
>> thinking clearly today. By version bisection, I determined that it
>> broke with revision 5690:
>>
>>
>> http://matplotlib.svn.sourceforge.net/viewvc/matplotlib?view=rev&revision=5690 
>>
>>
>> I've verified this twice (cleaning all build and installation remnants
>> out in between), and I'm pretty sure that's the revision where things 
>> break.
>>
>> But there's nothing in that revision that leaps out as a cause
>> (particularly since the effect is global and affects plots that weren't
>> even updated in that revision).
>
> I think this problem is actually caused by contour_demo.py which sets
>
> >>> matplotlib.rcParams['xtick.direction'] = 'out'
> >>> matplotlib.rcParams['ytick.direction'] = 'out'
>
> Every plot called after contour_demo.py will probably have ticks 
> outside of the axes. I think this problem has to do with how Python 
> does imports (I'm on shaky ground here, so my explanation may be 
> wrong). If matplotlib is already imported then a new call to `import 
> matplotlib` doesn't actually do anything. So the above change rcParams 
> never gets reinitialized to the default values.
>
> Try adding
>
> >>> matplotlib.rcParams['xtick.direction'] = 'in'
> >>> matplotlib.rcParams['ytick.direction'] = 'in'
>
> to the end of contour_demo.py.
>
>
> -Tony
>
>>
>> One other tidbit of information -- the example plots work fine when
>> called directly. This only seems to affect plots generated for the 
>> docs.
>>
>> Any ideas?
>>
>> Mike
>>
>> -- 
>> Michael Droettboom
>> Science Software Branch
>> Operations and Engineering Division
>> Space Telescope Science Institute
>> Operated by AURA for NASA
>>
>>
>> ------------------------------------------------------------------------- 
>>
>> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
>> Studies have shown that voting for your favorite open source project,
>> along with a healthy diet, reduces your potential for chronic lameness
>> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Removing all contents of the doc/matplotlibrc and 
doc/pyplots/matplotlibrc files (reverting to the defaults), or removing 
the files completely, does not fix the tick outside the axes issue... 
Removing my ~/.matplotlib/matplotlibrc does not resolve the issue. 
However, other rcparam values do change and have the expected impact, so 
it isn't just that the doc/matplotlibrc and doc/pyplot/matplotlibrc 
aren't being read.
Cheers,
Mike
Fernando Perez wrote:
> On Mon, Jul 7, 2008 at 10:14 AM, Michael Droettboom <md...@st...> wrote:
> 
>> Yes, but curiously, it doesn't change between those revisions...
>> 
>
> But it's different from *your* user rc file. That might point to the
> setting that causes the problem...
>
> f
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Jul 7, 2008, at 12:19 PM, Michael Droettboom wrote:
> I noticed today that the examples in the docs have the ticks on the
> outside of the axes. You can see it here:
>
> http://matplotlib.sourceforge.net/doc/html/users/pyplot_tutorial.html
>
> I'm having a terrible time tracking down the cause -- maybe I'm just 
> not
> thinking clearly today. By version bisection, I determined that it
> broke with revision 5690:
>
>
> http://matplotlib.svn.sourceforge.net/viewvc/matplotlib?view=rev&revision=5690
>
> I've verified this twice (cleaning all build and installation remnants
> out in between), and I'm pretty sure that's the revision where 
> things break.
>
> But there's nothing in that revision that leaps out as a cause
> (particularly since the effect is global and affects plots that 
> weren't
> even updated in that revision).
I think this problem is actually caused by contour_demo.py which sets
 >>> matplotlib.rcParams['xtick.direction'] = 'out'
 >>> matplotlib.rcParams['ytick.direction'] = 'out'
Every plot called after contour_demo.py will probably have ticks 
outside of the axes. I think this problem has to do with how Python 
does imports (I'm on shaky ground here, so my explanation may be 
wrong). If matplotlib is already imported then a new call to `import 
matplotlib` doesn't actually do anything. So the above change rcParams 
never gets reinitialized to the default values.
Try adding
 >>> matplotlib.rcParams['xtick.direction'] = 'in'
 >>> matplotlib.rcParams['ytick.direction'] = 'in'
to the end of contour_demo.py.
-Tony
>
> One other tidbit of information -- the example plots work fine when
> called directly. This only seems to affect plots generated for the 
> docs.
>
> Any ideas?
>
> Mike
>
> -- 
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
>
> -------------------------------------------------------------------------
> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
> Studies have shown that voting for your favorite open source project,
> along with a healthy diet, reduces your potential for chronic lameness
> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

On Mon, Jul 7, 2008 at 10:14 AM, Michael Droettboom <md...@st...> wrote:
> Yes, but curiously, it doesn't change between those revisions...
But it's different from *your* user rc file. That might point to the
setting that causes the problem...
f

Hopefully this isn't old news for you. Since the 0.98 release, the histogram plot doesn't work properly with 2D arrays: it is quite slow and the output is wrong. Passing a flattened array produces the quick, correct output that we are accustomed to. Here is the test code I ran, and the attached image shows the output compared with the previous version.
import numpy as n
import matplotlib.pyplot as p
a = n.random.normal(size=10000)
a = a.reshape((100,100)) # make a 2D array of normally-distributed random numbers
p.hist(a)
Thanks for your work on matplotlib!
Andrew Hawryluk
Calgary, Canada
 <<hist-comparison.png>> 

Yes, but curiously, it doesn't change between those revisions...
Mike
Fernando Perez wrote:
> On Mon, Jul 7, 2008 at 9:19 AM, Michael Droettboom <md...@st...> wrote:
>
> 
>> One other tidbit of information -- the example plots work fine when
>> called directly. This only seems to affect plots generated for the docs.
>> 
>
> Are the docs built with a custom .matplotlibrc? Maybe there's a
> different setting there that's triggering a bug...
>
> Cheers,
>
> f
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Mon, Jul 7, 2008 at 9:19 AM, Michael Droettboom <md...@st...> wrote:
> One other tidbit of information -- the example plots work fine when
> called directly. This only seems to affect plots generated for the docs.
Are the docs built with a custom .matplotlibrc? Maybe there's a
different setting there that's triggering a bug...
Cheers,
f

I noticed today that the examples in the docs have the ticks on the 
outside of the axes. You can see it here:
 http://matplotlib.sourceforge.net/doc/html/users/pyplot_tutorial.html
I'm having a terrible time tracking down the cause -- maybe I'm just not 
thinking clearly today. By version bisection, I determined that it 
broke with revision 5690:
 
http://matplotlib.svn.sourceforge.net/viewvc/matplotlib?view=rev&revision=5690
I've verified this twice (cleaning all build and installation remnants 
out in between), and I'm pretty sure that's the revision where things break.
But there's nothing in that revision that leaps out as a cause 
(particularly since the effect is global and affects plots that weren't 
even updated in that revision).
One other tidbit of information -- the example plots work fine when 
called directly. This only seems to affect plots generated for the docs.
Any ideas?
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Thanks Michael and Eric for your constructive replies and your help.
It is know clear for me why my code was luckily working with 0.91 but 
not with 0.98. This is certainly going to help me finding a robust 
solution for both cases.
Regards,
David
Michael Droettboom a écrit :
> David,
> 
> There is no code in matplotlib to autoscale shared axes, and as far as I 
> know, there never has been. Unfortunately, it's more or less a 
> requirement that you have to set the limits manually when using shared 
> axes. (All of the shared axes examples do this).
> 
> I think, 0.91 is broken also, it just happens to use the first plotted 
> line rather than the second (it truncates the red line, rather than the 
> blue one). The difference in ordering is due to the fact that sharing 
> is now always bi-directional in 0.98 and it uses dictionaries with a 
> somewhat random ordering, rather than lists. If you see different 
> behavior than my plots (attached) let me know.
> Eric Firing wrote:
>> This looks like a rather fundamental problem in the present 
>> architecture. I thought I had a solution figured out, but I was 
>> wrong, and now I am not confident that I can come up with a good 
>> solution quickly. Some consultation with, or intervention by, Mike D. 
>> may be needed--he might even see a solution in a matter of minutes. 
>> The problem is that the dataLim for each axes object is a Bbox, and in 
>> this implementation there is no way to share an x-interval or a 
>> y-interval among Bboxes. So there is a single xaxis major locator 
>> that is shared, but it is only getting its dataLim updated from one of 
>> the shared axes.
>> 
> 
> Just to clarify: this limitation is not new to the 0.98 architecture. 
> 0.91 also uses Bboxes for data limits and does not directly share the x- 
> or y-limits: they are updated using callbacks when the limits change. 
> The difference in 0.98 is that the references between axes are set up 
> (effectively) bidirectionally so there's no longer a notion of a 
> "controlling" axes and "following" axes. It was easy in 0.91 to set up 
> a situation where shared axes would only update if one, but not both, of 
> the pair were zoomed/panned, and that bug has been fixed.
> 
> Back when the 0.98 transformations were being written, John and I had a 
> long discussion about whether data limits should be Bbox-like or 
> pair-of-intervals-like, and we ultimately decided to leave things as-is 
> to avoid creating too much newness at once. IMHO, however, the real 
> problem is that the shared axes mechanism doesn't know whether the 
> limits are changing because of autoscaling (in which case the limits 
> should be unioned together), or panning/zooming, in which case the 
> limits need to be replaced. The second problem is probably necessary to 
> fix whether we use Bboxes or not.
> 
> Cheers,
> Mike
> 
> ------------------------------------------------------------------------
> 
> 
> ------------------------------------------------------------------------
> 

David,
There is no code in matplotlib to autoscale shared axes, and as far as I 
know, there never has been. Unfortunately, it's more or less a 
requirement that you have to set the limits manually when using shared 
axes. (All of the shared axes examples do this).
I think, 0.91 is broken also, it just happens to use the first plotted 
line rather than the second (it truncates the red line, rather than the 
blue one). The difference in ordering is due to the fact that sharing 
is now always bi-directional in 0.98 and it uses dictionaries with a 
somewhat random ordering, rather than lists. If you see different 
behavior than my plots (attached) let me know. 
Eric Firing wrote:
> This looks like a rather fundamental problem in the present 
> architecture. I thought I had a solution figured out, but I was wrong, 
> and now I am not confident that I can come up with a good solution 
> quickly. Some consultation with, or intervention by, Mike D. may be 
> needed--he might even see a solution in a matter of minutes. The 
> problem is that the dataLim for each axes object is a Bbox, and in this 
> implementation there is no way to share an x-interval or a y-interval 
> among Bboxes. So there is a single xaxis major locator that is shared, 
> but it is only getting its dataLim updated from one of the shared axes.
> 
Just to clarify: this limitation is not new to the 0.98 architecture. 
0.91 also uses Bboxes for data limits and does not directly share the x- 
or y-limits: they are updated using callbacks when the limits change. 
The difference in 0.98 is that the references between axes are set up 
(effectively) bidirectionally so there's no longer a notion of a 
"controlling" axes and "following" axes. It was easy in 0.91 to set up 
a situation where shared axes would only update if one, but not both, of 
the pair were zoomed/panned, and that bug has been fixed.
Back when the 0.98 transformations were being written, John and I had a 
long discussion about whether data limits should be Bbox-like or 
pair-of-intervals-like, and we ultimately decided to leave things as-is 
to avoid creating too much newness at once. IMHO, however, the real 
problem is that the shared axes mechanism doesn't know whether the 
limits are changing because of autoscaling (in which case the limits 
should be unioned together), or panning/zooming, in which case the 
limits need to be replaced. The second problem is probably necessary to 
fix whether we use Bboxes or not.
Cheers,
Mike