matplotlib-devel Mailing List for matplotlib

John Hunter wrote:
[...]
> 
> The code is trying to add a non-unitized quantity (eg an errorbar
> width but just guessing) of int type with a unitized quantity
> TaggedValue (this is from the mockup basic_units testing package).
> You'd have to dig a little bit to find out where the non-unitized
> quantity is entering. errorbar is a complex example that was once
> (and I think still is) working on the 91 branch. You may want to
> compare the errorbar function on the branch vs the trunk and see what
> new feature and code change broke the units support. Again, it might
> be cleaner to have an ErrorbarItem that stores the errorbar function
> inputs and updates artist primitives on unit change rather than try
> and propagate the original unitized data down to the artist layer. As
> Eric suggested, doing these one at a time is probably a good idea, and
> errorbar is a good test case because it is so damned hairy :-)
One of the reasons for doing all the conversions at the higher level 
than the primitive artist is that often one *has* to do the conversion 
at that higher level in order to do the calculations required to draw 
the item; so a consequence of putting the conversion in the primitive 
artists is that the conversion facilities have to live at *both* levels, 
which makes the code harder to understand and maintain. The only 
penalty in taking the conversion out of the primitive artists is that a 
user who wants to support units in a custom plot type, using primitive 
artists, must include the unit conversion etc. I don't think this is a 
big problem for new code, though, because if the conversion is at that 
higher level only, then it is easy to show how to do it (there will be 
plenty of examples), and to ensure that there are enough helper 
functions to make it easy to code. Maybe there already are. Or maybe 
deriving from a PlotItem base class would make it easier. (Or maybe 
this is a place where decorators would simplify things? Just a random 
idea, not thought out.)
Eric

John Hunter wrote:
> The code is trying to add a non-unitized quantity (eg an errorbar
> width but just guessing) of int type with a unitized quantity
> TaggedValue (this is from the mockup basic_units testing package).
> You'd have to dig a little bit to find out where the non-unitized
> quantity is entering. errorbar is a complex example that was once
> (and I think still is) working on the 91 branch. You may want to
> compare the errorbar function on the branch vs the trunk and see what
> new feature and code change broke the units support. Again, it might
> be cleaner to have an ErrorbarItem that stores the errorbar function
> inputs and updates artist primitives on unit change rather than try
> and propagate the original unitized data down to the artist layer. As
> Eric suggested, doing these one at a time is probably a good idea, and
> errorbar is a good test case because it is so damned hairy :-)
Ok, so what I'm taking from your responses is that it's not a waste of time to
fix these, but that it is likely more involved than something I can do when I
have only a short time to hack. I'll file these away (though if anyone else
feels motivated, feel free!) :)
Ryan
-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Michael Droettboom wrote:
> Andrew Hawryluk wrote:
> 
>> I’m really excited about the new path simplification option for vector 
>> output formats. I tried it the first time yesterday and reduced a PDF 
>> from 231 kB to 47 kB. Thanks very much for providing this feature!
>>
>> However, I have noticed that the simplified paths often look more 
>> jagged than the original, at least for my data. I can recreate the 
>> effect with the following:
>>
>> [start]
>>
>> import numpy as np
>>
>> import matplotlib.pyplot as plt
>>
>> x = np.arange(-3,3,0.001)
>>
>> y = np.exp(-x**2) + np.random.normal(scale=0.001,size=x.size)
>>
>> plt.plot(x,y)
>>
>> plt.savefig('test.png')
>>
>> plt.savefig('test.pdf')
>>
>> [end]
>>
>> A sample output is attached, and close inspection shows that the PNG 
>> is a smooth curve with a small amount of noise while the PDF version 
>> has very noticeable changes in direction from one line segment to the 
>> next.
>>
>> <<test.png>> <<test.pdf>>
>>
>> The simplification algorithm (agg_py_path_iterator.h) does the following:
>>
>> If line2 is nearly parallel to line1, add the parallel component to 
>> the length of line1, leaving it direction unchanged
>>
>> which results in a new data point, not contained in the original data. 
>> Line1 will continue to be lengthened until it has deviated from the 
>> data curve enough that the next true data point is considered 
>> non-parallel. The cycle then continues. The result is a line that 
>> wanders around the data curve, and only the first point is guaranteed 
>> to have existed in the original data set.
>>
>> Instead, could the simplification algorithm do:
>>
>> If line2 is nearly parallel to line1, combine them by removing the 
>> common point, leaving a single line where both end points existed in 
>> the original data
>> 
I've attached a patch that will only include points from the original 
data in the simplified path. I hesitate to commit it to SVN, as these 
things are very hard to get right -- and just because it appears to work 
better on this data doesn't mean it doesn't create a regression on 
something else... ;) That said, it would be nice to confirm that this 
solution works, because it has the added benefit of being a little 
simpler computationally. Be sure to blitz your build directory when 
testing the patch -- distutils won't pick it up as a dependency.
I've attached two PDFs -- one with the original (current trunk) 
behavior, and one with the new behavior. I plotted the unsimplified 
plot in thick blue behind the simplified plot in green, so you can see 
how much deviation there is between the original data and the simplified 
line (you'll want to zoom way in with your PDF viewer to see it.)
I've also included a new version of your test script which detects "new" 
data values in the simplified path, and also seeds the random number 
generator so that results are comparable. I also set the 
solid_joinstyle to "round", as it makes the wiggliness less pronounced. 
(There was another thread on this list recently about making that the 
default setting).
Cheers,
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Fri, Jan 16, 2009 at 2:02 PM, Ryan May <rm...@gm...> wrote:
> Hi,
>
> In fixing the recursion bug in the units support, I went through the examples in
> units/ and found two broken examples (broken before I fixed the recursion bug):
>
> 1) artist_tests.py
> Traceback (most recent call last):
> File "artist_tests.py", line 30, in <module>
> lc = collections.LineCollection(verts, axes=ax)
> File
> "/home/rmay/.local/lib64/python2.5/site-packages/matplotlib/collections.py", line
> 917, in __init__
> self.set_segments(segments)
> File
> "/home/rmay/.local/lib64/python2.5/site-packages/matplotlib/collections.py", line
> 927, in set_segments
> seg = np.asarray(seg, np.float_)
> File "/home/rmay/.local/lib64/python2.5/site-packages/numpy/core/numeric.py",
> line 230, in asarray
> return array(a, dtype, copy=False, order=order)
> ValueError: setting an array element with a sequence.
The collection is trying to explicitly cast to a float when creating
the array instead of doing a conversion of the unit type first. The
set_segments method should convert to float using the
ax.convert_xunits before setting the array, and register to listen for
a unit change so that if for example the axis units are changed from
inches to cm, the segments are reset. Eg in Line2D, the set_axes
method calls:
 def set_axes(self, ax):
 Artist.set_axes(self, ax)
 if ax.xaxis is not None:
 self._xcid = ax.xaxis.callbacks.connect('units', self.recache)
 if ax.yaxis is not None:
 self._ycid = ax.yaxis.callbacks.connect('units', self.recache)
and later "recache"::
 def recache(self):
 #if self.axes is None: print 'recache no axes'
 #else: print 'recache units', self.axes.xaxis.units,
self.axes.yaxis.units
 if ma.isMaskedArray(self._xorig) or ma.isMaskedArray(self._yorig):
 x = ma.asarray(self.convert_xunits(self._xorig), float)
 y = ma.asarray(self.convert_yunits(self._yorig), float)
 x = ma.ravel(x)
 y = ma.ravel(y)
So the artist has to keep track of the original units data, and the
converted value. For simple "unit" types like datetime, this is not
so important because you convert once and you are done. For true
unitized types like basic_unit where we can switch the axis from
inches to cm, someone has to track the original unit data to convert
it to floats on a unit change. In a prior thread, I indicated I
thought the current implementation needs a rethinking, because it may
be easier for everyone concerned if the original data storage and
conversion happens at a higher layer, eg the hypothetical PlotItem
layer. As Eric pointed out, this would have the added benefit of
significantly thinning out the axes.py module code.
> 2) bar_unit_demo.py
> Traceback (most recent call last):
> File "bar_unit_demo.py", line 15, in <module>
> p1 = ax.bar(ind, menMeans, width, color='r', bottom=0*cm, yerr=menStd)
> File "/home/rmay/.local/lib64/python2.5/site-packages/matplotlib/axes.py", line
> 4134, in bar
> fmt=None, ecolor=ecolor, capsize=capsize)
> File "/home/rmay/.local/lib64/python2.5/site-packages/matplotlib/axes.py", line
> 4678, in errorbar
> in cbook.safezip(y,yerr)]
> TypeError: unsupported operand type(s) for -: 'int' and 'TaggedValue'
>
> If anyone has any quick ideas on what might have gone wrong (or if these are just
> outdated), let me know. Otherwise, I'll start digging...
The code is trying to add a non-unitized quantity (eg an errorbar
width but just guessing) of int type with a unitized quantity
TaggedValue (this is from the mockup basic_units testing package).
You'd have to dig a little bit to find out where the non-unitized
quantity is entering. errorbar is a complex example that was once
(and I think still is) working on the 91 branch. You may want to
compare the errorbar function on the branch vs the trunk and see what
new feature and code change broke the units support. Again, it might
be cleaner to have an ErrorbarItem that stores the errorbar function
inputs and updates artist primitives on unit change rather than try
and propagate the original unitized data down to the artist layer. As
Eric suggested, doing these one at a time is probably a good idea, and
errorbar is a good test case because it is so damned hairy :-)
JDH

Hi,
In fixing the recursion bug in the units support, I went through the examples in
units/ and found two broken examples (broken before I fixed the recursion bug):
1) artist_tests.py
Traceback (most recent call last):
 File "artist_tests.py", line 30, in <module>
 lc = collections.LineCollection(verts, axes=ax)
 File
"/home/rmay/.local/lib64/python2.5/site-packages/matplotlib/collections.py", line
917, in __init__
 self.set_segments(segments)
 File
"/home/rmay/.local/lib64/python2.5/site-packages/matplotlib/collections.py", line
927, in set_segments
 seg = np.asarray(seg, np.float_)
 File "/home/rmay/.local/lib64/python2.5/site-packages/numpy/core/numeric.py",
line 230, in asarray
 return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.
2) bar_unit_demo.py
Traceback (most recent call last):
 File "bar_unit_demo.py", line 15, in <module>
 p1 = ax.bar(ind, menMeans, width, color='r', bottom=0*cm, yerr=menStd)
 File "/home/rmay/.local/lib64/python2.5/site-packages/matplotlib/axes.py", line
4134, in bar
 fmt=None, ecolor=ecolor, capsize=capsize)
 File "/home/rmay/.local/lib64/python2.5/site-packages/matplotlib/axes.py", line
4678, in errorbar
 in cbook.safezip(y,yerr)]
TypeError: unsupported operand type(s) for -: 'int' and 'TaggedValue'
If anyone has any quick ideas on what might have gone wrong (or if these are just
outdated), let me know. Otherwise, I'll start digging...
Ryan
-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Hmm, I tried "svnmerge.py avail" from the branch after committing to the
trunk. I see now that I should have committed to the branch first (which
seems an inversion to me). Duly noted for the future, though.
Still working on seamless git-svn and svnmerge.py integration,
Andrew
John Hunter wrote:
> On Fri, Jan 16, 2009 at 12:38 PM, Andrew Straw <str...@as...> wrote:
>> John Hunter wrote:
>>> Andrew, since you are the original author of the isnan port, could you
>>> patch the branch and the trunk to take care of this?
>> Done in r6791 and r6792.
>>
>> Sorry for the trouble.
>>
>> Now I just hope we don't get a problem with "long long", although now if
>> _ISOC99_SOURCE is defined, we'll preferentially use "int64_t" out of
>> <stdint.h>, so I should think this is more portable on sane platforms.
>>
>> This one of many reasons why I stick to Python...
> 
> Thanks Andrew for applying this, and Georg, I forgot to mention in my
> last post thanks especially for tracking down this nasty bug. Andrew,
> for future reference, when fixing a bug on the branch, it is best to
> svnmerge it onto the rather than committing it separately since
> subsequent merges will bring it over an confuse the commit log.
> Instructions at
> 
> http://matplotlib.sourceforge.net/devel/coding_guide.html#using-svnmerge
> 
> Thanks again!
> JDH
> 

On Fri, Jan 16, 2009 at 12:38 PM, Andrew Straw <str...@as...> wrote:
> John Hunter wrote:
>> Andrew, since you are the original author of the isnan port, could you
>> patch the branch and the trunk to take care of this?
>
> Done in r6791 and r6792.
>
> Sorry for the trouble.
>
> Now I just hope we don't get a problem with "long long", although now if
> _ISOC99_SOURCE is defined, we'll preferentially use "int64_t" out of
> <stdint.h>, so I should think this is more portable on sane platforms.
>
> This one of many reasons why I stick to Python...
Thanks Andrew for applying this, and Georg, I forgot to mention in my
last post thanks especially for tracking down this nasty bug. Andrew,
for future reference, when fixing a bug on the branch, it is best to
svnmerge it onto the rather than committing it separately since
subsequent merges will bring it over an confuse the commit log.
Instructions at
 http://matplotlib.sourceforge.net/devel/coding_guide.html#using-svnmerge
Thanks again!
JDH

John Hunter wrote:
> Andrew, since you are the original author of the isnan port, could you
> patch the branch and the trunk to take care of this?
Done in r6791 and r6792.
Sorry for the trouble.
Now I just hope we don't get a problem with "long long", although now if
_ISOC99_SOURCE is defined, we'll preferentially use "int64_t" out of
<stdint.h>, so I should think this is more portable on sane platforms.
This one of many reasons why I stick to Python...
-Andrew
> 
> JDH
> 
> On Fri, Jan 16, 2009 at 8:07 AM, George <gw...@em...> wrote:
>> Hello.
>>
>> I am terribly sorry. I was mistaken last night. I had the latest Matplotlib
>> version 0.98.5.2 and I thought the bug was fixed but it hasn't. Let me explain.
>>
>> In the file MPL_isnan.h line 26 there is a declaration:
>>
>> typedef long int MPL_Int64
>>
>> This is fine for Linux 64-bit, but NOT for Windows XP 64-bit! For Windows the
>> declaration should be:
>>
>> typedef long long MPL_Int64
>>
>> This bug has caused me a LOT of late nights and last night was one of them. The
>> declaration is correct for Linux 64-bit and I guess Matplotlib was developed on
>> Linux because of this declaration. That is also why I thought the bug was fixed
>> but this morning I realised that I was looking at the wrong console.
>>
>> So, in summary. For Matplotlib 0.98.5.2 and Numpy 1.2.1 to work without any
>> problems. This means compiling and using Numpy and Matplotlib on Windows XP
>> 64-bit using AMD 64-bit compile environment, change line 26 in the file
>> MPL_isnan.h from long int to long long.\
>>
>> I also previously suggested switching MKL and ACML etc. but with this change
>> everything is fine. One can choose any math library and it works.
>>
>> Writing a small test application using sizeof on different platforms highlights
>> the problem.
>>
>> Thanks.
>>
>> George.
>>
>>
>> _______________________________________________
>> Numpy-discussion mailing list
>> Num...@sc...
>> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>>
> 

Since I suspect this change will be a little bit of work, I just wanted 
to put my hand up and say I'm looking into it so we don't duplicate 
effort here.
I think it's a worthwhile experiment, in any case.
Mike
Andrew Hawryluk wrote:
>
> I’m really excited about the new path simplification option for vector 
> output formats. I tried it the first time yesterday and reduced a PDF 
> from 231 kB to 47 kB. Thanks very much for providing this feature!
>
> However, I have noticed that the simplified paths often look more 
> jagged than the original, at least for my data. I can recreate the 
> effect with the following:
>
> [start]
>
> import numpy as np
>
> import matplotlib.pyplot as plt
>
> x = np.arange(-3,3,0.001)
>
> y = np.exp(-x**2) + np.random.normal(scale=0.001,size=x.size)
>
> plt.plot(x,y)
>
> plt.savefig('test.png')
>
> plt.savefig('test.pdf')
>
> [end]
>
> A sample output is attached, and close inspection shows that the PNG 
> is a smooth curve with a small amount of noise while the PDF version 
> has very noticeable changes in direction from one line segment to the 
> next.
>
> <<test.png>> <<test.pdf>>
>
> The simplification algorithm (agg_py_path_iterator.h) does the following:
>
> If line2 is nearly parallel to line1, add the parallel component to 
> the length of line1, leaving it direction unchanged
>
> which results in a new data point, not contained in the original data. 
> Line1 will continue to be lengthened until it has deviated from the 
> data curve enough that the next true data point is considered 
> non-parallel. The cycle then continues. The result is a line that 
> wanders around the data curve, and only the first point is guaranteed 
> to have existed in the original data set.
>
> Instead, could the simplification algorithm do:
>
> If line2 is nearly parallel to line1, combine them by removing the 
> common point, leaving a single line where both end points existed in 
> the original data
>
> Thanks again,
>
> Andrew Hawryluk
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by:
> SourcForge Community
> SourceForge wants to tell your story.
> http://p.sf.net/sfu/sf-spreadtheword
> ------------------------------------------------------------------------
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Michiel de Hoon wrote:
> I've written a patch that fixes this bug; see
>
> https://sourceforge.net/tracker/?func=detail&atid=560722&aid=2508440&group_id=80706
>
> --Michiel
> 
Just commited your patch (SVN r6787) - thanks Michiel.
-Jeff
>
> --- On Mon, 1/12/09, Tony Yu <ts...@gm...> wrote:
>
> 
>> From: Tony Yu <ts...@gm...>
>> Subject: [matplotlib-devel] Jagged plot in macosx backend
>> To: "matplotlib development list" <mat...@li...>
>> Date: Monday, January 12, 2009, 2:59 PM
>> There appears to be a bug in the macosx backend. When I plot
>> large numbers with small variations in the value, the
>> numbers seem to be coarsely rounded off. This bug
>> doesn't appear with other backends (I tried WxAgg and
>> TkAgg). Below is a simple script showing the problem and the
>> resulting plot on the macosx backend.
>>
>> Thanks,
>> -Tony
>>
>> Mac OS X 10.5.6
>> Matplotlib svn r6779
>>
>> #~~~~~~~~
>>
>> import numpy as np
>> import matplotlib.pyplot as plt
>>
>> x = np.linspace(0, 1)
>> plt.plot(x, x + 1e6)
>> plt.show()------------------------------------------------------------------------------
>> This SF.net email is sponsored by:
>> SourcForge Community
>> SourceForge wants to tell your story.
>> http://p.sf.net/sfu/sf-spreadtheword_______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>> 
>
>
> 
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by:
> SourcForge Community
> SourceForge wants to tell your story.
> http://p.sf.net/sfu/sf-spreadtheword
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jef...@no...
325 Broadway Office : Skaggs Research Cntr 1D-113
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg

I've written a patch that fixes this bug; see
https://sourceforge.net/tracker/?func=detail&atid=560722&aid=2508440&group_id=80706
--Michiel
--- On Mon, 1/12/09, Tony Yu <ts...@gm...> wrote:
> From: Tony Yu <ts...@gm...>
> Subject: [matplotlib-devel] Jagged plot in macosx backend
> To: "matplotlib development list" <mat...@li...>
> Date: Monday, January 12, 2009, 2:59 PM
> There appears to be a bug in the macosx backend. When I plot
> large numbers with small variations in the value, the
> numbers seem to be coarsely rounded off. This bug
> doesn't appear with other backends (I tried WxAgg and
> TkAgg). Below is a simple script showing the problem and the
> resulting plot on the macosx backend.
> 
> Thanks,
> -Tony
> 
> Mac OS X 10.5.6
> Matplotlib svn r6779
> 
> #~~~~~~~~
> 
> import numpy as np
> import matplotlib.pyplot as plt
> 
> x = np.linspace(0, 1)
> plt.plot(x, x + 1e6)
> plt.show()------------------------------------------------------------------------------
> This SF.net email is sponsored by:
> SourcForge Community
> SourceForge wants to tell your story.
> http://p.sf.net/sfu/sf-spreadtheword_______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel