matplotlib-devel Mailing List for matplotlib

On Dec 2, 2005, at 3:43 PM, Eric Firing wrote:
> I don't understand the rationale for defining these functions, and in 
> particular the "resize" function that is causing problems. It looks 
> to me like all three numeric packages include both resize functions 
> and resize methods, with the latter differing in that they work in 
> place and require contiguous arrays. What your patch is doing is 
> removing access to the resize function, so there is no way to resize a 
> noncontiguous array.
I didn't realize such a function existed, and
was mostly following Travis' "list of necessary
changes" (in his book, Section 2.6.1). Of course
given that the right function exists from a top-
level import, that would be preferred over the
customization that I was doing.
I apologize if I gave the impression that the
patch was meant to be complete/correct/ideal.
It was mostly intended as a preliminary query
to see if there was any interest in pursuing
the specific backend numeric library approach,
since I had seen the discussion on a unified
build (which I didn't understand well enough
to implement.)
John Hunter wrote:
> A number of the symbols in numerix.mlab that Daishi originally defined
> were from various scipy (non-core) proper modules and I could not find
> these in the core. Will scipy core not be providing the equivalent of
> MLab.py?
I did originally attempt to build vs. just
scipy_core, but failed for the same reason.
I was impatient to get things working for
my own purposes (scratching my own itch
and all that), so didn't put much effort
into pursuing this aspect further - but I
should have mentioned it in the original
message; sorry about that.
I'm happy to see that the patch was useful
enough to incorporate into the tree and
improved upon.
Eric Firing wrote:
> Here is another anomaly that I needed to work around:
>
> >>> bb = scipy.array([1.1,2,2,3.3])
> >>> 1.1 in bb
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> ValueError: The truth value of an array with more than one element is 
> ambiguous. Use a.any() or a.all()
> >>> import numarray
> >>> nn = numarray.array([1.1,2.2,3.3])
> >>> 1.1 in nn
> True
>
> x in A behaves completely differently if A is a scipy ndarray than if 
> it is a numarray array. This seems to me like a bug in scipy; the 
> num* behavior is certainly easier to use, in that it preserves the 
> list-like behavior when used in a list-like context.
I hit this also. Another behavioral difference
that pops up is:
---
In [3]: if scipy.arange(10) == None:
 ...: pass
 ...:
------------------------------------------------------------------------ 
---
exceptions.ValueError Traceback (most 
recent call last)
/usr/local/python/python_2005年11月10日/<ipython console>
ValueError: The truth value of an array with more than one element is 
ambiguous. Use a.any() or a.all()
In [4]:
---
versus:
---
In [7]: if Numeric.arange(10) == None:
 ...: pass
 ...:
In [8]:
---
This was the reason for, e.g., this part of the patch:
Index: lib/matplotlib/contour.py
===================================================================
RCS file: /cvsroot/matplotlib/matplotlib/lib/matplotlib/contour.py,v
retrieving revision 1.16
diff -r1.16 contour.py
---
659c661
< if linewidths == None:
---
 > if linewidths is None:
---
(In this case I believe the new scipy approach
is less ambiguous and therefore better - the
intended semantics AFAICT in contour.py is for
'is', not '==' anyways.).
Thanks,
d

Eric Firing wrote:
I replied privately to Eric by mistake.
> John, Travis,
> 
> Now I find a genuine difference between scipy and num* that is causing a 
> problem: for a masked array, say "A", A.mask is a property in scipy but 
> a function in num*. I can work around it, but it seems like an anomaly 
> in scipy. What it means is that in matplotlib we can't use A.mask or 
> A.mask(), but must instead use the ma.getmask() function.
Paul is the author of masked arrays. He implemented the new behavior. 
I think mask as a property/attribute is right.
> 
> 
> Here is another anomaly that I needed to work around:
> 
> >>> bb = scipy.array([1.1,2,2,3.3])
> >>> 1.1 in bb
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> ValueError: The truth value of an array with more than one element is 
> ambiguous. Use a.any() or a.all()
> >>> import numarray
> >>> nn = numarray.array([1.1,2.2,3.3])
> >>> 1.1 in nn
> True
This is a bug in scipy core that has been fixed for several weeks. It 
came about as an unintentional side-effect of raising an error for 
truth-testing of arrays.
Notice:
 >>> 1.1 in scipy.array([1.1,2.2,3.3])
True
-Travis

John, Travis,
Now I find a genuine difference between scipy and num* that is causing a 
problem: for a masked array, say "A", A.mask is a property in scipy but 
a function in num*. I can work around it, but it seems like an anomaly 
in scipy. What it means is that in matplotlib we can't use A.mask or 
A.mask(), but must instead use the ma.getmask() function.
Here is another anomaly that I needed to work around:
 >>> bb = scipy.array([1.1,2,2,3.3])
 >>> 1.1 in bb
Traceback (most recent call last):
 File "<stdin>", line 1, in ?
ValueError: The truth value of an array with more than one element is 
ambiguous. Use a.any() or a.all()
 >>> import numarray
 >>> nn = numarray.array([1.1,2.2,3.3])
 >>> 1.1 in nn
True
x in A behaves completely differently if A is a scipy ndarray than if it 
is a numarray array. This seems to me like a bug in scipy; the num* 
behavior is certainly easier to use, in that it preserves the list-like 
behavior when used in a list-like context.
In any case, I have committed changes to make contour work; but 
examples/masked_demo.py still doesn't work with scipy. It generates no 
exception, but the plot is completely wrong. I can't spend any more 
time on this right now.
Eric

John,
I committed fixes to several files; looks OK now, at least for 
examples/image_masked.py.
Eric
John Hunter wrote:
>>>>>>"Eric" == Eric Firing <ef...@ha...> writes:
> 
> 
> Eric> John, It looks like you are referring to bugs in the
> Eric> Colormap.__call__ method that were introduced when I added
> Eric> support for masked values out-of-bounds values. I have
> Eric> committed a change that fixes the bugs I could find there
> Eric> (as tested with examples/image_masked.py), so that the
> Eric> colormap call produces exactly the same output with scipy as
> Eric> with Numeric or numarray--the same rgba values.
> Eric> Unfortunately, although that demo now runs with all three,
> Eric> it produces different plots. It appears that somewhere else
> Eric> in the code--I'm pretty sure it is not within colors.py--the
> Eric> floating point rgba values are getting rounded or, more
> Eric> likely, truncated, when using scipy.
> 
> Hey Eric,
> 
> Thanks for taking a look at this. The only other logical place for a
> problem to reside is in colors.normalize, since the pipleline is
> 
> rgba = cmap(normalize(X))
> 
> The relevant bit of code is here -- if you or anyone else sees an
> obvious candidate that might cause this truncation, let me know.... 
> 
> 
> class normalize:
> def __init__(self, vmin=None, vmax=None, clip = True):
> """
> Normalize a given value to the 0-1 range
> 
> If vmin or vmax is not given, they are taken from the input's
> minimum and maximum value respectively. If clip is True and
> the given value falls outside the range, the returned value
> will be 0 or 1, whichever is closer. Returns 0 if vmin==vmax.
> Works with scalars or arrays, including masked arrays. If clip
> is True, masked values on input will be set to 1 on output; if
> clip is False, the mask will be propagated to the output.
> """
> self.vmin = vmin
> self.vmax = vmax
> self.clip = clip
> 
> def __call__(self, value):
> 
> if isinstance(value, (int, float)):
> vtype = 'scalar'
> val = ma.array([value])
> else:
> vtype = 'array'
> val = ma.asarray(value)
> 
> self.autoscale(val)
> vmin, vmax = self.vmin, self.vmax
> if vmin > vmax:
> raise ValueError("minvalue must be less than or equal to maxvalue")
> elif vmin==vmax:
> return 0.*value
> else:
> if self.clip:
> val = clip(val.filled(vmax), vmin, vmax)
> result = (1.0/(vmax-vmin))*(val-vmin)
> if vtype == 'scalar':
> result = result[0]
> return result
> 
> def autoscale(self, A):
> if not self.scaled():
> if self.vmin is None: self.vmin = ma.minimum(A)
> if self.vmax is None: self.vmax = ma.maximum(A)
> 
> def scaled(self):
> 'return true if vmin and vmax set'
> return (self.vmin is not None and self.vmax is not None)
> 
> 
> JDH

John,
The problem is that a bunch of scipy support is missing. I think I have 
most of it fixed; I will do a little more, commit, and send another 
message in a few minutes.
Eric
John Hunter wrote:
>>>>>>"Eric" == Eric Firing <ef...@ha...> writes:
> 
> 
> Eric> John, It looks like you are referring to bugs in the
> Eric> Colormap.__call__ method that were introduced when I added
> Eric> support for masked values out-of-bounds values. I have
> Eric> committed a change that fixes the bugs I could find there
> Eric> (as tested with examples/image_masked.py), so that the
> Eric> colormap call produces exactly the same output with scipy as
> Eric> with Numeric or numarray--the same rgba values.
> Eric> Unfortunately, although that demo now runs with all three,
> Eric> it produces different plots. It appears that somewhere else
> Eric> in the code--I'm pretty sure it is not within colors.py--the
> Eric> floating point rgba values are getting rounded or, more
> Eric> likely, truncated, when using scipy.
> 
> Hey Eric,
> 
> Thanks for taking a look at this. The only other logical place for a
> problem to reside is in colors.normalize, since the pipleline is
> 
> rgba = cmap(normalize(X))
> 
> The relevant bit of code is here -- if you or anyone else sees an
> obvious candidate that might cause this truncation, let me know.... 
> 
> 
> class normalize:
> def __init__(self, vmin=None, vmax=None, clip = True):
> """
> Normalize a given value to the 0-1 range
> 
> If vmin or vmax is not given, they are taken from the input's
> minimum and maximum value respectively. If clip is True and
> the given value falls outside the range, the returned value
> will be 0 or 1, whichever is closer. Returns 0 if vmin==vmax.
> Works with scalars or arrays, including masked arrays. If clip
> is True, masked values on input will be set to 1 on output; if
> clip is False, the mask will be propagated to the output.
> """
> self.vmin = vmin
> self.vmax = vmax
> self.clip = clip
> 
> def __call__(self, value):
> 
> if isinstance(value, (int, float)):
> vtype = 'scalar'
> val = ma.array([value])
> else:
> vtype = 'array'
> val = ma.asarray(value)
> 
> self.autoscale(val)
> vmin, vmax = self.vmin, self.vmax
> if vmin > vmax:
> raise ValueError("minvalue must be less than or equal to maxvalue")
> elif vmin==vmax:
> return 0.*value
> else:
> if self.clip:
> val = clip(val.filled(vmax), vmin, vmax)
> result = (1.0/(vmax-vmin))*(val-vmin)
> if vtype == 'scalar':
> result = result[0]
> return result
> 
> def autoscale(self, A):
> if not self.scaled():
> if self.vmin is None: self.vmin = ma.minimum(A)
> if self.vmax is None: self.vmax = ma.maximum(A)
> 
> def scaled(self):
> 'return true if vmin and vmax set'
> return (self.vmin is not None and self.vmax is not None)
> 
> 
> JDH

>>>>> "Eric" == Eric Firing <ef...@ha...> writes:
 Eric> John, It looks like you are referring to bugs in the
 Eric> Colormap.__call__ method that were introduced when I added
 Eric> support for masked values out-of-bounds values. I have
 Eric> committed a change that fixes the bugs I could find there
 Eric> (as tested with examples/image_masked.py), so that the
 Eric> colormap call produces exactly the same output with scipy as
 Eric> with Numeric or numarray--the same rgba values.
 Eric> Unfortunately, although that demo now runs with all three,
 Eric> it produces different plots. It appears that somewhere else
 Eric> in the code--I'm pretty sure it is not within colors.py--the
 Eric> floating point rgba values are getting rounded or, more
 Eric> likely, truncated, when using scipy.
Hey Eric,
Thanks for taking a look at this. The only other logical place for a
problem to reside is in colors.normalize, since the pipleline is
 rgba = cmap(normalize(X))
The relevant bit of code is here -- if you or anyone else sees an
obvious candidate that might cause this truncation, let me know.... 
class normalize:
 def __init__(self, vmin=None, vmax=None, clip = True):
 """
 Normalize a given value to the 0-1 range
 If vmin or vmax is not given, they are taken from the input's
 minimum and maximum value respectively. If clip is True and
 the given value falls outside the range, the returned value
 will be 0 or 1, whichever is closer. Returns 0 if vmin==vmax.
 Works with scalars or arrays, including masked arrays. If clip
 is True, masked values on input will be set to 1 on output; if
 clip is False, the mask will be propagated to the output.
 """
 self.vmin = vmin
 self.vmax = vmax
 self.clip = clip
 def __call__(self, value):
 if isinstance(value, (int, float)):
 vtype = 'scalar'
 val = ma.array([value])
 else:
 vtype = 'array'
 val = ma.asarray(value)
 self.autoscale(val)
 vmin, vmax = self.vmin, self.vmax
 if vmin > vmax:
 raise ValueError("minvalue must be less than or equal to maxvalue")
 elif vmin==vmax:
 return 0.*value
 else:
 if self.clip:
 val = clip(val.filled(vmax), vmin, vmax)
 result = (1.0/(vmax-vmin))*(val-vmin)
 if vtype == 'scalar':
 result = result[0]
 return result
 def autoscale(self, A):
 if not self.scaled():
 if self.vmin is None: self.vmin = ma.minimum(A)
 if self.vmax is None: self.vmax = ma.maximum(A)
 def scaled(self):
 'return true if vmin and vmax set'
 return (self.vmin is not None and self.vmax is not None)
JDH

John,
It looks like you are referring to bugs in the Colormap.__call__ method 
that were introduced when I added support for masked values 
out-of-bounds values. I have committed a change that fixes the bugs I 
could find there (as tested with examples/image_masked.py), so that the 
colormap call produces exactly the same output with scipy as with 
Numeric or numarray--the same rgba values. Unfortunately, although that 
demo now runs with all three, it produces different plots. It appears 
that somewhere else in the code--I'm pretty sure it is not within 
colors.py--the floating point rgba values are getting rounded or, more 
likely, truncated, when using scipy.
Eric
John Hunter wrote:
> I just committed some changes to fix the scipy core patch. The
> problem was that Daishi apparently had the full scipy installed, so
> many of the imports that worked in his tests did not work for scipy
> core.
> 
> Most things seem to be working, but there is a problem in the
> colormapping code in colors.py. The index into the LUT, which should
> be an integer array, is coming out as a float array for scipy core.
> I'm not sure why. Perry wrote this section of the code so he has a
> better understanding of it than I do (around line 240 in colors.py),
> this line is failing
> 
> #print 'types', typecode(self._lut), typecode(xa), xa.shape
> rgba = take(self._lut, xa)
> 
> There may be other problems, but this is the only one I know of
> currently. 
> 
> A number of the symbols in numerix.mlab that Daishi originally defined
> were from various scipy (non-core) proper modules and I could not find
> these in the core. Will scipy core not be providing the equivalent of
> MLab.py? There are a few of these functions used in matplotlib, and I
> just redefined them manually for now in numerix.mlab. Eg, mean, std,
> hanning, and a few more. Others, like cov, I buried the import into
> the function that needs them, eg in matplotlib.mlab in the
> detrend_linear function. If these aren't going to be in the core,
> what do people suggest we do for numerix.mlab.
> 
> If anyone has any suggestions for help on the colormap LUT problem
> referenced above, that would be great.
> 
> Thanks,
> JDH
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
> for problems? Stop! Download the new AJAX search engine that makes
> searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

I just committed some changes to fix the scipy core patch. The
problem was that Daishi apparently had the full scipy installed, so
many of the imports that worked in his tests did not work for scipy
core.
Most things seem to be working, but there is a problem in the
colormapping code in colors.py. The index into the LUT, which should
be an integer array, is coming out as a float array for scipy core.
I'm not sure why. Perry wrote this section of the code so he has a
better understanding of it than I do (around line 240 in colors.py),
this line is failing
 #print 'types', typecode(self._lut), typecode(xa), xa.shape
 rgba = take(self._lut, xa)
There may be other problems, but this is the only one I know of
currently. 
A number of the symbols in numerix.mlab that Daishi originally defined
were from various scipy (non-core) proper modules and I could not find
these in the core. Will scipy core not be providing the equivalent of
MLab.py? There are a few of these functions used in matplotlib, and I
just redefined them manually for now in numerix.mlab. Eg, mean, std,
hanning, and a few more. Others, like cov, I buried the import into
the function that needs them, eg in matplotlib.mlab in the
detrend_linear function. If these aren't going to be in the core,
what do people suggest we do for numerix.mlab.
If anyone has any suggestions for help on the colormap LUT problem
referenced above, that would be great.
Thanks,
JDH