Hi everybody, I have a problem with LinearSegmentedColormap. In the example below (see PS), I make a colormap, and use it to plot an EllipseCollection. My plot is parameterized by a quantity that I have named "large_value". For large_value equal to 257, a blue point is obtained at (x=0.3, y=0.4). But for large_value equal to 258, it becomes black. This is because of the way LinearSegmentedColormap is working. It has a parameter N which allows to set the "number of colors": http://matplotlib.org/api/colors_api.html#matplotlib.colors.LinearSegmentedColormap It is 256 by default, so if I increase N to a greater value, the point remains blue for large_value equal to 258. Now, my real plot (not this dummy example) is such that I need N to be very large so as to obtain the right colors on my plot, although very few colors are used at the end. However, when N is too large, the plot becomes very slow, and a lot of memory is used; I think because an array is probably built with this size, although in theory there is no need to construct such a complete array. Is there an easy workaround, or have I to study and modify the matplotlib code myself? Thanks, TP PS: Here is the test code: ################## from pylab import * from matplotlib.colors import LinearSegmentedColormap from matplotlib.collections import CircleCollection ioff() large_value = 257 # blue below this value #large_value = 258 # black above this value N = 1e5 # 256 by default cdict = { 'blue': [(0.0, 0.0, 0.0), (2*1/large_value, 1, 1) , (1.0, 1.0, 1.0)] , 'green': [(0.0, 0.0, 0.0), (2*1/large_value, 0, 0) , (1.0, 1.0, 1.0)] , 'red': [(0.0, 0.0, 0.0), (2*1/large_value, 0, 0), (1.0, 1.0, 1.0)] } measures= array([[ 0.2, 0.3, 1], [ 0.3, 0.4, 2], [ 0.5, 0.6, large_value]]) cmap = LinearSegmentedColormap( "cmap foobar" , cdict # , N= N ) ) fig = figure() axes = fig.add_subplot(111) ec = CircleCollection( [80] , offsets = measures[:,:2] , transOffset = axes.transData ) ec.set_array( measures[:,2] ) ec.set_cmap( cmap ) axes.add_collection( ec ) show() ##################
On 2012年11月19日 11:42 AM, TP wrote: > Hi everybody, > > I have a problem with LinearSegmentedColormap. > In the example below (see PS), I make a colormap, and use it to plot an > EllipseCollection. My plot is parameterized by a quantity that I have named > "large_value". For large_value equal to 257, a blue point is obtained at > (x=0.3, y=0.4). But for large_value equal to 258, it becomes black. > > This is because of the way LinearSegmentedColormap is working. It has a > parameter N which allows to set the "number of colors": > > http://matplotlib.org/api/colors_api.html#matplotlib.colors.LinearSegmentedColormap > > It is 256 by default, so if I increase N to a greater value, the point remains > blue for large_value equal to 258. > > Now, my real plot (not this dummy example) is such that I need N to be very > large so as to obtain the right colors on my plot, although very few colors > are used at the end. > However, when N is too large, the plot becomes very slow, and a lot of memory > is used; I think because an array is probably built with this size, although > in theory there is no need to construct such a complete array. > > Is there an easy workaround, or have I to study and modify the matplotlib code > myself? It is not entirely clear to me what you are trying to do, but it sounds like increasing N is not the right way to do it. Three things might help you find a better way: 1) The colormap is intended to work with a norm that handles the translation from your data numbers to the 0-1.0 range used to select values from the colormap (with exceptions--see below). You can choose a non-default norm, you can write your own, or you can set the parameters (vmin, vmax) of the standard linear norm. 2) By creating a colormap and calling its set_under, set_over, and set_invalid methods, you can control the colors assigned to data values that your norm maps respectively to negative numbers, numbers greater than 1, and masked values. See http://matplotlib.org/examples/pylab_examples/contourf_demo.html for an example of using set_under and set_over. See http://matplotlib.org/examples/pylab_examples/image_masked.html for another example, and for an example of controlling the norm parameters or using an alternative norm. 3) It is also possible to index directly into the colormap if you use a norm that returns an integer data type. An example of such is the BoundaryNorm. http://matplotlib.org/examples/pylab_examples/multicolored_line.html If all you need is a single assignment of a color to a "large value", then using the set_over method will take care of it. Eric > > Thanks, > > TP > > PS: Here is the test code: > ################## > from pylab import * > from matplotlib.colors import LinearSegmentedColormap > from matplotlib.collections import CircleCollection > > ioff() > large_value = 257 # blue below this value > #large_value = 258 # black above this value > N = 1e5 # 256 by default > > cdict = { 'blue': [(0.0, 0.0, 0.0), > (2*1/large_value, 1, 1) > , (1.0, 1.0, 1.0)] > , 'green': [(0.0, 0.0, 0.0), > (2*1/large_value, 0, 0) > , (1.0, 1.0, 1.0)] > , 'red': [(0.0, 0.0, 0.0), > (2*1/large_value, 0, 0), > (1.0, 1.0, 1.0)] } > > measures= array([[ 0.2, 0.3, 1], > [ 0.3, 0.4, 2], > [ 0.5, 0.6, large_value]]) > > cmap = LinearSegmentedColormap( "cmap foobar" > , cdict > # , N= N ) > ) > > fig = figure() > axes = fig.add_subplot(111) > ec = CircleCollection( [80] > , offsets = measures[:,:2] > , transOffset = axes.transData > ) > > ec.set_array( measures[:,2] ) > ec.set_cmap( cmap ) > axes.add_collection( ec ) > > show() > ################## > > ------------------------------------------------------------------------------ > Monitor your physical, virtual and cloud infrastructure from a single > web console. Get in-depth insight into apps, servers, databases, vmware, > SAP, cloud infrastructure, etc. Download 30-day Free Trial. > Pricing starts from 795ドル for 25 servers or applications! > http://p.sf.net/sfu/zoho_dev2dev_nov > _______________________________________________ > Matplotlib-users mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users >
On Monday, November 19, 2012 13:53:21 Eric Firing wrote: > It is not entirely clear to me what you are trying to do, but it sounds > like increasing N is not the right way to do it. Three things might help > you find a better way: > > 1) The colormap is intended to work with a norm that handles the > translation from your data numbers to the 0-1.0 range used to select > values from the colormap (with exceptions--see below). You can choose a > non-default norm, you can write your own, or you can set the parameters > (vmin, vmax) of the standard linear norm. > > 2) By creating a colormap and calling its set_under, set_over, and > set_invalid methods, you can control the colors assigned to data values > that your norm maps respectively to negative numbers, numbers greater > than 1, and masked values. See > http://matplotlib.org/examples/pylab_examples/contourf_demo.html for an > example of using set_under and set_over. See > http://matplotlib.org/examples/pylab_examples/image_masked.html for > another example, and for an example of controlling the norm parameters > or using an alternative norm. > > 3) It is also possible to index directly into the colormap if you use a > norm that returns an integer data type. An example of such is the > BoundaryNorm. > http://matplotlib.org/examples/pylab_examples/multicolored_line.html > > If all you need is a single assignment of a color to a "large value", > then using the set_over method will take care of it. > > Eric Thanks for your answer. My goal is to keep the correct color, i.e. blue, for the point located at (x=0.3, y=0.4), even if there are very large values of z on the plot. As I said, increasing N is not satisfying because it leads to large amounts of memory to be used. But for the time being, this is the only solution I have found. I cannot use the set_over method to do that, because the "large value" is not the only one. Indeed, what I want to do is an imshow plot, with a colorbar containing three different linear portions: * one portion for the values of z contained between the minimum and maximum value of z in some measure points. * one portion for the values of z below the minimum z in the measure points. * one portion for the values of z above the maximum z in the measure points. My problem is that I may have very large values on the plot in the range below or above the measures z. So I have exactly the problem shown in my dummy example of the previous post: all my measures have the same color, although they should not, because I have created a colormap that should handle this situation (three different linear portions in the map). The only workaround I have found is to increase the value of N, but in my case it has to be very large, such that the plot is very slow to display, or even can ask for huge amounts of memory. Thus it seems to me that my dummy example given in the previous post covers exactly the problem encountered in my real-world imshow function. Is there a memory-efficient workaround in my dummy example (instead of increasing N)? Thanks, TP
On Thursday, November 22, 2012 23:51:08 TP wrote:
> Thus it seems to me that my dummy example given in the previous post covers
> exactly the problem encountered in my real-world imshow function.
>
> Is there a memory-efficient workaround in my dummy example (instead of
> increasing N)?
I have modified LinearSegmentedColormap so as to solve my problem. The
difference is that I do not create an huge array in my test case, but instead I
interpolate linearly in the colormap. This is a quick and dirty code that
does work in my case, but which does not deal with all cases (no management of
transparency, no discontinuity in the colormap, etc.)
#####################
from __future__ import division
from pylab import *
from matplotlib.colors import LinearSegmentedColormap
from matplotlib.collections import CircleCollection
from scipy.interpolate import interp1d
class ContinuousLinearSegmentedColormap( LinearSegmentedColormap ):
def __init__(self, name, segmentdata, gamma=1.0):
LinearSegmentedColormap.__init__( self
, name, segmentdata, gamma = gamma )
def _init(self):
self.N = len( self._segmentdata['red'] )
self._lut = np.ones((self.N, 5), np.float)
for i in range( self.N ):
self._lut[i, 0] = self._segmentdata['red'][i][0]
# 2 because I do not manage discontinuities in color
self._lut[i, 1] = self._segmentdata['red'][i][2]
self._lut[i, 2] = self._segmentdata['green'][i][2]
self._lut[i, 3] = self._segmentdata['blue'][i][2]
self._isinit = True
def __call__(self, X, alpha=None, bytes=False):
if not self._isinit: self._init()
mask_bad = None
if not cbook.iterable(X):
vtype = 'scalar'
xa = np.array([X])
else:
vtype = 'array'
xma = ma.array(X, copy=False)
mask_bad = xma.mask
xa = xma.data.copy() # Copy here to avoid side effects.
del xma
lut = self._lut.copy()
rgba = np.empty(shape=xa.shape+(4,), dtype=lut.dtype)
# We construct interpolation functions.
fred = interp1d( lut[:,0], lut[:,1])
fgreen = interp1d( lut[:,0], lut[:,2])
fblue = interp1d( lut[:,0], lut[:,3])
rgba[:,3] = 1 # alpha=1 for the time being
for i in range( xa.shape[0] ):
rgba[i,0] = fred( xa[i] )
rgba[i,1] = fgreen( xa[i] )
rgba[i,2] = fblue( xa[i] )
if vtype == 'scalar':
rgba = tuple(rgba[0,:])
return rgba
ioff()
large_value = 257 # blue above this value
large_value = 258 # black above this value
large_value = 1e8
cdict = { 'blue': [(0.0, 0.0, 0.0)
, (2*1/large_value, 1, 1)
, (1.0, 1.0, 1.0)]
, 'green': [(0.0, 0.0, 0.0)
, (2*1/large_value, 0, 0)
, (1.0, 1.0, 1.0)]
, 'red': [(0.0, 0.0, 0.0)
, (2*1/large_value, 0, 0)
, (1.0, 1.0, 1.0)] }
measures= array( [[ 0.2, 0.3, 0],
[ 0.3, 0.4, 2],
[ 0.5, 0.6, large_value]] )
cmap = ContinuousLinearSegmentedColormap( "cmap foobar"
, cdict
)
fig = figure()
axes = fig.add_subplot(111)
ec = CircleCollection( [80]
, offsets = measures[:,:2]
, transOffset = axes.transData
)
ec.set_array( measures[:,2] )
ec.set_cmap( cmap )
axes.add_collection( ec )
show()
#####################
On 2012年11月26日 11:37 AM, TP wrote:
> On Thursday, November 22, 2012 23:51:08 TP wrote:
>> Thus it seems to me that my dummy example given in the previous post covers
>> exactly the problem encountered in my real-world imshow function.
>>
>> Is there a memory-efficient workaround in my dummy example (instead of
>> increasing N)?
>
> I have modified LinearSegmentedColormap so as to solve my problem. The
> difference is that I do not create an huge array in my test case, but instead I
> interpolate linearly in the colormap. This is a quick and dirty code that
> does work in my case, but which does not deal with all cases (no management of
> transparency, no discontinuity in the colormap, etc.)
I'm glad you found a solution, but my sense is that the problem is that
you are trying to make the colormap do the work of the norm. The
colormap is just a set of discrete colors, with a linear mapping to the
0-1 scale (apart from the special under, over, and invalid values). The
norm is for mapping your data to those colors, however you like, by
mapping your data to the 0-1 range (again with possible under, over, and
invalid values). Did you consider making a custom norm instead of
modifying the colormap?
Eric
>
> #####################
> from __future__ import division
> from pylab import *
> from matplotlib.colors import LinearSegmentedColormap
> from matplotlib.collections import CircleCollection
>
> from scipy.interpolate import interp1d
>
>
> class ContinuousLinearSegmentedColormap( LinearSegmentedColormap ):
>
> def __init__(self, name, segmentdata, gamma=1.0):
>
> LinearSegmentedColormap.__init__( self
> , name, segmentdata, gamma = gamma )
>
> def _init(self):
>
> self.N = len( self._segmentdata['red'] )
> self._lut = np.ones((self.N, 5), np.float)
> for i in range( self.N ):
> self._lut[i, 0] = self._segmentdata['red'][i][0]
> # 2 because I do not manage discontinuities in color
> self._lut[i, 1] = self._segmentdata['red'][i][2]
> self._lut[i, 2] = self._segmentdata['green'][i][2]
> self._lut[i, 3] = self._segmentdata['blue'][i][2]
>
> self._isinit = True
>
>
> def __call__(self, X, alpha=None, bytes=False):
>
> if not self._isinit: self._init()
> mask_bad = None
> if not cbook.iterable(X):
> vtype = 'scalar'
> xa = np.array([X])
> else:
> vtype = 'array'
> xma = ma.array(X, copy=False)
> mask_bad = xma.mask
> xa = xma.data.copy() # Copy here to avoid side effects.
> del xma
>
> lut = self._lut.copy()
> rgba = np.empty(shape=xa.shape+(4,), dtype=lut.dtype)
>
> # We construct interpolation functions.
> fred = interp1d( lut[:,0], lut[:,1])
> fgreen = interp1d( lut[:,0], lut[:,2])
> fblue = interp1d( lut[:,0], lut[:,3])
>
> rgba[:,3] = 1 # alpha=1 for the time being
> for i in range( xa.shape[0] ):
> rgba[i,0] = fred( xa[i] )
> rgba[i,1] = fgreen( xa[i] )
> rgba[i,2] = fblue( xa[i] )
>
> if vtype == 'scalar':
> rgba = tuple(rgba[0,:])
> return rgba
>
>
> ioff()
>
>
> large_value = 257 # blue above this value
> large_value = 258 # black above this value
> large_value = 1e8
>
> cdict = { 'blue': [(0.0, 0.0, 0.0)
> , (2*1/large_value, 1, 1)
> , (1.0, 1.0, 1.0)]
> , 'green': [(0.0, 0.0, 0.0)
> , (2*1/large_value, 0, 0)
> , (1.0, 1.0, 1.0)]
> , 'red': [(0.0, 0.0, 0.0)
> , (2*1/large_value, 0, 0)
> , (1.0, 1.0, 1.0)] }
>
> measures= array( [[ 0.2, 0.3, 0],
> [ 0.3, 0.4, 2],
> [ 0.5, 0.6, large_value]] )
>
> cmap = ContinuousLinearSegmentedColormap( "cmap foobar"
> , cdict
> )
>
> fig = figure()
> axes = fig.add_subplot(111)
> ec = CircleCollection( [80]
> , offsets = measures[:,:2]
> , transOffset = axes.transData
> )
>
> ec.set_array( measures[:,2] )
> ec.set_cmap( cmap )
> axes.add_collection( ec )
>
> show()
> #####################
>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from 795ドル for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
On Monday, November 26, 2012 12:06:40 Eric Firing wrote: > I'm glad you found a solution, but my sense is that the problem is that > you are trying to make the colormap do the work of the norm. The > colormap is just a set of discrete colors, with a linear mapping to the > 0-1 scale (apart from the special under, over, and invalid values). The > norm is for mapping your data to those colors, however you like, by > mapping your data to the 0-1 range (again with possible under, over, and > invalid values). Did you consider making a custom norm instead of > modifying the colormap? Yes, I did. The problem with the default colormap is that it has not enough colors. I have found (I may be wrong) that no norm can change this state of affair. If you are able to find a norm to make my example work, i.e. to obtain the middle point in blue when large_value is for example 1e10, I am interested. TP
On 2012年11月26日 12:18 PM, TP wrote: > On Monday, November 26, 2012 12:06:40 Eric Firing wrote: >> I'm glad you found a solution, but my sense is that the problem is that >> you are trying to make the colormap do the work of the norm. The >> colormap is just a set of discrete colors, with a linear mapping to the >> 0-1 scale (apart from the special under, over, and invalid values). The >> norm is for mapping your data to those colors, however you like, by >> mapping your data to the 0-1 range (again with possible under, over, and >> invalid values). Did you consider making a custom norm instead of >> modifying the colormap? > > Yes, I did. > The problem with the default colormap is that it has not enough colors. I have > found (I may be wrong) that no norm can change this state of affair. If you are > able to find a norm to make my example work, i.e. to obtain the middle point in > blue when large_value is for example 1e10, I am interested. But how many colors can you actually distinguish on the screen, or in a plot? My impression is that the problem is not lack of colors, but rather mapping to the color you want. There is no reason that having a value in your *data* of 1e10 has to affect how numbers in your data over a "normal" range are mapped. You are trying to illustrate the problem with an example using 3 colors, so how can the number of colors in the colormap be the fundamental limitation? Eric > > TP > > ------------------------------------------------------------------------------ > Monitor your physical, virtual and cloud infrastructure from a single > web console. Get in-depth insight into apps, servers, databases, vmware, > SAP, cloud infrastructure, etc. Download 30-day Free Trial. > Pricing starts from 795ドル for 25 servers or applications! > http://p.sf.net/sfu/zoho_dev2dev_nov > _______________________________________________ > Matplotlib-users mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users >
On Monday, November 26, 2012 14:10:31 Eric Firing wrote: > But how many colors can you actually distinguish on the screen, or in a > plot? My impression is that the problem is not lack of colors, but > rather mapping to the color you want. There is no reason that having a > value in your *data* of 1e10 has to affect how numbers in your data over > a "normal" range are mapped. > > You are trying to illustrate the problem with an example using 3 colors, > so how can the number of colors in the colormap be the fundamental > limitation? Ok, I understand. I think that my linear interpolation code has to somewhat be written in a norm instead. At some time, I have looked at examples on Matplotlib website, and at the code of pyshared/matplotlib/colors.py, but without having the "flash" to write the norm. The next time I will try to write a norm instead (I will put the code here of course). TP