matplotlib-devel Mailing List for matplotlib

Charles Moad wrote:
> 	It must be some kind of race condition. I can easily recreate the error on agg 
> and non-agg wx and gtk backends (don't have qt). Here is what I do to cause the 
> error fast.
> 
> plot(rand(100)) # just to have something up
> f = get_current_fig_manager()
> f.canvas<hold down tab>
> # the key repeat will cause rapid fire on the completion
> 
> Straight Gtk periodically spits out something other than seg fault:
> "In [6]:f.canvas.Fatal Python error: PyEval_SaveThread: NULL tstate"
Ah, then it _may_ be an old readline bug. An ipython user discovered it:
http://www.scipy.net/roundup/ipython/issue30
and then it went to python's SF tracker:
http://sourceforge.net/tracker/index.php?func=detail&aid=1166660&group_id=5470&atid=105470
You may have a python version still with the bug in it.
Cheers,
f

	It must be some kind of race condition. I can easily recreate the error on agg 
and non-agg wx and gtk backends (don't have qt). Here is what I do to cause the 
error fast.
plot(rand(100)) # just to have something up
f = get_current_fig_manager()
f.canvas<hold down tab>
# the key repeat will cause rapid fire on the completion
Straight Gtk periodically spits out something other than seg fault:
"In [6]:f.canvas.Fatal Python error: PyEval_SaveThread: NULL tstate"
Hope this helps,
	Charlie
John Hunter wrote:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
> 
> Fernando> Charles Moad wrote:
> >> I can't seem to crash TkAgg, but WXAgg and GTKAgg still do.
> >> This seems to not be related to the repr issue from before.
> >> This is all mpl-cvs btw.
> 
> Fernando> OK, I'm afraid this one may require John's help to
> Fernando> debug. I don't know the mpl internals nearly well
> Fernando> enough to even begin to guess where this could be coming
> Fernando> from.
> 
> Fernando> The fact that it's a segfault and not an uncaught
> Fernando> exception means the bug is triggered inside C code.
> Fernando> <TAB> triggers dir() and getattr() calls, which have to
> Fernando> traverse the object's internal dictionary. If the
> Fernando> object is implemented in C/C++, that traversal can do
> Fernando> arbitrary things, including any number of
> Fernando> segfault-inducing manipulations.
> 
> The canvas object (eg FigureCanvasGTKAgg) is implemented in python.
> It inherits from a gtk base class and an Agg base class. It would be
> useful to test whether you get this behavior on the GTK and WX
> backends (no Agg). My guess is that this is some is caused by a
> threading problem, but can't be sure. This would explain why you see
> it with GTKAgg and WXAgg but not TkAgg.
> 
> JDH

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:
 Fernando> Charles Moad wrote:
 >> I can't seem to crash TkAgg, but WXAgg and GTKAgg still do.
 >> This seems to not be related to the repr issue from before.
 >> This is all mpl-cvs btw.
 Fernando> OK, I'm afraid this one may require John's help to
 Fernando> debug. I don't know the mpl internals nearly well
 Fernando> enough to even begin to guess where this could be coming
 Fernando> from.
 Fernando> The fact that it's a segfault and not an uncaught
 Fernando> exception means the bug is triggered inside C code.
 Fernando> <TAB> triggers dir() and getattr() calls, which have to
 Fernando> traverse the object's internal dictionary. If the
 Fernando> object is implemented in C/C++, that traversal can do
 Fernando> arbitrary things, including any number of
 Fernando> segfault-inducing manipulations.
The canvas object (eg FigureCanvasGTKAgg) is implemented in python.
It inherits from a gtk base class and an Agg base class. It would be
useful to test whether you get this behavior on the GTK and WX
backends (no Agg). My guess is that this is some is caused by a
threading problem, but can't be sure. This would explain why you see
it with GTKAgg and WXAgg but not TkAgg.
JDH

Charles Moad wrote:
> 	I can't seem to crash TkAgg, but WXAgg and GTKAgg still do. This seems to not 
> be related to the repr issue from before. This is all mpl-cvs btw.
OK, I'm afraid this one may require John's help to debug. I don't know the 
mpl internals nearly well enough to even begin to guess where this could be 
coming from.
The fact that it's a segfault and not an uncaught exception means the bug is 
triggered inside C code. <TAB> triggers dir() and getattr() calls, which have 
to traverse the object's internal dictionary. If the object is implemented in 
C/C++, that traversal can do arbitrary things, including any number of 
segfault-inducing manipulations.
One last thing: do you have Qt installed? If so, can you test with QtAgg as 
your backend? That might help John narrow which parts of the code may or may 
not be involved in this.
Sorry to be so vague...
Cheers,
f

>>>>> "Eric" == Eric Firing <ef...@ha...> writes:
 Eric> Jeff, I agree with John's comments.
 Eric> Based on a look at the code, but no testing, it looks like
 Eric> your patch is well along the way. I don't have anything
 Eric> ready to merge with it now, but I think I could work on it a
 Eric> bit this weekend, if that would help.
 Eric> One thing I would look at is the handling of the color
 Eric> spacing and the ticks and tick labelling. Allowing linear
 Eric> spacing is a good idea, but we may need more than a single
 Eric> binary kwarg: either two kwargs, or one with string
 Eric> arguments to allow more choice. Specifically, in my
 Eric> experience, it is often useful to give clevels like this:
 Eric> [-100] + frange(-5, 5) + [100]. The middle range may or may
 Eric> not be linear, but the two end points have arbitrary very
 Eric> large absolute values to ensure they will catch the extremes
One suggestions, instead of a linear_spacing kwarg, have something
neutral like "ticks". If ticks is a string like "linearq"
create the ticks accordingly. If it is a sequence, assume it is the
tick locations, eg [-100, -5, -4, ..., 4, 5, 100]. If it is
callable, call it (the latter would support matplotlib.ticker.Locator
instances as well as user functions)
 if is_string_like(ticks):
 if ticks == 'linear':
 theticks = do_something()
 elif ticks == 'log':
 theticks = do_something_else()
 else: 
 raise ValueError("ticks must be log|linear'")
 elif iterable(ticks):
 theticks = ticks
 elif callable(ticks):
 theticks = ticks()
 else:
 raise TypeError("ticks must be a string, a sequence, or callable")
 
 Eric> in the dataset. This is also the case for which Phil is
 Eric> requesting triangles at the ends, I think. In my Matlab
 Eric> code, I don't bother drawing triangles (which have a certain
 Eric> logic and appeal, but would make the colorbar code
 Eric> substantially more complicated); instead I simply make the
 Eric> first and last color patch have the length of the average of
The triangles per-se aren't too hard. Here is an example showing how
to draw them. I haven't used these before so I don't know how to set
their color, but perhaps one of you could plug this in. This would
also have to be adapted to support horizontal colorbars too
 from matplotlib.patches import Polygon
 import matplotlib.numerix as nx
 from pylab import figure, show
 fig = figure()
 ax = fig.add_subplot(111)
 im = ax.imshow(nx.mlab.rand(20,20))
 cax = fig.colorbar(im)
 l,b,w,h = cax.get_position()
 t = b+h
 r = l+w
 midx = l+w/2.
 upper = (l,t), (midx, t+w), (r,t)
 lower = (l,b), (midx, b-w), (r,b)
 pupper = Polygon(upper, fc='red', ec='k', lw=0.5, 
 transform=fig.transFigure)
 plower = Polygon(lower, fc='blue', ec='k', lw=0.5, 
 transform=fig.transFigure)
 fig.patches.append(pupper)
 fig.patches.append(plower)
 show()
 Eric> their immediate neighbors. Then I don't put a tick or label
 Eric> at the ends of the colorbar, but instead start and end at
 Eric> the second and next-to-last clev.
 Eric> Another aspect I customized for Matlab, and would like to do
 Eric> for mpl, is in selecting the size and shape of the colorbar.
 Eric> To my eye, plots look much nicer, and are just as
 Eric> functional, with colorbars smaller in both dimensions than
 Eric> the present default. Any possible change in this aspect,
 Eric> however, would be independent of your changes to support
 Eric> discrete colorbars, and would probably best be left for a
 Eric> separate patch.
Are you aware that the cax arg to colorbar lets you specify the
axes for the colorbar?
JDH

	I can't seem to crash TkAgg, but WXAgg and GTKAgg still do. This seems to not 
be related to the repr issue from before. This is all mpl-cvs btw.
- Charlie
Fernando Perez wrote:
> Charles Moad wrote:
> 
>> I still get an even more random segfault when using the ipython 
>> -pylab option. Here is about the closest thing I can do to reproduce 
>> (most of the time).
>>
>> plot(rand(100))
>> f = get_current_fig_manager()
>> f.canvas.get_width_height()
>> s = f.canvas.tostring_rgb()
>> f.canvas.<tab>Segmentation fault
>>
>> Here is more verbosity if it helps.
> 
> 
> What backend are you using? Can you try with different ones (ipython 
> -pylab adapts automatically, just change your default backend in your 
> ~/.matplotlib/matplotlibrc file). What version of matplotlib are you 
> using?
> 
> Here's what I get, using matplotlib 0.83.2 (official release):
> 
> - TkAgg, QtAgg, WXAgg: no problem, can't reproduce your crash, works 
> normally.
> 
> - GTKAgg: instant crash on my box (even before getting to where it 
> crashes for you):
> 
> In [1]: plot(rand(100))
> ---------------------------------------------------------------------------
> exceptions.SystemError Traceback (most 
> recent call last)
> 
> 
> SystemError: Objects/moduleobject.c:48: bad argument to internal function
> Segmentation fault
> 
> 
> I won't worry about this one, as it may be that I screwed something up 
> in the build, and I don't use GTK ever anyway.
> 
> But knowing what backends trigger your crash will help narrow things down.
> 
> Cheers,
> 
> f

Jeff,
I agree with John's comments.
Based on a look at the code, but no testing, it looks like your patch is 
well along the way. I don't have anything ready to merge with it now, 
but I think I could work on it a bit this weekend, if that would help.
One thing I would look at is the handling of the color spacing and the 
ticks and tick labelling. Allowing linear spacing is a good idea, but 
we may need more than a single binary kwarg: either two kwargs, or one 
with string arguments to allow more choice. Specifically, in my 
experience, it is often useful to give clevels like this: [-100] + 
frange(-5, 5) + [100]. The middle range may or may not be linear, but 
the two end points have arbitrary very large absolute values to ensure 
they will catch the extremes in the dataset. This is also the case for 
which Phil is requesting triangles at the ends, I think. In my Matlab 
code, I don't bother drawing triangles (which have a certain logic and 
appeal, but would make the colorbar code substantially more 
complicated); instead I simply make the first and last color patch have 
the length of the average of their immediate neighbors. Then I don't 
put a tick or label at the ends of the colorbar, but instead start and 
end at the second and next-to-last clev.
Another aspect I customized for Matlab, and would like to do for mpl, is 
in selecting the size and shape of the colorbar. To my eye, plots look 
much nicer, and are just as functional, with colorbars smaller in both 
dimensions than the present default. Any possible change in this 
aspect, however, would be independent of your changes to support 
discrete colorbars, and would probably best be left for a separate patch.
A Matlab example illustrating both the different treatment of 
end-intervals and the altered sizing is 
http://moli.soest.hawaii.edu/nbpalmer/np0305a/cont_a.html
In any case, I don't want to hold you up; you've made great progress, 
and the result looks useful as-is.
Eric
John Hunter wrote:
>>>>>>"Jeff" == Jeff Whitaker <js...@fa...> writes:
> 
> 
> Jeff> Here's a patch against 0.83.2 that modifies colorbar to use
> Jeff> contourf instead of imshow (so that only those colors used
> Jeff> by contourf are shown on the colorbar). It also adds a few
> Jeff> keyword options to colorbar to 1) allow for linear spacing
> Jeff> of colors on the colorbar (even if the contour levels are
> Jeff> not linear), and 2) draw lines on the colorbar between the
> Jeff> individual colors. Eric and Jim: I think you have both been
> Jeff> hacking on colorbar, perhaps we could merge your changes
> Jeff> into this patch? Otherwise, if no one has any objections,
> Jeff> I'll commit this to CVS later this week.
> 
> I haven't tested this but I did read over it and have one minor
> comment
> 
> I tend to avoid catching all exceptions as in
> 
> + try:
> + # mappable image is from contourf
> + clevs = array(mappable.levels.tolist() + [mappable.level_upper])
> + iscontourf = True
> + except:
> + # from imshow or pcolor.
> + iscontourf = False
> + norm = mappable.norm
> + if norm.vmin is None or norm.vmax is None:
> + mappable.autoscale()
> + cmin = norm.vmin
> + cmax = norm.vmax
> + clevs = linspace(cmin, cmax, cmap.N)
> 
> because they can lead to hard to track bugs. I suggest either
> catching an AttributeError or testing explicitly for ContourMappable,
> eg
> 
> if isinstance(mappable, ContourMappable):
> # do something
> elif isinstance(mappable, ScalarMappable):
> # do something else
> else:
> Raise TypeError("don't know how to handle type %s"%type(mappable))
> 
> Thanks,
> JDH