matplotlib-devel Mailing List for matplotlib

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:
 Fernando> Fernando Perez wrote:
 >> OK, a bit more info. It turns out that the crash happens
 >> whenever the _first_ figure window is deleted with a
 >> close(fignum) command. So in order to really block the
 >> problem, I had to create a dummy 'sentinel' window, numbered
 >> -666, for which close() is never called. It's OK to close this
 >> window via the window manager by clicking on its close button,
 >> but pylab.close() must NEVER be called on it. The current code
 >> looks like this:
 Fernando> More info, sorry about the noise. It's NOT OK to close
 Fernando> the sentinel in any way whatsoever, even via the window
 Fernando> manager. If this window is closed, through any
 Fernando> mechanism, Tk/VTK is hosed.
Just a guess,
The problem may be arising when the backend tries to quit when the
total figure count reaches zero. The relevant backend_tkagg section
is
 def destroy(self, *args):
 if Gcf.get_num_fig_managers()==0 and not matplotlib.is_interactive():
 if self.window is not None:
 self.window.quit()
 if self.window is not None:
 #print 'calling window destroy'
 self.window.destroy()
 self.window = None
Try playing with this function and see if you can deduce where the
problem is.
JDH

Fernando Perez wrote:
> OK, a bit more info. It turns out that the crash happens whenever the _first_ 
> figure window is deleted with a close(fignum) command. So in order to really 
> block the problem, I had to create a dummy 'sentinel' window, numbered -666, 
> for which close() is never called. It's OK to close this window via the 
> window manager by clicking on its close button, but pylab.close() must NEVER 
> be called on it. The current code looks like this:
More info, sorry about the noise. It's NOT OK to close the sentinel in any 
way whatsoever, even via the window manager. If this window is closed, 
through any mechanism, Tk/VTK is hosed.
This is what you get on screen if you close the very first Tk figure window, 
once MayaVi has run and made a figure:
In [8]: Generic Warning: In
/usr/local/installers/src/vtk/VTK/Rendering/vtkTkRenderWidget.cxx, line 633
A TkRenderWidget is being destroyed before it associated vtkRenderWindow is
destroyed. This is very bad and usually due to the order in which objects are
being destroyed. Always destroy the vtkRenderWindow before destroying the user
interface components.
Any attempt to use mayavi afterwards produces this traceback:
---------------------------------------------------------------------------
_tkinter.TclError Traceback (most recent call
last)
/home/sandberg/WavePropagation3D/Codes/Test/wave1DTEST.py
 269 mm.title('Non-projected D2 wrt prolates')
 270
--> 271 imv.surf(range(nnod),range(nnod),255*improc.mat2gray(D2p))
 272
 273
/usr/lib/python2.3/site-packages/mayavi/tools/imv.py in surf(x, y, z, warp,
scale, viewer, f_args, f_keyw)
 267 # do the mayavi stuff.
 268 if not viewer:
--> 269 v = mayavi.mayavi()
 270 else:
 271 v = viewer
/usr/lib/python2.3/site-packages/mayavi/Main.py in mayavi(geometry)
 1826 t = Tkinter.Toplevel (r)
 1827 t.withdraw ()
-> 1828 app = MayaViTkGUI (t, geometry)
 1829 return app
 1830
/usr/lib/python2.3/site-packages/mayavi/Main.py in __init__(self, master,
geometry)
 918 self.renwin_frame = Tkinter.Frame (master_f)
 919 self.renwin_frame.pack (side='left', fill='both', expand=1)
--> 920 self.renwin = Misc.RenderWindow.RenderWindow (self.renwin_frame)
 921 self.renwin.Render ()
 922
/usr/lib/python2.3/site-packages/mayavi/Misc/RenderWindow.py in __init__(self,
master)
 86 else:
 87 tkw = vtkRenderWidget.vtkTkRenderWidget (self.frame,
width=600,
---> 88 height=505)
 89 self.tkwidget = tkw
 90 self.tkwidget.pack (expand='true',fill='both')
/usr/local/lib/python2.3/site-packages/vtk_python/vtk/tk/vtkTkRenderWidget.py in
__init__(self, master, cnf, **kw)
 104
 105 kw['rw'] = renderWindow.GetAddressAsString("vtkRenderWindow")
--> 106 Tkinter.Widget.__init__(self, master, 'vtkTkRenderWidget', cnf,
kw)
 107
 108 self._CurrentRenderer = None
/usr/src/build/475206-i386/install/usr/lib/python2.3/lib-tk/Tkinter.py in
__init__(self, master, widgetName, cnf, kw, extra)
 1833 classes.append((k, cnf[k]))
 1834 del cnf[k]
-> 1835 self.tk.call(
 1836 (widgetName, self._w) + extra + self._options(cnf))
 1837 for k, v in classes:
TclError: invalid command name "vtkTkRenderWidget"
More interestingly, this leaves python in some very strange state. If you 
close ipyhton, instead of a system prompt you get the '>>>' python prompt, but 
you can't execute _anything_ there. Even a simple '1+1' fails, all you can do 
is quit. So basically the closing of that first matplotlib figure window is 
destroying enough in the python internals to render it completely unusable.
Regards,
f

Fernando Perez wrote:
> Hi all,
> 
> we've just run into a nasty problem with the TkAgg backend if close('all') is 
> called. In our setup, we use ipython+pylab with TkAgg because we also need 
> MayaVi to be active, and the GTK/WX backends block Tk windows. But if a call 
> is made to close('all'), matplotlib closes not only all of its own windows, 
> but also it destroys the MayaVi window in some very nasty way. The VTK 
> wrapper complains loudly about improper deletions, and afterwards running any 
> mayavi code is impossible (Tcl/Tk errors come from inside python itself).
> 
> The problem is that matplotlib should not be touching any windows that don't 
> belong to it. I quickly wrote the following wrapper code to use here to work 
> around this bug:
OK, a bit more info. It turns out that the crash happens whenever the _first_ 
figure window is deleted with a close(fignum) command. So in order to really 
block the problem, I had to create a dummy 'sentinel' window, numbered -666, 
for which close() is never called. It's OK to close this window via the 
window manager by clicking on its close button, but pylab.close() must NEVER 
be called on it. The current code looks like this:
# Temporary hack around a matplotlib figure closing bug
import matplotlib.pylab as mm
try:
 mm.all_figures
except AttributeError:
 mm.all_figures = []
# hack: sentinel to prevent pylab from destroying tk windows. NEVER make a 
-666 figure!
mm.figure(-666)
def figure(num=1,*args,**kw):
 """Wrapper around mm.figure which updates a global list of held figures."""
 if num == -666:
 raise ValueError,'-666 is an internal sentinel, do not use for your 
figures'
 mm.figure(num,*args,**kw)
 mm.all_figures.append(num)
def close(*args):
 """Close all open figures managed by our figure() wrapper."""
 if len(args)==1 and args[0]=='all':
 print 'Closing figures:',mm.all_figures
 map(mm.close,mm.all_figures)
 mm.all_figures = []
 else:
 mm.close(*args)
Regards,
f

Hi all,
we've just run into a nasty problem with the TkAgg backend if close('all') is 
called. In our setup, we use ipython+pylab with TkAgg because we also need 
MayaVi to be active, and the GTK/WX backends block Tk windows. But if a call 
is made to close('all'), matplotlib closes not only all of its own windows, 
but also it destroys the MayaVi window in some very nasty way. The VTK 
wrapper complains loudly about improper deletions, and afterwards running any 
mayavi code is impossible (Tcl/Tk errors come from inside python itself).
The problem is that matplotlib should not be touching any windows that don't 
belong to it. I quickly wrote the following wrapper code to use here to work 
around this bug:
try:
 mm.all_figures
except AttributeError:
 mm.all_figures = []
def figure(num=1):
 """Wrapper around mm.figure which updates a global list of held figures."""
 mm.figure(num)
 mm.all_figures.append(num)
def close_all():
 """Close all open figures managed by our figure() wrapper."""
 print 'Closing figures:',mm.all_figures
 map(mm.close,mm.all_figures)
 mm.all_figures = []
but it would be nice to see the TkAgg backend do the right thing.
I should also mention that when I first wrote my wrapper code, I used in 
figure() the following:
mm.all_figures.append(mm.figure(num))
to append the actual figure handles. This didn't work because the close() 
call to a handle seems to be also broken. I worked around this other bug by 
using figure numbers instead of handles.
While we're at it, I think it would be nice to extend the close() syntax to 
allow a sequence of integers or figure handles to be passed to it, so that one 
could simply say
close([1,3,5,21,101])
to only close those windows, or the same with their respective figure handles 
(nicely stored by ipython in the _NN variables).
Regards,
f

>>>>> "Sigve" == Sigve Tjora <si...@tj...> writes:
 Ted> All I really interested in is seeing the QtAgg front end in
 Ted> matplotlib as a supported component (which we'd be happy to
 Ted> help maintain) so whatever gets us there the quickest sounds
 Ted> good to me.
 Sigve> I am in the same position as you; I just want a Qt-backend
 Sigve> for Matplotlib.
Well, actually it looks like Ted wants a front end and Sigve wants a
backend. Let's hope these differences are not irreconcilable!
<wink>
JDH

Hi again Ted,
As far as I can tell, we are using the same method to connect between 
Agg and Qt. You are doing it from C++, while I am doing it with python 
only. Another difference is that I subclass QWidget instead of QLabel. 
Can you please elaborate on why have you choosen QLabel over QWidget? I 
don’t see what QLabel gives you that you don’t have in QWidget. Perhaps 
you get double buffering? I am just curious and trying to learn... I don’t 
think there should be any major speed penalties by doing it all from 
python, but I haven’t benchmarked anything yet.
Please do incorporate whatever you need from my code into your version. 
I am not the right person to decide what should go into Matplotlib, any 
solution is fine by me. I am in the same position as you; I just want a 
Qt-backend for Matplotlib.
Good luck!
Best regards,
Sigve I
Ted Drain skrev:
> We've also been working on a Qt front end. It's been going slow 
> because we've only been able to spend about 1 oerson-day per week on 
> it. But, starting this week I have someone full time on it. He's 
> gotten a basic widget working using the same organization that the GTK 
> code uses. Sigve's code has a lot more functionality in the tool bars 
> than ours right now.
>
> Here's where we're at:
> - Finished C++ routine to convert AGG -> QPixmap (in /src/_qtagg.cpp)
> - Finished low level widget to display the AGG pixmap using a widget 
> derived from QLabel (in ./lib/matplotlib/backends/backend_qtagg.py)
> - Working on a higher level window widget to display the plot and the 
> tool bar. We implemented the tool bar as a Qt tool bar which allows 
> you drag it around, detach it, etc.
>
> The guy working on it doesn't think it would take more than a day or 
> two to incorporate Sigve's code for the tool bar handling into our 
> code. We're close enough to finishing that it probably doesn't make a 
> lot of sense to try and set up a collaboration right now but I'm open 
> to that possibility if that's what people want. All I really 
> interested in is seeing the QtAgg front end in matplotlib as a 
> supported component (which we'd be happy to help maintain) so whatever 
> gets us there the quickest sounds good to me.
>
> Ted
>
> PS: here's the code we used to do the AGG->QPixmap conversion. It may 
> not be the fastest way to do it but it does work which is all we were 
> after for the first cut.
>
> PyQObject* pyDrawable = static_cast< PyQObject* >( args[0].ptr() );
> QLabel* label = static_cast< QLabel* >( pyDrawable->obj );
>
> RendererAgg* aggRenderer = static_cast< RendererAgg* >( args[1].ptr() );
>
> unsigned int width = aggRenderer->get_width();
> unsigned int height = aggRenderer->get_height();
>
> QImage image( aggRenderer->pixBuffer, width, height, 32, 0, 256,
> QImage::LittleEndian );
> QPixmap pixmap;
> pixmap.convertFromImage( image, QPixmap::Color );
>
> label->setPixmap( pixmap );

John Hunter wrote:
> Sigve> That probably leaves us with the OSX problems, though. Ahh,
> Sigve> fixed in CVS, ok then.
>
>I'm lost here -- as far as I know the endianess problem *was not*
>fixed in CVS. I did add an agg method tostring_bgra to CVS which
>appears to fix the reversed color problem. I haven't had a chance to
>test this on OSX since my powerbook is at home...
>
> 
>
Sorry about that, it came out wrong. Yes the inversed color could be 
fixed (I have not tested the CVS-version yet), the endiannes is not. You 
could try to change the endian in FigureCanvasQtAgg.draw()
 self.qimage = qt.QImage(self.stringBuffer,
 self.renderer.width,
 self.renderer.height,
 32,
 None,
 0,
 qt.QImage.IgnoreEndian,
 )
to
 self.qimage = qt.QImage(self.stringBuffer,
 self.renderer.width,
 self.renderer.height,
 32,
 None,
 0,
 qt.QImage.BigEndian,
 )
Or perhaps qt.QImage.LittleEndian if that is the way the conversion 
should go.
Good luck!
Regards,
Sigve

>>>>> "Ted" == Ted Drain <ted...@jp...> writes:
 Ted> We've also been working on a Qt front end. It's been going
 Ted> slow because we've only been able to spend about 1 oerson-day
 Ted> per week on it. But, starting this week I have someone full
 Ted> time on it. He's gotten a basic widget working using the
 Ted> same organization that the GTK code uses. Sigve's code has a
 Ted> lot more functionality in the tool bars than ours right now.
Great -- it looks like there will be a good opportunity to merge the
best features from both backends -- two functional qt backends is
definitely an improvement over what we had a week ago! Particularly
useful might be to retain the current implementation which uses string
methods to transfer the agg image to the qt canvas as a fallback for
your extension code method. That way, people who have trouble getting
the extension code version working can always fallback to something
slower that still works.
 Ted> The guy working on it doesn't think it would take more than a
 Ted> day or two to incorporate Sigve's code for the tool bar
 Ted> handling into our code. We're close enough to finishing that
 Ted> it probably doesn't make a lot of sense to try and set up a
 Ted> collaboration right now but I'm open to that possibility if
 Ted> that's what people want. All I really interested in is
 Ted> seeing the QtAgg front end in matplotlib as a supported
 Ted> component (which we'd be happy to help maintain) so whatever
 Ted> gets us there the quickest sounds good to me.
Whatever works for you and Sigve works for me. My guess is that it
will be easy to cross-pollinate from one implementation to the other.
Thanks!
JDH

Steve Chaplin skrev:
>Well done. I tried it out on Linux (Fedora 3) and it works reasonably
>well.
>
> 
>
Thats nice.
>I noticed a few things:
>It puts the toolbar at the top rather than the bottom of the window.
> 
>
Fix by changing in NavigationToolbar2QtAgg.__init:
 qt.QToolBar.__init__(self, qt.QString(""), window, qt.Qt.DockTop)
to
 qt.QToolBar.__init__(self, qt.QString(""), window, qt.Qt.DockBottom)
>The graph is smaller and the colours are different than that produced
>with GTKAgg, which is strange since they both use Agg rendering.
> 
>
Colors are fixed in CVS, i think.
>I think it would be better to have a 'backend_qt.py' file used as a base
>class (with a FigureCanvasQtAgg.draw() method that raises
>NotImplementedError), and with backend_qtagg.py overriding just those
>functions/methods which are specific to drawing using Agg. This would
>allow backend_qt.py code to be used to produce a QtCairo backend at a
>later date.
>
> 
>
I agree, but have not time to do it right now.
Regards,
Sigve

>>>>> "Sigve" == Sigve Tjora <si...@tj...> writes:
 Sigve> That probably leaves us with the OSX problems, though. Ahh,
 Sigve> fixed in CVS, ok then.
I'm lost here -- as far as I know the endianess problem *was not*
fixed in CVS. I did add an agg method tostring_bgra to CVS which
appears to fix the reversed color problem. I haven't had a chance to
test this on OSX since my powerbook is at home...
 Sigve> I think that QPixmap or better yet QCanvas would be the
 Sigve> appropriate if you are making a pure Qt-backend, but that
 Sigve> QImage is the choice for a backend with other rendering as
 Sigve> Agg or Cairo.
OK, I'll leave that to you, since I now next to nothing about qt.
 Sigve> Adding this to the end of FigureCanvasQtAgg.__init__ will
 Sigve> fix tracking:
 Sigve> self.setMouseTracking(True)
Yep, works. Easy enough.
Any thoughts about why the figure doesn't refresh during pan and zoom.
On my box (linux) the update only occurs after the mouse button is
released.
The other annoyance vis-a-vis nav toolbar I notice is the flicker when
doing the rubberbanding for zoom to rect. Any idea how to implement
double buffering to avoid the flicker?
Thanks!
JDH

John Hunter wrote:
> John> * The only explanation I can think of for why the colors
> John> are wrong is that qt has a different byte order for images
> John> than you are getting from agg (is could it be bgra instead
> John> of rgba, or that something is wrong with the endianess...)
> John> If you need a different byte ordering /pixel format, I can
> John> provide the required method in the agg backend. I note that
> John> the saved figures *do* have the right color scheme, which
> John> supports this idea.
>
>Another clue that endianess is playing a role here is that the colors
>are dramatically different on OSX (which I tested yesterday) ans linux
>which I tested today. But I think it's more than an endian problem,
>because the colors are wrong on both platforms. On linux, it looks
>like the endianess is right, but it's an rgb versus bgr thing, since
>the colors are backwards on
>
> from pylab import *
> subplot(211)
> plot([1,2,3], color='red')
>
> subplot(212)
> plot([1,2,3], color='blue')
>
> show()
>
>On OSX, it looks like the endianess is wrong because the colors are
>really off...
> 
>
This was actually wrong in windows also, read and blue had switched.
By using this draw, it is corrected.
 def draw(self):
 """
 Draw the figure using the renderer
 """
 FigureCanvasAgg.draw(self)
 self.stringBuffer = str(self.buffer_rgba())
 wrongqimage = qt.QImage(self.stringBuffer,
 self.renderer.width,
 self.renderer.height,
 32,
 None,
 0,
 qt.QImage.IgnoreEndian)
 self.qimage = wrongqimage.swapRGB()
 self.update()
That probably leaves us with the OSX problems, though. Ahh, fixed in 
CVS, ok then.
>Also, I took a quick at the qimage class docs at
>http://doc.trolltech.com/3.3/qimage.html and saw
>
> It is one of the two classes Qt provides for dealing with images, the
> other being QPixmap. QImage is designed and optimized for I/O and for
> direct pixel access/manipulation. QPixmap is designed and optimized
> for drawing. There are (slow) functions to convert between QImage and
> QPixmap: QPixmap::convertToImage() and QPixmap::convertFromImage().
>
>Does this suggest that QPixmap might be more appropriate / faster
>since you don't need to do the individual pixel manipulation provided
>by QImage (spoken as a total qt newbie, so please forgive any
>stupidities on my part...)
>
> 
>
I think that QPixmap or better yet QCanvas would be the appropriate if 
you are making a pure Qt-backend, but that QImage is the choice for a 
backend with other rendering as Agg or Cairo.
>An unrelated observation regarding event handling: the qt backend is
>not connecting to mouse motion unless a pan/zoom button is pressed and
>released, eg coords_demo.py does not work properly.
>
> 
>
Adding this to the end of FigureCanvasQtAgg.__init__ will fix tracking:
 self.setMouseTracking(True)
Regards,
Sigve

We've also been working on a Qt front end. It's been going slow because 
we've only been able to spend about 1 oerson-day per week on it. But, 
starting this week I have someone full time on it. He's gotten a basic 
widget working using the same organization that the GTK code 
uses. Sigve's code has a lot more functionality in the tool bars than 
ours right now.
Here's where we're at:
- Finished C++ routine to convert AGG -> QPixmap (in /src/_qtagg.cpp)
- Finished low level widget to display the AGG pixmap using a widget 
derived from QLabel (in ./lib/matplotlib/backends/backend_qtagg.py)
- Working on a higher level window widget to display the plot and the tool 
bar. We implemented the tool bar as a Qt tool bar which allows you drag it 
around, detach it, etc.
The guy working on it doesn't think it would take more than a day or two to 
incorporate Sigve's code for the tool bar handling into our code. We're 
close enough to finishing that it probably doesn't make a lot of sense to 
try and set up a collaboration right now but I'm open to that possibility 
if that's what people want. All I really interested in is seeing the QtAgg 
front end in matplotlib as a supported component (which we'd be happy to 
help maintain) so whatever gets us there the quickest sounds good to me.
Ted
PS: here's the code we used to do the AGG->QPixmap conversion. It may not 
be the fastest way to do it but it does work which is all we were after for 
the first cut.
PyQObject* pyDrawable = static_cast< PyQObject* >( args[0].ptr() );
QLabel* label = static_cast< QLabel* >( pyDrawable->obj );
RendererAgg* aggRenderer = static_cast< RendererAgg* >( args[1].ptr() );
unsigned int width = aggRenderer->get_width();
unsigned int height = aggRenderer->get_height();
QImage image( aggRenderer->pixBuffer, width, height, 32, 0, 256,
 QImage::LittleEndian );
QPixmap pixmap;
pixmap.convertFromImage( image, QPixmap::Color );
label->setPixmap( pixmap );
At 07:25 AM 1/31/2005, John Hunter wrote:
> >>>>> "John" == John Hunter <jdh...@ac...> writes:
>
> John> * The only explanation I can think of for why the colors
> John> are wrong is that qt has a different byte order for images
> John> than you are getting from agg (is could it be bgra instead
> John> of rgba, or that something is wrong with the endianess...)
> John> If you need a different byte ordering /pixel format, I can
> John> provide the required method in the agg backend. I note that
> John> the saved figures *do* have the right color scheme, which
> John> supports this idea.
>
>Another clue that endianess is playing a role here is that the colors
>are dramatically different on OSX (which I tested yesterday) ans linux
>which I tested today. But I think it's more than an endian problem,
>because the colors are wrong on both platforms. On linux, it looks
>like the endianess is right, but it's an rgb versus bgr thing, since
>the colors are backwards on
>
> from pylab import *
> subplot(211)
> plot([1,2,3], color='red')
>
> subplot(212)
> plot([1,2,3], color='blue')
>
> show()
>
>On OSX, it looks like the endianess is wrong because the colors are
>really off...
>
>Also, I took a quick at the qimage class docs at
>http://doc.trolltech.com/3.3/qimage.html and saw
>
> It is one of the two classes Qt provides for dealing with images, the
> other being QPixmap. QImage is designed and optimized for I/O and for
> direct pixel access/manipulation. QPixmap is designed and optimized
> for drawing. There are (slow) functions to convert between QImage and
> QPixmap: QPixmap::convertToImage() and QPixmap::convertFromImage().
>
>Does this suggest that QPixmap might be more appropriate / faster
>since you don't need to do the individual pixel manipulation provided
>by QImage (spoken as a total qt newbie, so please forgive any
>stupidities on my part...)
>
>
>An unrelated observation regarding event handling: the qt backend is
>not connecting to mouse motion unless a pan/zoom button is pressed and
>released, eg coords_demo.py does not work properly.
>
>Cheers!
>JDH
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
>Tool for open source databases. Create drag-&-drop reports. Save time
>by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
>Download a FREE copy at http://www.intelliview.com/go/osdn_nl
>_______________________________________________
>Matplotlib-devel mailing list
>Mat...@li...
>https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Ted Drain Jet Propulsion Laboratory ted...@jp... 

>>>>> "John" == John Hunter <jdh...@ac...> writes:
 John> * The only explanation I can think of for why the colors
 John> are wrong is that qt has a different byte order for images
 John> than you are getting from agg (is could it be bgra instead
 John> of rgba, or that something is wrong with the endianess...)
 John> If you need a different byte ordering /pixel format, I can
 John> provide the required method in the agg backend. I note that
 John> the saved figures *do* have the right color scheme, which
 John> supports this idea.
Another clue that endianess is playing a role here is that the colors
are dramatically different on OSX (which I tested yesterday) ans linux
which I tested today. But I think it's more than an endian problem,
because the colors are wrong on both platforms. On linux, it looks
like the endianess is right, but it's an rgb versus bgr thing, since
the colors are backwards on
 from pylab import *
 subplot(211)
 plot([1,2,3], color='red')
 subplot(212)
 plot([1,2,3], color='blue')
 show()
On OSX, it looks like the endianess is wrong because the colors are
really off...
Also, I took a quick at the qimage class docs at
http://doc.trolltech.com/3.3/qimage.html and saw
 It is one of the two classes Qt provides for dealing with images, the
 other being QPixmap. QImage is designed and optimized for I/O and for
 direct pixel access/manipulation. QPixmap is designed and optimized
 for drawing. There are (slow) functions to convert between QImage and
 QPixmap: QPixmap::convertToImage() and QPixmap::convertFromImage().
Does this suggest that QPixmap might be more appropriate / faster
since you don't need to do the individual pixel manipulation provided
by QImage (spoken as a total qt newbie, so please forgive any
stupidities on my part...)
An unrelated observation regarding event handling: the qt backend is
not connecting to mouse motion unless a pan/zoom button is pressed and
released, eg coords_demo.py does not work properly.
Cheers!
JDH

The gdk.threads_init() entry in the pygtk manual says:
http://www.pygtk.org/pygtk2reference/gdk-functions.html#function-gdk--
threads-init
"PyGTK does not release the GIL when calling a GTK or GDK function. Also
it does not acquire the GDK global lock (GGL). This means that, in
effect, Python threads can use the GIL alone to serialize access to the
GTK and GDK libraries. Of course, if there are non-Python threads
calling GTK or GDK functions the GGL must be used."
I read this to mean that Python programs with no non-Python threads,
like matplotlib, can run gtk.main in its own thread and do not need to
use gtk_protect (for gdk.threads_enter/leave GGL locking) since the
Python GIL will serialise access to GTK/GDK libraries.
That would be an easy solution to the interactive GTK problem - just
start gtk.main in its own thread with no need to alter any other code!
It sounds too simple to be true, have I missed something?
Steve

>>>>> "Sigve" == Sigve Tjora <pu...@tj...> writes:
 Sigve> Hi, I have written a basic QtAgg backend for
 Sigve> Matplot-lib. It is only tested on Windows, so some more
 Sigve> testing should be done, but it is my hope that it can some
 Sigve> day be included in the matplotlib distribution. I will test
 Sigve> on linux when I get the time, but all the code is python so
 Sigve> it should probably work ok.
Indeed -- it's a testament to QT and python that the untested code
appears to run the same on linux (per Steve's report) and OS X, which
I tested today. Good work!
 Sigve> See http://www.tjora.no/python/matplotlib/ for the code.
 Sigve> Most examples run ok, but the keypress_demo.py does not
 Sigve> work yet. Not all examples have been tested, just some
 Sigve> random double-clicking on the one that look interesting.
I think the problem here was that you had 
 def keyPressEvent(self, ev):
 self.keyPressEvent(ev.text())
 def keyReleaseEvent(self, ev): 
 self.keyReleaseEvent(ev.text())
where you meant
 def keyPressEvent(self, ev):
 self.key_press_event(ev.text())
 def keyReleaseEvent(self, ev): 
 self.key_release_event(ev.text())
I made these changes in CVS, and the keypress_demo indeed works.
 Sigve> A lot of the comments from the backend_template.py has not
 Sigve> been removed or fixed. The code is also somewhat rough
 Sigve> around the edges, but it works.
 Sigve> Is there any unittest-suite it is possible to run the
 Sigve> backend against?
The standard test suite is examples/backend_driver. You just need to
set the "backends" list to the list of backends you want to test. Eg
backends = ['QtAgg']
 Sigve> Feedback is welcome.
I agree with most of the comments Steve made. 
 * I think factoring the class into a module that handles the qt
 widgets, and have a small module that does qtagg would be a good
 thing. Steve would likely do a qtcairo backend if you did this...
 * The only explanation I can think of for why the colors are wrong
 is that qt has a different byte order for images than you are
 getting from agg (is could it be bgra instead of rgba, or that
 something is wrong with the endianess...) If you need a different
 byte ordering /pixel format, I can provide the required method in
 the agg backend. I note that the saved figures *do* have the
 right color scheme, which supports this idea.
 
 * The control key modifier is apparently not recognized -- this
 provides aspect ratio preserving zoom. I can't test any of the
 right click drag functionality right now, actually, since I'm on a
 mac w/o an external mouse plugged in **&$$ macs!
Thanks!
JDH

John Hunter a =E9crit :
> I have added some changes to CVS to support sharex and sharey kwargs
> (I thought this was a better name than twinx and twiny). I had to
> make some minor changes to axis to support sharing tick locators and
> formatters, but nothing substantial. You can now instantiate an axes
> with, eg
>=20
> subplot(212, sharex=3Dax1)
>=20
> and ditto for sharey. The view limits, transform function, and tick
> Locator and Formatter are shared. This allows you to pan and zoom on
> one axes and have the others follow, which is very nice. There is a
> new example showing how to use this example/shared_axis_demo.py.
>=20
Hi John,
I like this implementation a lot. Cool stuff !
> I was able to remove the TwinAxes class altogether and use the shared
> kwargs in its stead, which is cleaner. I preserved the "twin"
> convenience function (naming it twinx) and all it does is pass the
> proper kwargs and make the calls to tick_right.
> examples/two_scales.py is updated
>=20
you also need to call tick_left on the original axis, but that is a=20
minor correction.
> I don't think the datalim need to be shared because they are
> responsible only for autoscaling.
>
There is still a problem because autoscaling is disabled on the second=20
axes. I have an idea on how to solve this, I'll try it and send a patch=20
later in the week.
> One possibility is to leave inaxes, xdata and ydata alone which
> satisfies both problems above. And then to add a new attribute, axseq
> which is a list of (ax, xdata, ydata ) tuples to handle the
> overlapping axes case. Internally, we could use axseq so that pan/zoom
> will be handled properly in the two_scales case. The draw back here
> is that having more than one obvious way to do it may also confuse
> people down the road.
>=20
This sounds reasonable to me. I don't think it is confusing. If people=20
know what to do with multiple axes, they'll go with axseq, if not=20
inaxes, xdata and ydata will provide the most reasonable choice for them.
I'll play a little bit with this, and see if it breaks anything I use.
Cheers,
BC

Baptiste Carvello wrote:
> Hi,
>=20
> Steve Chaplin a =E9crit :
>>When thinking about using threads for interactive mode I had been
>>assuming people using the interactive mode would be using the pylab
>>interface, does anyone use the matplotlib class interface in interactiv=
e
>>mode?
>>
>=20
> I sometimes do.
Agreed.
One should also be able to execute interactively large scripts which may =
have=20
 been written to the OO interface. I know that I routinely test 10k LOC=
=20
modules from within ipython (though not matplotlib based). It would be I=
MHO=20
poor design to put all sorts of special-case warnings, where certain feat=
ures=20
of matplotlib would only be available in certain ways of using it. Speci=
al=20
cases lead to madness...
Note that I haven't followed the details close enough to vouch for any=20
particular approach. I just would like to see any solution implemented t=
o be=20
as clean and general as possible. As far as users are concerned, these=20
details should be, where possible, totally irrelevant (internal implement=
ation=20
only). As a reference, ipython tries mightily hard to make sure that _an=
y_=20
code which can be executed at a system command line as 'python foo.py'=20
produces identical behavior at an ipython prompt via 'run foo'. There ar=
e=20
special options to 'run' for modifying this, but the default attempts to=20
emulate python very faitfully, just with better tracebacks, debug support=
,=20
etc. And I consider deviations from this bevavior as bugs. It's not 100=
%=20
perfect, but it really works fairly well in most cases.
Regards,
f

Hi,
Steve Chaplin a =E9crit :
> What situation would yield an "Xlib async reply"?
> =20
using your mt module:
 >>> import mt
 >>> mt.gui_cmd('print "OK"')
 OK
 >>> import matplotlib
 >>> matplotlib.interactive(1)
 >>> import pylab
 Xlib: unexpected async reply (sequence 0x4b)!
This is on linux, I don't know what you get on other platforms, but I=20
guess it's just as bad.
> How does it break method calls of matplotlib objects? In interactive
> mode the gtk backend does not work (by itself) so there's nothing to
> break
Well, it depends on how ambitious you are :-)
> When thinking about using threads for interactive mode I had been
> assuming people using the interactive mode would be using the pylab
> interface, does anyone use the matplotlib class interface in interactiv=
e
> mode?
>=20
I sometimes do. However, I've painted the situation worse than it is. I=20
almost only use the setter/getter methods, which are not dangerous.
One annoying thing though: error_msg will hang the gui:
 >>> import mt
 >>> mt.gui_cmd('from pylab import *')
 >>> mt.gui_cmd('plot([1,2])')
 >>> mt.gui_cmd('plot([1,2],"0r")')
 *** gui hangs here, you get the prompt back, but no gui commands will=20
 work ***
> In import_gtk() there is time.sleep(1) - what is that for?
>=20
This is an attempt to avoid a race condition were a proxy function would=20
be called before the GTK thread is initialized. I don't know if this=20
would lead to a real problem. 1s wait not that bad anyway.
> Shouldn't widget creation be protected as well by wrapping things like
> new_figure_manager() and FigureCanvasGTK(). I know __init__ is wrapped
> but objects may have __new__ methods which call gdk/gtk functions befor=
e
> __init__ is called.
>=20
I didn't think of that. Most GTK object are not dangerous until they are=20
shown.
> How about if a widgets standard methods such as show(), present() etc
> are used - they are not protected (unless called from a protected
> method). I think this just applies to using the class interface in
> interactive mode, so it may be rarely encountered but might be a
> problem.
>=20
As I stated, I didn't try to protect methods inherited from GTK objects=20
(ie not defined in matplotlib). Which means people using them are on=20
their own. You have to draw the boundary somewhere, unless you want to=20
implement a full gui_thread.
> Why use gtk_protect(), exec() and GTK_TEMPLATE, when you could use:
>=20
> def gtk_protect (fun):
> def decorator (*args, **kwargs):
>
Well this was a try to set the proxy function name as the real function=20
name, in order to avoid weirdeness when reading object help. But I could=20
not set the argument spec correctly, so I ended up adding the 'proxy=20
from' line to the docstring. So I think all the GTK_TEMPLATE machinery=20
serves no purpose now.
As a final note, I have to say that the latest version of interactive.py=20
is good enough for my needs, so I'm no longer very motivated for pushing=20
this patch.
If you just need the pylab interface, you can also do it without=20
patching matplotlib. Just create a protected_pylab.py module, where you=20
would start the gui thread, define the gtk_protect function, import=20
pylab and define command=3Dgtk_protect(pylab.command) for every command i=
n=20
pylab. Then doing 'from protected_pylab import *' would give you a=20
working pylab interface, with just the error_msg problem.
Cheers,
Baptiste

Steve Chaplin wrote:
> To run a background process as a thread you would need to redirect its
> stdout somewhere (StringIO for example). The problem with using threads
> for this is that they share the stdout with the main thread, so if you
> redirect the background threads stdout you will redirect the python
> interpreters stdout too!
> You could run a background process as an actual process and not a thread
> (and Python 2.4 has a new subprocess module which unifies the previous
> solutions of os.system, os.spawn*, os.popen*, popen2.*, commands.*). It
> lets you redirect stdout, but is used for running executables, not
> calling python functions.
> 
> It has me stumped too, how can you call a python function and redirect
> its stdout independent of the python interpreters stdout?
Not as far as I know. That's why I think the only solution for the whole 
'backgrounding' problem with stdout is to put everything into a gui window (a 
la mathematica notebooks). I've been thinking about it, and the proper way to 
write it is probably with _two_ python processes. One would be the gui 
environment, and the other would be just a 'kernel' executing commands. The 
stdout of the kernel would then be connected to an object capable of putting 
the results of each command next to the input cell which generated them. I 
bet that's pretty much how Mathematica is organized, simply because it looks 
like the only reasonable way to deal with the issue.
Cheers,
f

On Fri, 2005年01月28日 at 21:36 +0100, Sigve Tjora wrote: 
> Hi,
> I have written a basic QtAgg backend for Matplot-lib. It is only tested
> on Windows, so some more testing should be done, but it is my hope that
> it can some day be included in the matplotlib distribution. I will test
> on linux when I get the time, but all the code is python so it should
> probably work ok.
Well done. I tried it out on Linux (Fedora 3) and it works reasonably
well.
I noticed a few things:
It puts the toolbar at the top rather than the bottom of the window.
The graph is smaller and the colours are different than that produced
with GTKAgg, which is strange since they both use Agg rendering.
I think it would be better to have a 'backend_qt.py' file used as a base
class (with a FigureCanvasQtAgg.draw() method that raises
NotImplementedError), and with backend_qtagg.py overriding just those
functions/methods which are specific to drawing using Agg. This would
allow backend_qt.py code to be used to produce a QtCairo backend at a
later date.
I copied the file into cvs so other people can try it out.
Regards
Steve

On Wed, 2005年01月26日 at 22:18 +0100, Baptiste Carvello wrote:
I've finally got round installing the interactive gui thread patch to
have a look at it. It worked quite well for a few simple plots I tried.
> A bug in the handling of threads_enter()/threads_leave() leads to a 
> deadlock, while a bug with a gui thread yields "Xlib async reply". 
> Choose which ugly death you prefer !
What situation would yield an "Xlib async reply"?
 
> There are several strategies:
...
> * wrap only pylab functions (draw_if_interactive etc), as you seem to 
> suggest in your second email: imho this is not acceptable because it 
> breaks method calls of matplotlib objects
How does it break method calls of matplotlib objects? In interactive
mode the gtk backend does not work (by itself) so there's nothing to
break, and in non-interactive mode the gui_cmd() function would not be
used.
When thinking about using threads for interactive mode I had been
assuming people using the interactive mode would be using the pylab
interface, does anyone use the matplotlib class interface in interactive
mode?
I have a few questions about patch itself:
In import_gtk() there is time.sleep(1) - what is that for?
Shouldn't widget creation be protected as well by wrapping things like
new_figure_manager() and FigureCanvasGTK(). I know __init__ is wrapped
but objects may have __new__ methods which call gdk/gtk functions before
__init__ is called.
How about if a widgets standard methods such as show(), present() etc
are used - they are not protected (unless called from a protected
method). I think this just applies to using the class interface in
interactive mode, so it may be rarely encountered but might be a
problem.
Why use gtk_protect(), exec() and GTK_TEMPLATE, when you could use:
def gtk_protect (fun):
 def decorator (*args, **kwargs):
	if IN_GTK or threading.currentThread() is Thread:
 return fun(*args, **kwargs)
 IN_GTK.append (fun)
 gtk.threads_enter()
 try:
 res = fun(*args, **kwargs)
 finally:
 gtk.threads_leave()
 IN_GTK.pop()
 return res 
 return decorator
Steve

On Thu, 2005年01月27日 at 17:14 -0700, Fernando Perez wrote: 
> > - runsource() and runcode() are synchronised using wait()/notify() so
> > they deal with one code object at a time. If self.code_to_run is changed
> > to a code queue (using list.append() and list.pop(0)) then the wait
> > ()/notify() could be removed and the interpreter thread would return
> > immediately it adds the code to self.code_to_run and not when the main
> > thread has executed the code.
> 
> On second thought, I don't think this is a good idea in general. The problem 
> I see is the serialization of output. Imagine you start a long running job 
> and you get the prompt back right away. If your job prints to stdout along 
> the way (as is very common with scientific codes), your input line will get 
> clobbered by this output.
> 
> If you could run in a gui window where you could associate the stdout for each 
> input 'cell' with its own output one, it would be different. This is exactly 
> how Mathematica notebooks work, they stack a queue of cells for running 
> sequentially, you get your typing prompt back immediately, but the output of 
> each command is collected in an Out[] cell next to the corresponding In[] one. 
> In a plain terminal, I don't see how we can make this work (in fact, in a 
> terminal Mathematica also serializes output for this very reason).
> 
> I have been toying with the idea of adding a way to 'background' processes 
> automatically to ipython, by creating a list of worker threads for long 
> running jobs. But the problem of handling stdout has me stumped.
> 
> I'm open to comments, though, or perhaps I'm missing something obvious here.
So removing wait()/notify() would not be such a good idea, it might be
useful if you are only running gui code, but not when the command prints
to stdout.
To run a background process as a thread you would need to redirect its
stdout somewhere (StringIO for example). The problem with using threads
for this is that they share the stdout with the main thread, so if you
redirect the background threads stdout you will redirect the python
interpreters stdout too!
You could run a background process as an actual process and not a thread
(and Python 2.4 has a new subprocess module which unifies the previous
solutions of os.system, os.spawn*, os.popen*, popen2.*, commands.*). It
lets you redirect stdout, but is used for running executables, not
calling python functions.
It has me stumped too, how can you call a python function and redirect
its stdout independent of the python interpreters stdout?
Steve

Hi,
I have written a basic QtAgg backend for Matplot-lib. It is only tested
on Windows, so some more testing should be done, but it is my hope that
it can some day be included in the matplotlib distribution. I will test
on linux when I get the time, but all the code is python so it should
probably work ok.
See http://www.tjora.no/python/matplotlib/ for the code.
Most examples run ok, but the keypress_demo.py does not work yet. Not
all examples have been tested, just some random double-clicking on the
one that look interesting.
A lot of the comments from the backend_template.py has not been removed
or fixed. The code is also somewhat rough around the edges, but it works.
Is there any unittest-suite it is possible to run the backend against?
Feedback is welcome.
Best regards,
Sigve Tjora

On Thu, 2005年01月27日 at 11:43 -0600, John Hunter wrote:
> I probably just forgot to update interactive.py last time Fernando
> sent it to me. Feel free to do update interactive.py and its
> interface and command line args (eg, pylab or not) as you see fit.
> Just update the changelog so I can include them in the next release
> notes. 
> 
> After you're done, you may want to take a look at interactive.html
> and/or \section{Interactive} of the user's guide, and make updates as
> you see fit.
I replaced examples/interactive.py with Fernando's updated version
(adding a comment that gtk.main runs in the main thread and replacing
gtk.timeout_add with gobject.timeout_add).
I looked at interactive.html.template and user_guide.tex and saw a
reference to ipython, but no mention of interactive.py, so it does not
look like they need updating. I seem to recall interactive.py being
mentioned in the docs in the past, but they must have been changed since
then.
Steve

Steve Chaplin wrote:
> On Tue, 2005年01月25日 at 22:25 -0700, Fernando Perez wrote:
> 
>>In case anyone is interested, here's a lightweight, standalone (non-ipython) 
>>GTK threaded interactive interpreter with matplotlib support. This code is 
>>also based on the same ASPN recipe, it's basically the prototype I used to 
>>understand that recipe when I was adding the matplotlib support to IPython.
> 
> Thanks, its a good example of code.InteractiveConsole, when I saw codeop in 
> the ASPN recipe I wondered why code.py was not used.
> 
> I noticed a few small points:
> - GTKInterpreter says "Run a gtk mainloop() in a separate thread.", yet
> gtk.main() runs in the main thread, and the interpreter/console runs in
> a separate thread.
You are correct.
> - gtk.timeout_add() is now deprecated in favour of gobject.timeout_add()
Feel free to fix it, I didn't know better.
> - runsource() and runcode() are synchronised using wait()/notify() so
> they deal with one code object at a time. If self.code_to_run is changed
> to a code queue (using list.append() and list.pop(0)) then the wait
> ()/notify() could be removed and the interpreter thread would return
> immediately it adds the code to self.code_to_run and not when the main
> thread has executed the code.
On second thought, I don't think this is a good idea in general. The problem 
I see is the serialization of output. Imagine you start a long running job 
and you get the prompt back right away. If your job prints to stdout along 
the way (as is very common with scientific codes), your input line will get 
clobbered by this output.
If you could run in a gui window where you could associate the stdout for each 
input 'cell' with its own output one, it would be different. This is exactly 
how Mathematica notebooks work, they stack a queue of cells for running 
sequentially, you get your typing prompt back immediately, but the output of 
each command is collected in an Out[] cell next to the corresponding In[] one. 
 In a plain terminal, I don't see how we can make this work (in fact, in a 
terminal Mathematica also serializes output for this very reason).
I have been toying with the idea of adding a way to 'background' processes 
automatically to ipython, by creating a list of worker threads for long 
running jobs. But the problem of handling stdout has me stumped.
I'm open to comments, though, or perhaps I'm missing something obvious here.
Best,
f