Hello all, I would like to submit the following branch on github for review and merging into matplotlib trunk: http://github.com/ellisonbg/matplotlib/commits/guisupport This branch implements the logic needed for the qt4 and wx backends to fully work with the upcoming IPython 0.11 release. In our testing we have run many of the mpl examples (including the new animation examples) in both qt4/wx in both the terminal based IPython and the new IPython Qt GUI. For background on these changes please see this thread: http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel It is important to note that we have not updated the other matplotlib backends (gtk, tk, etc.) to have this logic. This is mainly because we know almost nothing about these toolkits and could really use some help from folks who are experts at the respective toolkits. We have done some minimal testing and these other backends do work for simple examples in the terminal IPython, but they won't work in all cases and will definitely not work in the new IPython Qt based GUI. We would love feedback and help testing as these changes are significant (even though only a few lines of code). To test this stuff you will need to grab the following IPython development branch: http://github.com/ipython/ipython/tree/newkernel You should be able to run the examples in regular ipython: ipython --pylab qt4|wx Or the new GUI ipythonqt --pylab qt4|wx Cheers, Brian
On 09/03/2010 12:37 PM, Brian Granger wrote: > Hello all, > > I would like to submit the following branch on github for review and > merging into matplotlib trunk: > > http://github.com/ellisonbg/matplotlib/commits/guisupport > > This branch implements the logic needed for the qt4 and wx backends to > fully work with the upcoming IPython 0.11 release. In our testing we > have run many of the mpl examples (including the new animation > examples) in both qt4/wx in both the terminal based IPython and the > new IPython Qt GUI. For background on these changes please see this > thread: > > http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel > > It is important to note that we have not updated the other matplotlib > backends (gtk, tk, etc.) to have this logic. This is mainly because > we know almost nothing about these toolkits and could really use some > help from folks who are experts at the respective toolkits. We have > done some minimal testing and these other backends do work for simple > examples in the terminal IPython, but they won't work in all cases and > will definitely not work in the new IPython Qt based GUI. > > We would love feedback and help testing as these changes are > significant (even though only a few lines of code). To test this > stuff you will need to grab the following IPython development branch: > > http://github.com/ipython/ipython/tree/newkernel > > You should be able to run the examples in regular ipython: > > ipython --pylab qt4|wx > > Or the new GUI > > ipythonqt --pylab qt4|wx > Brian and others, It's not quite that simple. After some initial thrashing around, I installed zmq from source, and then pyzmq--but I can't import zmq: In [1]: import zmq --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /home/efiring/<ipython console> in <module>() /usr/local/lib/python2.6/dist-packages/zmq/__init__.py in <module>() 24 #----------------------------------------------------------------------------- 25 ---> 26 from zmq import _zmq 27 from zmq._zmq import * 28 ImportError: libzmq.so.0: cannot open shared object file: No such file or directory But it is right where it should be: efiring@manini:~/programs/py/ipython/pyzmq_git$ ll /usr/local/lib/libzmq* -rw-r--r-- 1 root root 4736930 2010年09月03日 14:35 /usr/local/lib/libzmq.a -rwxr-xr-x 1 root root 825 2010年09月03日 14:35 /usr/local/lib/libzmq.la lrwxrwxrwx 1 root root 15 2010年09月03日 14:35 /usr/local/lib/libzmq.so -> libzmq.so.0.0.0 lrwxrwxrwx 1 root root 15 2010年09月03日 14:35 /usr/local/lib/libzmq.so.0 -> libzmq.so.0.0.0 -rwxr-xr-x 1 root root 1993116 2010年09月03日 14:35 /usr/local/lib/libzmq.so.0.0.0 And it looks like the extension was built OK, complete with linking: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -I/usr/include/python2.6 -c zmq/_zmq.c -o build/temp.linux-x86_64-2.6/zmq/_zmq.o zmq/_zmq.c:1995: warning: ‘__pyx_doc_3zmq_4_zmq_7Message___len__’ defined but not used gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/zmq/_zmq.o -L/usr/local/lib -lzmq -o build/lib.linux-x86_64-2.6/zmq/_zmq.so I'm stumped. Eric > Cheers, > > Brian
Eric, Looks like you are on Linux. If so, you will have to make sure you do two things before doing python setup.py install on pyzmq: 1. Edit the setup.cfg (copy the template) to point to your zeromq install. 2. Use the --rpath option: python setup.py build_ext --rpath=/opt/zeromq-dev/lib --inplace Alternatively, you can set the LD_LIBRARY_PATH to point to your zeromq lib path. Also, for now, we suggest using the stable 2.0.7 releases of both pyzmq and zeromq. Sorry about the hitches, parts of this are definitely still rough and we hope to iron out these things. I should mention that zeromq/pyzmq are only required for the new Qt gui, the old terminal ipython doesn't require them. Let us know if you run into any other issues. Cheers, Brian On Fri, Sep 3, 2010 at 6:57 PM, Eric Firing <ef...@ha...> wrote: > On 09/03/2010 12:37 PM, Brian Granger wrote: >> Hello all, >> >> I would like to submit the following branch on github for review and >> merging into matplotlib trunk: >> >> http://github.com/ellisonbg/matplotlib/commits/guisupport >> >> This branch implements the logic needed for the qt4 and wx backends to >> fully work with the upcoming IPython 0.11 release. In our testing we >> have run many of the mpl examples (including the new animation >> examples) in both qt4/wx in both the terminal based IPython and the >> new IPython Qt GUI. For background on these changes please see this >> thread: >> >> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel >> >> It is important to note that we have not updated the other matplotlib >> backends (gtk, tk, etc.) to have this logic. This is mainly because >> we know almost nothing about these toolkits and could really use some >> help from folks who are experts at the respective toolkits. We have >> done some minimal testing and these other backends do work for simple >> examples in the terminal IPython, but they won't work in all cases and >> will definitely not work in the new IPython Qt based GUI. >> >> We would love feedback and help testing as these changes are >> significant (even though only a few lines of code). To test this >> stuff you will need to grab the following IPython development branch: >> >> http://github.com/ipython/ipython/tree/newkernel >> >> You should be able to run the examples in regular ipython: >> >> ipython --pylab qt4|wx >> >> Or the new GUI >> >> ipythonqt --pylab qt4|wx >> > > Brian and others, > > It's not quite that simple. After some initial thrashing around, I > installed zmq from source, and then pyzmq--but I can't import zmq: > > In [1]: import zmq > --------------------------------------------------------------------------- > ImportError Traceback (most recent call last) > /home/efiring/<ipython console> in <module>() > > /usr/local/lib/python2.6/dist-packages/zmq/__init__.py in <module>() > 24 > #----------------------------------------------------------------------------- > > 25 > ---> 26 from zmq import _zmq > 27 from zmq._zmq import * > 28 > > ImportError: libzmq.so.0: cannot open shared object file: No such file > or directory > > But it is right where it should be: > > efiring@manini:~/programs/py/ipython/pyzmq_git$ ll /usr/local/lib/libzmq* > -rw-r--r-- 1 root root 4736930 2010年09月03日 14:35 /usr/local/lib/libzmq.a > -rwxr-xr-x 1 root root 825 2010年09月03日 14:35 /usr/local/lib/libzmq.la > lrwxrwxrwx 1 root root 15 2010年09月03日 14:35 /usr/local/lib/libzmq.so > -> libzmq.so.0.0.0 > lrwxrwxrwx 1 root root 15 2010年09月03日 14:35 > /usr/local/lib/libzmq.so.0 -> libzmq.so.0.0.0 > -rwxr-xr-x 1 root root 1993116 2010年09月03日 14:35 > /usr/local/lib/libzmq.so.0.0.0 > > And it looks like the extension was built OK, complete with linking: > > gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall > -Wstrict-prototypes -fPIC -I/usr/local/include -I/usr/include/python2.6 > -c zmq/_zmq.c -o build/temp.linux-x86_64-2.6/zmq/_zmq.o > zmq/_zmq.c:1995: warning: ‘__pyx_doc_3zmq_4_zmq_7Message___len__’ > defined but not used > gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions > build/temp.linux-x86_64-2.6/zmq/_zmq.o -L/usr/local/lib -lzmq -o > build/lib.linux-x86_64-2.6/zmq/_zmq.so > > > I'm stumped. > > Eric > >> Cheers, >> >> Brian > > > ------------------------------------------------------------------------------ > This SF.net Dev2Dev email is sponsored by: > > Show off your parallel programming skills. > Enter the Intel(R) Threading Challenge 2010. > http://p.sf.net/sfu/intel-thread-sfd > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgr...@ca... ell...@gm...
On Fri, Sep 3, 2010 at 6:57 PM, Eric Firing <ef...@ha...> wrote: > It's not quite that simple. After some initial thrashing around, I > installed zmq from source, and then pyzmq--but I can't import zmq: > Mhh, sorry to see you burn up on this, Eric. Brian is the zmq expert, not me, but it *may* be a version compatibility issue, perhaps? I am using these versions of zmq/pyzmq: - zmq 2.0.8: http://www.zeromq.org/local--files/area:download/zeromq-2.0.8.tar.gz - pyzmq 2.0.7: http://github.com/downloads/zeromq/pyzmq/pyzmq-2.0.7.tar.gz These two worked fine for me, with: - for zmq: ./configure --prefix=$HOME/usr/opt make && make install note that $HOME/usr/opt/lib is in my LD_LIBRARY_PATH, and the include/ dir in my include path, etc. - for pyzmq: python setup.py install --prefix=$HOME/usr/opt And at that point it just worked. A few questions: - is it possible that you did a build from the head of either zmq/pyzmq git tree? that might cause an incompatibility. - could it be that /usr/local/lib isn in your LD_LIBRARY_PATH? I'll be available over the weekend and will be more than happy to try and help sort this out. Regards, and sorry for the hassle f
On 09/03/2010 12:37 PM, Brian Granger wrote: > Hello all, > > I would like to submit the following branch on github for review and > merging into matplotlib trunk: > > http://github.com/ellisonbg/matplotlib/commits/guisupport > > This branch implements the logic needed for the qt4 and wx backends to > fully work with the upcoming IPython 0.11 release. In our testing we > have run many of the mpl examples (including the new animation > examples) in both qt4/wx in both the terminal based IPython and the > new IPython Qt GUI. For background on these changes please see this > thread: > > http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel > > It is important to note that we have not updated the other matplotlib > backends (gtk, tk, etc.) to have this logic. This is mainly because > we know almost nothing about these toolkits and could really use some > help from folks who are experts at the respective toolkits. We have > done some minimal testing and these other backends do work for simple > examples in the terminal IPython, but they won't work in all cases and > will definitely not work in the new IPython Qt based GUI. > > We would love feedback and help testing as these changes are > significant (even though only a few lines of code). To test this > stuff you will need to grab the following IPython development branch: > > http://github.com/ipython/ipython/tree/newkernel > > You should be able to run the examples in regular ipython: > > ipython --pylab qt4|wx > > Or the new GUI > > ipythonqt --pylab qt4|wx Brian, Fernando, I have been doing a little testing with ipython 0.10 versus ipython-newkernel, both modes, and with mpl svn versus your guisupport. There are so many possible modes of operation and combinations of versions and backends that all this will take some time to sort out. Can you give me simple examples of what does *not* work correctly when you use mpl *svn* with ipython-newkernel, in either or both of the console or gui modes, but *does* work with your guisupport version? Thanks. Eric > > Cheers, > > Brian >
Hi Eric, On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing <ef...@ha...> wrote: > > I have been doing a little testing with ipython 0.10 versus > ipython-newkernel, both modes, and with mpl svn versus your guisupport. > There are so many possible modes of operation and combinations of > versions and backends that all this will take some time to sort out. > > Can you give me simple examples of what does *not* work correctly when > you use mpl *svn* with ipython-newkernel, in either or both of the > console or gui modes, but *does* work with your guisupport version? Thanks for your testing, Eric. With matplotlib *alone*, I can't find a way to crash/lock/whatever the combo of matplotlib(svn)+ipython-newkernel. The reason, i believe, is that guisupport.py is available in ipython itself, and it goes out of its way to avoid creating a second main qt app, letting matplotlib create it. Since that main app is alive all the time, there's only one app and one event loop and life is good. But if I were to open another library that uses Qt and makes a new main qApp unconditionally, we'd have problems. Brian and Evan have recently just added the guisupport.py patch to Enthought's ETS, so that now it probably will be pretty hard to actually see the problem: if both ipython and ets go out of their way to avoid the nested main app issue, mpl can get away with making one unconditionally and things will probably work OK. But the idea is for all of us (ipython, ets, mpl, etc) to agree on a collaborative protocol with a simple api: check for one special '_in_event_loop' flag in the main toolkit before making one. That will make it easier to have interactive code that uses Wx or Qt from more than one library coexisting in the same process. I'll let Brian fill in with more details when he has some availability, but I think that's the gist of it. Regards, f
On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing <ef...@ha...> wrote: > On 09/03/2010 12:37 PM, Brian Granger wrote: >> Hello all, >> >> I would like to submit the following branch on github for review and >> merging into matplotlib trunk: >> >> http://github.com/ellisonbg/matplotlib/commits/guisupport >> >> This branch implements the logic needed for the qt4 and wx backends to >> fully work with the upcoming IPython 0.11 release. In our testing we >> have run many of the mpl examples (including the new animation >> examples) in both qt4/wx in both the terminal based IPython and the >> new IPython Qt GUI. For background on these changes please see this >> thread: >> >> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel >> >> It is important to note that we have not updated the other matplotlib >> backends (gtk, tk, etc.) to have this logic. This is mainly because >> we know almost nothing about these toolkits and could really use some >> help from folks who are experts at the respective toolkits. We have >> done some minimal testing and these other backends do work for simple >> examples in the terminal IPython, but they won't work in all cases and >> will definitely not work in the new IPython Qt based GUI. >> >> We would love feedback and help testing as these changes are >> significant (even though only a few lines of code). To test this >> stuff you will need to grab the following IPython development branch: >> >> http://github.com/ipython/ipython/tree/newkernel >> >> You should be able to run the examples in regular ipython: >> >> ipython --pylab qt4|wx >> >> Or the new GUI >> >> ipythonqt --pylab qt4|wx > > Brian, Fernando, > > I have been doing a little testing with ipython 0.10 versus > ipython-newkernel, both modes, and with mpl svn versus your guisupport. > There are so many possible modes of operation and combinations of > versions and backends that all this will take some time to sort out. > > Can you give me simple examples of what does *not* work correctly when > you use mpl *svn* with ipython-newkernel, in either or both of the > console or gui modes, but *does* work with your guisupport version? The problems are when matplotlib and enthought.traits/pyface/mayavi stuff are used together. There are two types of problems: * Multiple apps are created. The enthought stuff used to not check for existing apps, but that has been fixed. * Event loop is started multiple times. This one is more subtle and on some toolkits the error is not fatal. This problem shows up when IPython is run in terminal mode and event loop integration is handled through PyOS_InputHook. In these situations, if matplotlib or ets start the event loop themselves, IPython will hang. Unfortunately, I am having trouble getting an install of both matplotlib svn and ets svn on the same machine, so I can't reproduce any of the failures ATM. I am trying to get things installed so I can reproduce the problems. Cheers, Brian > Thanks. > > Eric > >> >> Cheers, >> >> Brian >> > > ------------------------------------------------------------------------------ > This SF.net Dev2Dev email is sponsored by: > > Show off your parallel programming skills. > Enter the Intel(R) Threading Challenge 2010. > http://p.sf.net/sfu/intel-thread-sfd > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgr...@ca... ell...@gm...
On 09/07/2010 11:07 AM, Fernando Perez wrote: > Hi Eric, > > On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<ef...@ha...> wrote: >> >> I have been doing a little testing with ipython 0.10 versus >> ipython-newkernel, both modes, and with mpl svn versus your guisupport. >> There are so many possible modes of operation and combinations of >> versions and backends that all this will take some time to sort out. >> >> Can you give me simple examples of what does *not* work correctly when >> you use mpl *svn* with ipython-newkernel, in either or both of the >> console or gui modes, but *does* work with your guisupport version? > > Thanks for your testing, Eric. > > With matplotlib *alone*, I can't find a way to crash/lock/whatever the > combo of matplotlib(svn)+ipython-newkernel. > > The reason, i believe, is that guisupport.py is available in ipython > itself, and it goes out of its way to avoid creating a second main qt > app, letting matplotlib create it. Since that main app is alive all > the time, there's only one app and one event loop and life is good. > But if I were to open another library that uses Qt and makes a new > main qApp unconditionally, we'd have problems. > > Brian and Evan have recently just added the guisupport.py patch to > Enthought's ETS, so that now it probably will be pretty hard to > actually see the problem: if both ipython and ets go out of their way > to avoid the nested main app issue, mpl can get away with making one > unconditionally and things will probably work OK. > > But the idea is for all of us (ipython, ets, mpl, etc) to agree on a > collaborative protocol with a simple api: check for one special > '_in_event_loop' flag in the main toolkit before making one. That > will make it easier to have interactive code that uses Wx or Qt from > more than one library coexisting in the same process. Fernando, There are two parts to guisupport: ensure a single main app, and ensure no more than one call to the mainloop. The first makes perfect sense, and cannot cause any problems that I can see. The second one is the one that I think may be both unnecessary and undesirable. The reason is that the gui toolkit mainloop functions or methods are designed for nested calls. This permits blocking within a running mainloop, and allows show() to block when pyplot is not in interactive mode. This is what is lost with the guisupport mods. Some changes to mainloop logic may well be needed, but I don't think that prohibiting nested calls to the underlying toolkit mainloop function is necessary or desirable. Eric > > I'll let Brian fill in with more details when he has some > availability, but I think that's the gist of it. > > Regards, > > f
Eric, Sorry about the delay, I was on vacation last week...comments inline below... On Tue, Sep 7, 2010 at 2:26 PM, Eric Firing <ef...@ha...> wrote: > On 09/07/2010 11:07 AM, Fernando Perez wrote: >> Hi Eric, >> >> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<ef...@ha...> wrote: >>> >>> I have been doing a little testing with ipython 0.10 versus >>> ipython-newkernel, both modes, and with mpl svn versus your guisupport. >>> There are so many possible modes of operation and combinations of >>> versions and backends that all this will take some time to sort out. >>> >>> Can you give me simple examples of what does *not* work correctly when >>> you use mpl *svn* with ipython-newkernel, in either or both of the >>> console or gui modes, but *does* work with your guisupport version? >> >> Thanks for your testing, Eric. >> >> With matplotlib *alone*, I can't find a way to crash/lock/whatever the >> combo of matplotlib(svn)+ipython-newkernel. >> >> The reason, i believe, is that guisupport.py is available in ipython >> itself, and it goes out of its way to avoid creating a second main qt >> app, letting matplotlib create it. Since that main app is alive all >> the time, there's only one app and one event loop and life is good. >> But if I were to open another library that uses Qt and makes a new >> main qApp unconditionally, we'd have problems. >> >> Brian and Evan have recently just added the guisupport.py patch to >> Enthought's ETS, so that now it probably will be pretty hard to >> actually see the problem: if both ipython and ets go out of their way >> to avoid the nested main app issue, mpl can get away with making one >> unconditionally and things will probably work OK. >> >> But the idea is for all of us (ipython, ets, mpl, etc) to agree on a >> collaborative protocol with a simple api: check for one special >> '_in_event_loop' flag in the main toolkit before making one. That >> will make it easier to have interactive code that uses Wx or Qt from >> more than one library coexisting in the same process. > > Fernando, > There are two parts to guisupport: ensure a single main app, and ensure > no more than one call to the mainloop. Yes, that is a good summary. > The first makes perfect sense, > and cannot cause any problems that I can see. The second one is the one > that I think may be both unnecessary and undesirable. The reason is > that the gui toolkit mainloop functions or methods are designed for > nested calls. This permits blocking within a running mainloop, and > allows show() to block when pyplot is not in interactive mode. This is > what is lost with the guisupport mods. Some changes to mainloop logic > may well be needed, but I don't think that prohibiting nested calls to > the underlying toolkit mainloop function is necessary or desirable. This is a very good point and is something that we have thought carefully about. You are very correct, that the functions in guisupport cannot be used to do a nested mainloop. Nested calls to the mainloop should be done in the usual manner by simply calling the appropriate gui toolkit method for doing so. We probably need to clarify this point, but the focus of the functions in guisupport are *only* the first and main invocation of the event loop. Basically, we want to ensure that: * Projects don't accidentally do nested mainloops because there were not aware that the outermost eventloop was already running. * Projects start the outermost eventloop in a manner that other projects will be able to reliably detect. I should mention the other approach that we have tried, but that failed: * Have IPython startup, create an app and start the main loop. * Then monkeypatch the gui toolkit so that the mainloop calls are no-ops. * Further monkeypatch the gui toolkit so that it appears that the mainloop is running (even when it may not be because of PyOS_InputHook magic). This allowed us to do everything, BUT obviously, nested mainloops failed. Thus, making sure that nested mainloops still work was the main reason we have created guisupport. We should better document these details though. Cheers, Brian > Eric > >> >> I'll let Brian fill in with more details when he has some >> availability, but I think that's the gist of it. >> >> Regards, >> >> f > > > ------------------------------------------------------------------------------ > This SF.net Dev2Dev email is sponsored by: > > Show off your parallel programming skills. > Enter the Intel(R) Threading Challenge 2010. > http://p.sf.net/sfu/intel-thread-sfd > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgr...@ca... ell...@gm...
On 09/13/2010 08:54 AM, Brian Granger wrote: > Eric, > > Sorry about the delay, I was on vacation last week...comments inline below... > > On Tue, Sep 7, 2010 at 2:26 PM, Eric Firing<ef...@ha...> wrote: >> On 09/07/2010 11:07 AM, Fernando Perez wrote: >>> Hi Eric, >>> >>> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<ef...@ha...> wrote: >>>> >>>> I have been doing a little testing with ipython 0.10 versus >>>> ipython-newkernel, both modes, and with mpl svn versus your guisupport. >>>> There are so many possible modes of operation and combinations of >>>> versions and backends that all this will take some time to sort out. >>>> >>>> Can you give me simple examples of what does *not* work correctly when >>>> you use mpl *svn* with ipython-newkernel, in either or both of the >>>> console or gui modes, but *does* work with your guisupport version? >>> >>> Thanks for your testing, Eric. >>> >>> With matplotlib *alone*, I can't find a way to crash/lock/whatever the >>> combo of matplotlib(svn)+ipython-newkernel. >>> >>> The reason, i believe, is that guisupport.py is available in ipython >>> itself, and it goes out of its way to avoid creating a second main qt >>> app, letting matplotlib create it. Since that main app is alive all >>> the time, there's only one app and one event loop and life is good. >>> But if I were to open another library that uses Qt and makes a new >>> main qApp unconditionally, we'd have problems. >>> >>> Brian and Evan have recently just added the guisupport.py patch to >>> Enthought's ETS, so that now it probably will be pretty hard to >>> actually see the problem: if both ipython and ets go out of their way >>> to avoid the nested main app issue, mpl can get away with making one >>> unconditionally and things will probably work OK. >>> >>> But the idea is for all of us (ipython, ets, mpl, etc) to agree on a >>> collaborative protocol with a simple api: check for one special >>> '_in_event_loop' flag in the main toolkit before making one. That >>> will make it easier to have interactive code that uses Wx or Qt from >>> more than one library coexisting in the same process. >> >> Fernando, > > >> There are two parts to guisupport: ensure a single main app, and ensure >> no more than one call to the mainloop. > > Yes, that is a good summary. > >> The first makes perfect sense, >> and cannot cause any problems that I can see. The second one is the one >> that I think may be both unnecessary and undesirable. The reason is >> that the gui toolkit mainloop functions or methods are designed for >> nested calls. This permits blocking within a running mainloop, and >> allows show() to block when pyplot is not in interactive mode. This is >> what is lost with the guisupport mods. Some changes to mainloop logic >> may well be needed, but I don't think that prohibiting nested calls to >> the underlying toolkit mainloop function is necessary or desirable. > > This is a very good point and is something that we have thought > carefully about. You are very correct, that the functions in > guisupport cannot be used to do a nested mainloop. Nested calls to > the mainloop should be done in the usual manner by simply calling the > appropriate gui toolkit method for doing so. We probably need to > clarify this point, but the focus of the functions in guisupport are > *only* the first and main invocation of the event loop. Basically, we > want to ensure that: > > * Projects don't accidentally do nested mainloops because there were > not aware that the outermost eventloop was already running. > * Projects start the outermost eventloop in a manner that other > projects will be able to reliably detect. > > I should mention the other approach that we have tried, but that failed: > > * Have IPython startup, create an app and start the main loop. > * Then monkeypatch the gui toolkit so that the mainloop calls are no-ops. > * Further monkeypatch the gui toolkit so that it appears that the > mainloop is running (even when it may not be because of PyOS_InputHook > magic). > > This allowed us to do everything, BUT obviously, nested mainloops > failed. Thus, making sure that nested mainloops still work was the > main reason we have created guisupport. We should better document > these details though. Brian, Thanks for the clarification. Your basic approach sounds fine. I need to think a little more about how to handle this in mpl. The proposed modification is not quite right yet. I probably can't work on it until the weekend. Eric > > Cheers, > > Brian > >> Eric >> >>> >>> I'll let Brian fill in with more details when he has some >>> availability, but I think that's the gist of it. >>> >>> Regards, >>> >>> f >> >> >> ------------------------------------------------------------------------------ >> This SF.net Dev2Dev email is sponsored by: >> >> Show off your parallel programming skills. >> Enter the Intel(R) Threading Challenge 2010. >> http://p.sf.net/sfu/intel-thread-sfd >> _______________________________________________ >> Matplotlib-devel mailing list >> Mat...@li... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel >> > > >
On 09/13/2010 05:46 PM, Brian Granger wrote: > On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<ef...@ha...> wrote: >> On 09/03/2010 12:37 PM, Brian Granger wrote: >>> Hello all, >>> >>> I would like to submit the following branch on github for review and >>> merging into matplotlib trunk: >>> >>> http://github.com/ellisonbg/matplotlib/commits/guisupport >>> >>> This branch implements the logic needed for the qt4 and wx backends to >>> fully work with the upcoming IPython 0.11 release. In our testing we >>> have run many of the mpl examples (including the new animation >>> examples) in both qt4/wx in both the terminal based IPython and the >>> new IPython Qt GUI. For background on these changes please see this >>> thread: >>> >>> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel >>> >>> It is important to note that we have not updated the other matplotlib >>> backends (gtk, tk, etc.) to have this logic. This is mainly because >>> we know almost nothing about these toolkits and could really use some >>> help from folks who are experts at the respective toolkits. We have >>> done some minimal testing and these other backends do work for simple >>> examples in the terminal IPython, but they won't work in all cases and >>> will definitely not work in the new IPython Qt based GUI. >>> >>> We would love feedback and help testing as these changes are >>> significant (even though only a few lines of code). To test this >>> stuff you will need to grab the following IPython development branch: >>> >>> http://github.com/ipython/ipython/tree/newkernel >>> >>> You should be able to run the examples in regular ipython: >>> >>> ipython --pylab qt4|wx >>> >>> Or the new GUI >>> >>> ipythonqt --pylab qt4|wx >> >> Brian, Fernando, >> >> I have been doing a little testing with ipython 0.10 versus >> ipython-newkernel, both modes, and with mpl svn versus your guisupport. >> There are so many possible modes of operation and combinations of >> versions and backends that all this will take some time to sort out. >> >> Can you give me simple examples of what does *not* work correctly when >> you use mpl *svn* with ipython-newkernel, in either or both of the >> console or gui modes, but *does* work with your guisupport version? > > The problems are when matplotlib and enthought.traits/pyface/mayavi > stuff are used together. There are two types of problems: > > * Multiple apps are created. The enthought stuff used to not check > for existing apps, but that has been fixed. That one is easy. > * Event loop is started multiple times. This one is more subtle and > on some toolkits the error is not fatal. This problem shows up when > IPython is run in terminal mode and event loop integration is handled > through PyOS_InputHook. In these situations, if matplotlib or ets > start the event loop themselves, IPython will hang. So, this can be checked with nothing but IPython and mpl. I think I may have seen this with some combination of configurations, though not with what I typically use. > > Unfortunately, I am having trouble getting an install of both > matplotlib svn and ets svn on the same machine, so I can't reproduce > any of the failures ATM. I am trying to get things installed so I can > reproduce the problems. At least twice in the last couple of years I have tried to get mayavi compiled and running, without fouling up my development versions of numpy and mpl. I never succeeded. Granted, I didn't allocate much time and mental energy to it. In any case, with the help of your recent explanations, I expect we can make mpl play by your suggested rules without sacrificing anything. Part of the change for mpl 1.0 was to factor all of the show logic out into backend_bases.py, leaving only the core mainloop call in the specific backends. I hope we can keep that aspect, retaining the ability to work with earlier ipython (0.10) and the ability for show to block or not depending on the interactivity state. Eric
On Tue, Sep 14, 2010 at 12:59 AM, Eric Firing <ef...@ha...> wrote: > On 09/13/2010 05:46 PM, Brian Granger wrote: >> >> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<ef...@ha...> wrote: >>> >>> On 09/03/2010 12:37 PM, Brian Granger wrote: >>>> >>>> Hello all, >>>> >>>> I would like to submit the following branch on github for review and >>>> merging into matplotlib trunk: >>>> >>>> http://github.com/ellisonbg/matplotlib/commits/guisupport >>>> >>>> This branch implements the logic needed for the qt4 and wx backends to >>>> fully work with the upcoming IPython 0.11 release. In our testing we >>>> have run many of the mpl examples (including the new animation >>>> examples) in both qt4/wx in both the terminal based IPython and the >>>> new IPython Qt GUI. For background on these changes please see this >>>> thread: >>>> >>>> >>>> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel >>>> >>>> It is important to note that we have not updated the other matplotlib >>>> backends (gtk, tk, etc.) to have this logic. This is mainly because >>>> we know almost nothing about these toolkits and could really use some >>>> help from folks who are experts at the respective toolkits. We have >>>> done some minimal testing and these other backends do work for simple >>>> examples in the terminal IPython, but they won't work in all cases and >>>> will definitely not work in the new IPython Qt based GUI. >>>> >>>> We would love feedback and help testing as these changes are >>>> significant (even though only a few lines of code). To test this >>>> stuff you will need to grab the following IPython development branch: >>>> >>>> http://github.com/ipython/ipython/tree/newkernel >>>> >>>> You should be able to run the examples in regular ipython: >>>> >>>> ipython --pylab qt4|wx >>>> >>>> Or the new GUI >>>> >>>> ipythonqt --pylab qt4|wx >>> >>> Brian, Fernando, >>> >>> I have been doing a little testing with ipython 0.10 versus >>> ipython-newkernel, both modes, and with mpl svn versus your guisupport. >>> There are so many possible modes of operation and combinations of >>> versions and backends that all this will take some time to sort out. >>> >>> Can you give me simple examples of what does *not* work correctly when >>> you use mpl *svn* with ipython-newkernel, in either or both of the >>> console or gui modes, but *does* work with your guisupport version? >> >> The problems are when matplotlib and enthought.traits/pyface/mayavi >> stuff are used together. There are two types of problems: >> >> * Multiple apps are created. The enthought stuff used to not check >> for existing apps, but that has been fixed. > > That one is easy. > >> * Event loop is started multiple times. This one is more subtle and >> on some toolkits the error is not fatal. This problem shows up when >> IPython is run in terminal mode and event loop integration is handled >> through PyOS_InputHook. In these situations, if matplotlib or ets >> start the event loop themselves, IPython will hang. > > So, this can be checked with nothing but IPython and mpl. I think I may have > seen this with some combination of configurations, though not with what I > typically use. > >> >> Unfortunately, I am having trouble getting an install of both >> matplotlib svn and ets svn on the same machine, so I can't reproduce >> any of the failures ATM. I am trying to get things installed so I can >> reproduce the problems. > > At least twice in the last couple of years I have tried to get mayavi > compiled and running, without fouling up my development versions of numpy > and mpl. I never succeeded. Granted, I didn't allocate much time and > mental energy to it. Yes, it can be a challenge, we will see how far I get... > In any case, with the help of your recent explanations, I expect we can make > mpl play by your suggested rules without sacrificing anything. Part of the > change for mpl 1.0 was to factor all of the show logic out into > backend_bases.py, leaving only the core mainloop call in the specific > backends. I hope we can keep that aspect, retaining the ability to work > with earlier ipython (0.10) and the ability for show to block or not > depending on the interactivity state. The refactored show logic does help a lot. I am pretty sure that my branch already does all the things you want though. If there are things that it is missing we can definitely add them. But when you get a chance to look at this, let us know and we can continue the discussion. Cheers, Brian > Eric > > > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgr...@ca... ell...@gm...