matplotlib-devel Mailing List for matplotlib

On 7/31/06, Ken McIvor <mc...@ii...> wrote:
> On Jul 30, 2006, at 8:07 AM, Bill Baxter wrote:
> > I went ahead and implemented this yesterday on a long plane flight.
> > The changed files (backend_bases.py, and widgets.py) are attached to
> > the above tracker entry. Also I changed backend_wx.py to grab the
> > mouse generally when you click on the graphs, so that panning and such
> > continues to track even when you go outside the window. Similar
> > changes should probably also be made to the other backends too.
>
> This sounds like it could be a pretty stellar improvement to the WX
> backend. Thanks!
Just to be clear, the only thing that applies to wx specifically is
that I made the wx backend grab the system pointer on mouse down and
release it on mouse up. (Basically just a change of 4 lines of code
-- two CaptureMouse() calls and two ReleaseMouse() calls). The rest
is GUI independent and should benefit all GUI backends.
>
> I should be at a work-related meeting on Tuesday with Matt Newville,
> the WX backend maintainer. I'll try to get some time set aside to
> get your changes merged on my laptop before them, so that we can
> check them out properly afterward. My big concern is to investigate
> how these changes may affect people who are embedding WxAgg figures
> in wxPython applications. If you've done what I think you have, that
> shouldn't be a problem.
Definitely more people should test it and review the code. It works
for me, but maybe there are some things I haven't thought of. Like
I've never tried embedding. But I don't think this should cause
problems there. On the other hand, the widget-level grabbing
infrastructure might have problems with a GUI back-end that doesn't do
system-level grabbing. For instance you start dragging on a Slider,
drag off the window, and let go. If you aren't doing system level
grabbing then you won't ever get the mouse up to tell you that you
should release the widget-level mouse grab.
> > I also added get_value methods to the CheckButtons and RadioButtons,
> > because it didn't seem like there was any good way to get that info.
>
> You may want to submit this portion of the changes as a separate
> patch. I've found that my patches get accepted faster when I only do
> one thing per patch.
Ok. Such a pain though... Are whole files acceptable instead of
diffs? It's relatively easy to do a windiff or whatever to look over
the changes and accept or reject line by line. (In fact Robert
Osfield who quite ably runs the OpenSceneGraph project feels so
strongly about it that he *only* allows patches to be submitted as
complete files so he can easilly use a visual diff tool to merge
changes).
> > Finally I don't really get what the deal is with the Slider's
> > "closedmin"/"closedmax" options.
>
> Sorry, I can't help you with this one.
>
> > Please let me know if there's a better way/place to submit patches.
>
> I've always done exactly what you're doing now: uploading it on SF
> then emailing the developers' list.
Ok. Thanks.
--Bill

Hi Ken,
Thanks for the reply.
On 7/31/06, Ken McIvor <mc...@ii...> wrote:
> On Jul 24, 2006, at 9:16 PM, Bill Baxter wrote:
> > I think all these problems could be fixed if the display interface
> > were turned into a separate process that communicates with the Python
> > program using pipes or some other IPC mechanism. I used this
> > technique in a (C/C++) image debugging utility program I wrote
> > (http://www.billbaxter.com/projects/imdebug) and it works quite well.
>
> The biggest problem I see with this approach is that matplotlib has a
> "display list" drawing model (as opposed to the "big matrix of
> pixels" model). An example of what I mean is that when someone
> resizes a {Gtk,Tk,Wx}Agg figure, the entire plot is re-rendered from
> scratch to the new pixel dimensions. It could be a failure of
> imagination, but I can't see how you could move the display interface
> to a subprocess without moving the rest of matplotlib along with it.
> I just can't see a clear line along which to separate out the "graph
> display server" part.
What's the problem with sending the "display list" or with having most
of matplotlib also exist in the separate process? As long as
evaluating that display list doesn't involve making extensive
callbacks into user code, then it shouldn't be a problem. Having
matplotlib live on both sides of the process boundary is ok in my
opinion, because it can be made to work that way. But having to have
the *users's* app on both sides is a problem. Or having to do so many
RPC callbacks back to users' code that a single rendering becomes
intolerably slow.
Conceptually, in terms of the "clear line", I think maybe a special
'axes' class would get you a lot of what is needed. That would act
like a proxy for 'axes' actually living in another process.
Anyway, clearly matplotlib shouldn't always use remote display. If
you're using for embedded graphing then that would be silly. It would
be more for general interactive use and debugging purposes.
> > Has anyone given thought to making matplotlib work in such a manner?
> > It would be hell to do in C or C++ I think, but with Python's
> > extensive RPC libs I bet it wouldn't be so bad.
>
> Python has extensive RPC libraries?!?! ;-)
Heh, well compared to C++ at least. :-) My only experience with it
was with doing some very simple operations using xmlrpc. But compared
with how hard that would have been for me to do in C++ I was
impressed.
> The point of this long-winded email is that you probably could build
> some kind of RPC system to run matplotlib remotely, but I believe it
> would be an awful lot of work. In terms of manhours, my opinion is
> that you'd come out ahead if you just focused on debugging your
> current problems.
Ok. It's good to know practically the best way to achieve "good
enough" but I like to think and discuss about how to achieve "best
possible", also. :-)
Regards
--bb

On Jul 30, 2006, at 8:07 AM, Bill Baxter wrote:
> I went ahead and implemented this yesterday on a long plane flight.
> The changed files (backend_bases.py, and widgets.py) are attached to
> the above tracker entry. Also I changed backend_wx.py to grab the
> mouse generally when you click on the graphs, so that panning and such
> continues to track even when you go outside the window. Similar
> changes should probably also be made to the other backends too.
This sounds like it could be a pretty stellar improvement to the WX 
backend. Thanks!
I should be at a work-related meeting on Tuesday with Matt Newville, 
the WX backend maintainer. I'll try to get some time set aside to 
get your changes merged on my laptop before them, so that we can 
check them out properly afterward. My big concern is to investigate 
how these changes may affect people who are embedding WxAgg figures 
in wxPython applications. If you've done what I think you have, that 
shouldn't be a problem.
> I also added get_value methods to the CheckButtons and RadioButtons,
> because it didn't seem like there was any good way to get that info.
You may want to submit this portion of the changes as a separate 
patch. I've found that my patches get accepted faster when I only do 
one thing per patch.
> Finally I don't really get what the deal is with the Slider's
> "closedmin"/"closedmax" options.
Sorry, I can't help you with this one.
> Please let me know if there's a better way/place to submit patches.
I've always done exactly what you're doing now: uploading it on SF 
then emailing the developers' list.
Ken

Andrew,
This looks very cool, and I'm looking forward to playing around with 
it. Thanks for the hard work!
Shooting from the hip, here are some initial comments. I may be able 
to submit patches for some of the more innocuous items later in the 
week.
1. It appears that as_sizer_element() uses the _axes_sizer_elements 
dictionary to cache MplAxesSizerElement instances. Using a 
WeakKeyDictionary from the "weakref" module instead of a regular 
dictionary may be necessary to allow the garbage collection of the 
MplAxesSizerElements when their associated Axes gets GC'd.
2. Convenience MplBoxSizer subclasses that let you omit the "orient" 
keyword might be nice:
 class MplHBoxSizer(MplBoxSizer):
 def __init__(self):
 MplBoxSizer.__init__(self, orientation='vertical')
3. Couldn't you just drop mplsizer.py into the "matplotlib.toolkits" 
virtual package? Maybe you can't -- I'm pretty new to applied python- 
eggery.
4. I feel we should avoid the whole European/American spelling 
problem that WX has. Why not make both 'align_centre' and 
'align_center' do the same thing?
5. Why not use shorter names, with less redundancy? (e.g. 
"matplotlib.toolkits.sizer", FigureSizer, Box, HBox, Grid, etc)
Ken

On Thu, Jul 27, 2006 at 01:33:42PM +0100, JIM MacDonald wrote:
> Another way to do it is to pass extra
> command line options to ps2pdf (-dAutoFilterColorImages=3Dfalse
> -sColorImageFilter=3DFlateEncode should do it for colour images). I
> thought embedding it in the PS file would be more flexible.
 +1 for that. I am having exactly the same problem.
--=20
 Ga=EBl

On Jul 24, 2006, at 9:16 PM, Bill Baxter wrote:
> I think all these problems could be fixed if the display interface
> were turned into a separate process that communicates with the Python
> program using pipes or some other IPC mechanism. I used this
> technique in a (C/C++) image debugging utility program I wrote
> (http://www.billbaxter.com/projects/imdebug) and it works quite well.
The biggest problem I see with this approach is that matplotlib has a 
"display list" drawing model (as opposed to the "big matrix of 
pixels" model). An example of what I mean is that when someone 
resizes a {Gtk,Tk,Wx}Agg figure, the entire plot is re-rendered from 
scratch to the new pixel dimensions. It could be a failure of 
imagination, but I can't see how you could move the display interface 
to a subprocess without moving the rest of matplotlib along with it. 
I just can't see a clear line along which to separate out the "graph 
display server" part.
> Has anyone given thought to making matplotlib work in such a manner?
> It would be hell to do in C or C++ I think, but with Python's
> extensive RPC libs I bet it wouldn't be so bad.
Python has extensive RPC libraries?!?! ;-)
In seriousness, RPC is an acknowledged weakness of Python's standard 
library. For example, Python gives you a sockets API that is much 
nicer to use than the BSD sockets API or WinSock, but which still 
makes you worry about all of the platform-specific idiosyncrasies of 
socket programming. It's my understanding that this is one of the 
big reasons Twisted came into being.
As another example, before Python 2.4 added the "subprocess" module, 
there was no portable way to spawn and communicate with 
subprocesses... you'd have to worry about the shell of whatever 
platform you were on mangling the commandline, you couldn't reliably 
interrupt subprocesses on win32 (no os.kill() to send a SIGINT with), 
etc.
The point of this long-winded email is that you probably could build 
some kind of RPC system to run matplotlib remotely, but I believe it 
would be an awful lot of work. In terms of manhours, my opinion is 
that you'd come out ahead if you just focused on debugging your 
current problems. I suspect that the matplotlib developers and 
irregular contributers like myself will more able to help with that 
debugging than we would be able to help with writing a display 
server... but please don't think that I'm speaking for everyone!
Ken

I submitted an enhancement request to the Sourceforge bug list about
grabbing behavior in the GUI-independent widgets.
http://sourceforge.net/tracker/index.php?func=detail&aid=1530104&group_id=80706&atid=560720
I went ahead and implemented this yesterday on a long plane flight.
The changed files (backend_bases.py, and widgets.py) are attached to
the above tracker entry. Also I changed backend_wx.py to grab the
mouse generally when you click on the graphs, so that panning and such
continues to track even when you go outside the window. Similar
changes should probably also be made to the other backends too.
I also added get_value methods to the CheckButtons and RadioButtons,
because it didn't seem like there was any good way to get that info.
Finally I don't really get what the deal is with the Slider's
"closedmin"/"closedmax" options. These don't seem very useful to me
(namely the =False variety). If it meant the max was actually
max-[machine epsilon] then ok, but it's implemented as "don't set the
value if it would get set to the max" i.e. how close you get to the
max/min depends on how patient you are carefully moving the mouse to
the end of the slider. Which is just frustrating to me. It's hard to
make it go "as high as it will go".
Please let me know if there's a better way/place to submit patches.
--bb

On 7/30/06, Andrew Straw <str...@as...> wrote:
> The basic
> idea is to create a layout engine for matplotlib. Not wanting to
> (re-)invent an API, I decided simply to imitate the layout engine I knew
> best, which is wxPython.
Very cool!

>>>>> "Eric" == Eric Firing <ef...@ha...> writes:
 Eric> John, I think we really need copy (and maybe deepcopy)
 Eric> functions that work with all transforms, not just Separable
 Eric> transforms. This looks fairly easy except for one thing:
 Eric> the transform creation functions return objects that don't
 Eric> provide any clean way of distinguishing among the types of
 Eric> transform. type(trans) reports <type 'Affine'>, regardless
I've seen this, I thinki it's a problem with pycxx but am not sure
 Eric> of what kind of transform it really is. I have not been
 Eric> able to figure out where this is coming from. One can't
 Eric> cleanly use hasattr(funcxy) to detect a Nonseparable
 Eric> transform because all transforms have the attribute, whether
 Eric> they use it or not. I could use "try: trans.get_funcxy()"
Again, this is a problem with pycxx. You cannot do inheritance where
B and C inherit some methods from A unless all methods are in A, B and
C. It's ugly but that is the way it is for now. So I define all the
methods in the base class and raise if they are not available.
Unfortunately, pycxx is not actively developed so I doubt this will
change .
 Eric> and catch the exception, but that is ugly. (And the second
 Eric> time I tried it, it hung ipython.)
 Eric> I suspect you have thought about this already--do you have
 Eric> any suggested solutions? Is there at least a simple way to
 Eric> get type(trans) to work right? From the code it looks like
 Eric> it should, so there appears to be a bug in the code or in
 Eric> cxx.
The best way may be for the extension code to provide a shallowcopy
method and require derived transform classes to implement it. All
references will be preserved, but a new object will be created.
We only need this for SeparableTransformation and
NonseparableTransformation but the methods will also have to be
defined virtually in the base classes.
We have to think about what should be preserved in the shallow
copies. For the use case at hand, we want to preserve the references
to the values but not the offset transform.
I'm not so sure that deepcopy is really needed. I can't think of a
use case off hand.
As I respond, I wonder if we are applying the right solution to the
wrong problem. I think these changes are worth doing because they are
easy and work with the existing code and are useful. But in the
longer run, I think the offsets, while useful, can be better
accomplished by developing a transform chain as Jouni suggested.
Normal affine multiplication doesn't work since the transformations
may be nonlinear. But we should be able to do something like (here in
python but this would probably be in the extension code)
class CompositeTransform:
 def __init__(self, transforms):
 self._transforms = transforms
 def xy_tup(self, xy):
 for transform in self._transforms:
 xy = transform.xy_tup(xy)
 return xy
Removing the offset transforms would break internal and external code,
but would probably be a cleaner solution in the long run.
JDH

John,
I think we really need copy (and maybe deepcopy) functions that work 
with all transforms, not just Separable transforms. This looks fairly 
easy except for one thing: the transform creation functions return 
objects that don't provide any clean way of distinguishing among the 
types of transform. type(trans) reports <type 'Affine'>, regardless of 
what kind of transform it really is. I have not been able to figure out 
where this is coming from. One can't cleanly use hasattr(funcxy) to 
detect a Nonseparable transform because all transforms have the 
attribute, whether they use it or not. I could use "try: 
trans.get_funcxy()" and catch the exception, but that is ugly. (And the 
second time I tried it, it hung ipython.)
I suspect you have thought about this already--do you have any suggested 
solutions? Is there at least a simple way to get type(trans) to work 
right? From the code it looks like it should, so there appears to be a 
bug in the code or in cxx.
Eric