matplotlib-devel Mailing List for matplotlib

I'm working on improving the plotting speed of one of our applications. As
part of that, I was looking at how quickly the QtAgg backend is drawing. I
added some print out's to the backend and noticed that it seems to be
drawing everything twice. In the code:
backend_qtagg.py: FigureCanvasQTAgg class:
 def draw( self ):
 """
 Draw the figure when xwindows is ready for the update
 """
 if DEBUG: print "FigureCanvasQtAgg.draw", self
 self.replot = True
 FigureCanvasAgg.draw(self)
 self.repaint( False )
FigureCanvasAgg.draw() will cause a draw the repaint call will cause a draw.
If I comment out the FCAgg.draw() call, my application (QtAgg embedded in a
PyQt program) works fine. I've also run several of the various example
scripts and they seem to work fine as well. Panning, zooming, redraws,
resize, and file saving seem to be unaffected.
Does anyone know why the extra draw call might be necessary?
Ted

I'm getting the traceback shown below in _get_configdir. I've attached a 
patch for cutils.py
[sierra@sierra-trac ~]$
/data0/sierra/trac/trac-0.11stable/bin/trac-admin /data0/www/trac/sierra
resync
Failed to open environment. not enough arguments for format string
Traceback (most recent call last):
...
"/data0/sierra/lib/python2.5/site-packages/matplotlib/__init__.py", line
400, in _get_configdir
 raise RuntimeError("'%s' is not a writable dir; you must set
%s/.matplotlib to be a writable dir. You can also set environment
variable MPLCONFIGDIR to any writable directory where you want
matplotlib data stored "%h)
TypeError: not enough arguments for format string
-- 
----------------
Mark E. Hamilton
Orion International Technologies, Inc.
Sandia National Laboratory, NM.
505-844-7666

Shannon Jaeger wrote:
> Hi Ryan and Eric,
> 
> I work on software that plots bits and pieces of radio astronomy
> data. We've run into similar issues and will be running into
> even more related to performance.
> 
> We have datasets that have millions of points, potentially into
> the billions in the not so distant future. Sometimes the way
> the users have choosen what bits and pieces to plot we need to
> break up the data into chunks and plot each chunk separately.
> To further complicate we allow them to overplot other large plots
> on top.
> 
> Anyway I'm very interested in learning more about how to use the
> quadmesh code and using it as well. Although I haven't played
> much with the source code at all, any advice on how to expose this
> at the API level would be appreciated. If modifying the backend
> is all that's needed we've already done a bit of this to tailor
> things to our needs.
Shannon,
I'm not sure what you mean by "expose this at the API level"; quadmesh 
is already exposed via the pcolormesh and pcolorfast axes methods. Do 
you mean via a pyplot/pylab wrapper? This can be done easily, but my 
intention was to work out the kinks in the pcolorfast method API before 
 wrapping it in pyplot, and possibly deprecate pcolormesh on the 
grounds that its functionality are included in pcolorfast. Instead of a 
proliferation of variations on this type of plot, I was hoping to 
consolidate everything into no more than two alternatives: the original 
pcolor, which can draw outlines around the boxes and draws only unmasked 
boxes; and pcolorfast, which omits those pesky outlines, and uses the 
fastest available algorithm for the type of data that are provided.
Eric
> 
> Thanks for any suggestions in advance.
> 
> Shannon
> ------------
> Shannon Jaeger
> Atacama Large Millimetre Array
> Dept. of Physics & Astronomy
> University of Calgary,
> Calgary, AB, Canada
> email: sdj...@uc...
> ph: +1 403 210 8495
> 
> 
> 
> On Thu, March 13, 2008 11:37 am, Eric Firing said:
>> Ryan,
>>
>> The pcolor implementation is fundamentally unsuited to large arrays.
>> Therefore I made the pcolorfast axes method, which tries to use the
>> fastest available Agg extension code, depending on the characteristics
>> of the spatial grid. If the grid is rectangular and regular in both
>> directions it uses a slight modification of the image code; if it is
>> rectangular but with irregular spacing, it uses the nonuniform image
>> code; and if it is not rectangular it uses the quadmesh code. It sounds
>> like what you need is the quadmesh version, which you can access either
>> via pcolormesh or pcolorfast. Neither is exposed via the pylab or
>> pyplot API at present; both are axes methods. The pcolorfast API also
>> may change slightly in the future; it probably needs a little more work.
>>
>> The quadmesh code has problems with masked arrays in the released
>> version of mpl, but not in the svn version. It is *much* faster than
>> pcolor, but may not be fast enough for your needs.
>>
>> If you are looking into what sounds like an OpenGL backend, or component
>> to a backend, then the place to start is still probably pcolormesh or
>> pcolorfast, not pcolor.
>>
>> Eric
>>
>> Ryan May wrote:
>>> Hi,
>>>
>>> I've been poking around with pcolor, trying to see what makes it tick,
>>> since its performance is biggest thing that drives me nuts about
>>> matplotlib. I do pcolor plots of weather radar data with ~100000
>>> polygons in them. Unfortunately, these polygons are at best trapezoids,
>>> so I can't treat it as image data. With this data, pcolor takes ages to
>>> zoom and pan, even on my new workstaion. My end goal is to be able to
>>> use OpenGL to do some 2D rendering, since in my experience, this simply
>>> flies at rendering such data for interactive analysis.
>>>
>>> I noticed that when I run the pcolor_demo.py (using current SVN trunk),
>>> 29396 separate calls are registered to RendererGDK->draw_path to simply
>>> draw the image the first time. Is there any reason why these can't be
>>> batched up and passed in one block to the renderer? This would make
>>> life easier in setting up OpenGL to render in one pass.
>>>
>>> Thanks
>>>
>>> Ryan
>>>
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2008.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>>
>>
>>
> 
> 

Ryan,
The pcolor implementation is fundamentally unsuited to large arrays. 
Therefore I made the pcolorfast axes method, which tries to use the 
fastest available Agg extension code, depending on the characteristics 
of the spatial grid. If the grid is rectangular and regular in both 
directions it uses a slight modification of the image code; if it is 
rectangular but with irregular spacing, it uses the nonuniform image 
code; and if it is not rectangular it uses the quadmesh code. It sounds 
like what you need is the quadmesh version, which you can access either 
via pcolormesh or pcolorfast. Neither is exposed via the pylab or 
pyplot API at present; both are axes methods. The pcolorfast API also 
may change slightly in the future; it probably needs a little more work.
The quadmesh code has problems with masked arrays in the released 
version of mpl, but not in the svn version. It is *much* faster than 
pcolor, but may not be fast enough for your needs.
If you are looking into what sounds like an OpenGL backend, or component 
to a backend, then the place to start is still probably pcolormesh or 
pcolorfast, not pcolor.
Eric
Ryan May wrote:
> Hi,
> 
> I've been poking around with pcolor, trying to see what makes it tick, 
> since its performance is biggest thing that drives me nuts about 
> matplotlib. I do pcolor plots of weather radar data with ~100000 
> polygons in them. Unfortunately, these polygons are at best trapezoids, 
> so I can't treat it as image data. With this data, pcolor takes ages to 
> zoom and pan, even on my new workstaion. My end goal is to be able to 
> use OpenGL to do some 2D rendering, since in my experience, this simply 
> flies at rendering such data for interactive analysis.
> 
> I noticed that when I run the pcolor_demo.py (using current SVN trunk), 
> 29396 separate calls are registered to RendererGDK->draw_path to simply 
> draw the image the first time. Is there any reason why these can't be 
> batched up and passed in one block to the renderer? This would make 
> life easier in setting up OpenGL to render in one pass.
> 
> Thanks
> 
> Ryan
> 

Hi,
I've been poking around with pcolor, trying to see what makes it tick, 
since its performance is biggest thing that drives me nuts about 
matplotlib. I do pcolor plots of weather radar data with ~100000 
polygons in them. Unfortunately, these polygons are at best trapezoids, 
so I can't treat it as image data. With this data, pcolor takes ages to 
zoom and pan, even on my new workstaion. My end goal is to be able to 
use OpenGL to do some 2D rendering, since in my experience, this simply 
flies at rendering such data for interactive analysis.
I noticed that when I run the pcolor_demo.py (using current SVN trunk), 
29396 separate calls are registered to RendererGDK->draw_path to simply 
draw the image the first time. Is there any reason why these can't be 
batched up and passed in one block to the renderer? This would make 
life easier in setting up OpenGL to render in one pass.
Thanks
Ryan
-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Eric Firing wrote:
> Jeff,
>
> When putting a legend on a basemap plot, I find that the legend command 
> picks up the coastline line collection and gives it an uninformative 
> label. I suspect this is rarely what one wants. Two suggestions:
>
> 1) In drawcoastlines, use "coastlines.set_label('_nolabel_') to suppress 
> inclusion in the legend, and similarly for drawcountries, etc.
> 
Eric:
Done for coastlines, rivers, countries and states.
> 2) Make drawcoastlines return the line collection that it creates. 
> Similarly, drawcountries could return the line collection, 
> fillcoastlines could return the list of patches, etc. 
All done.
> Drawparallels and 
> drawmeridians are a little more complicated because they make the labels 
> as well as the lines, so I am not sure what to do with them.
> 
drawmeridians and drawmeridians now return dictionaries, with the Line2D 
and Text instances associated with each latitude or longitude.
> Granted, one can grab the reference to the line collection from the end 
> of ax.collections, but it seems cheap, reasonable, and more convenient 
> to simply have it returned by the command that makes it.
>
> Eric
>
>
> 
Thanks for the suggestion.
-Jeff
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jef...@no...
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg

Jeff,
When putting a legend on a basemap plot, I find that the legend command 
picks up the coastline line collection and gives it an uninformative 
label. I suspect this is rarely what one wants. Two suggestions:
1) In drawcoastlines, use "coastlines.set_label('_nolabel_') to suppress 
inclusion in the legend, and similarly for drawcountries, etc.
2) Make drawcoastlines return the line collection that it creates. 
Similarly, drawcountries could return the line collection, 
fillcoastlines could return the list of patches, etc. Drawparallels and 
drawmeridians are a little more complicated because they make the labels 
as well as the lines, so I am not sure what to do with them.
Granted, one can grab the reference to the line collection from the end 
of ax.collections, but it seems cheap, reasonable, and more convenient 
to simply have it returned by the command that makes it.
Eric