matplotlib-devel Mailing List for matplotlib

Hello,
I posted this description of a different type of spectrogram to the 
matplotlib-users list a while back, and today I am ready to start work 
on actually implementing it. 
There didn't seem to be a tremendous amount of interest in the idea
(only one reply), yet it is important to my work right now. Chris's
idea of storing the data separately and then generating the appropriate
image is what I had in mind. The only additional concept I plan to
throw in is to use a numpy.memmap instance as the container of the data.
So, I wonder if y'all have any suggestions for me as I start this
project.
The basic functionality is this:
* Display of an image (with the ability to specify the extents of the
 axes) along with other vector elements (most notably, rectangles).
* When the axes is re-sized, or when the toolbar is used to pan, zoom, 
 or rectangle-select a different view, a new image is created and 
 displayed with extents that are appropriate to the new region of 
 the axes that are displayed.
Any tips would be appreciated.
Best Regards,
Glen Mabey
On Tue, Oct 24, 2006 at 09:41:38AM -0700, Christopher Barker wrote:
> Glen W. Mabey wrote:
> >I have come
> >to conclude that the traditional method for generating spectrograms is
> >inherently flawed. That is, the MATLAB approach of generating a picture
> >based on the psd's and then display it using imshow is a hack.
> 
> I'm not very familiar with spectrograms, but I suspect you are correct. 
> Top get the best results, your computations should be tied to the actual 
> resolution that you are displaying.
> 
> However, what this says to me is that using MPL directly may not be the 
> best way to go. It is designed for scalable, vector drawing -- 
> illustrated by your difficulty in finding the exact pixel size of the 
> axes. You might want to consider creating and manipulating the image you 
> want directly. This might best be accomplished by storing image data in 
> a numpy array, manipulating it as you need, then converting that array 
> to a PIL image, or wxImage (or directly to an Agg image in MPL?).
On Tue, Oct 24, 2006 at 10:03:02AM -0500, Glen W. Mabey wrote:
> On Tue, Oct 24, 2006 at 08:49:19AM -0500, John Hunter wrote:
> > >>>>> "Glen" == Glen W Mabey <Gle...@sw...> writes:
> > 
> > Glen> Hello, I have been unable to discover in the docs a method
> > Glen> for discovering the exact size in pixels of an axes.
> > 
> > Glen> The only way I have thought of is to get the size of the
> > Glen> canvas via FigureCanvas.get_width_height() and then multiply
> > Glen> by the results of axes.get_position(), but really I want to
> > Glen> have the exact size in pixels.
> > 
> > In [1]: ax = subplot(111)
> > 
> > In [2]: left, bottom, width, height = ax.bbox.get_bounds()
> > 
> > In [3]: print left, bottom, width, height
> > 
> > 80.0 48.0 496.0 384.0
> > 
> > However, this question looks like one that would be better answered by
> > describing what you are trying to do. There may be a more elegant
> > solution using some of matplotlib's built-in coordinate systems which
> > would prevent you from having to do raw pixel calculations, which is
> > not usually what you want. So you may want to describe your actual
> > problem rather than your solution <wink>
> 
> Happy to do so, and I'm glad you asked.
> 
> I have recently been working on a project where I need to show a
> spectrogram of some large data sets (200 million complex samples).
> Directly using the specgram function is not a good idea, and I have come
> to conclude that the traditional method for generating spectrograms is
> inherently flawed. That is, the MATLAB approach of generating a picture
> based on the psd's and then display it using imshow is a hack.
> 
> I arrived at this conclusion through my work-around to generating
> spectrograms for these large files. What I did was to choose a number
> of "data slice" to use, extract those at regular intervals throughout
> the file, and then set overlap=0.
> 
> For example, if I wanted to use a 1024-point FFT, and if my axes is
> about 500 pixels wide, then I would seek() through the file reading 1024
> contiguous samples and then skipping 1./500-th of the total samples, and
> reading in another slice. I would end up with 500*1024 points, and pass
> this data to specgram() with overlap=0.
> 
> Now, of course, there is a lot of information that was discarded, but it
> made the implementation tractable.
> 
> I would propose that the loss associated with this method was comparable
> to what is lost when the entire data set is used and then an image
> resize algorithm (hardly appropriate for this type of thing, IMHO)
> averages out a tremendous amount of the computations that were
> performed.
> 
> As I started to think about it, I concluded that the other extreme
> applies. For short data sets, it is much more appropriate to have the
> overlap increase automatically than to use an image interpolation
> function. 
> 
> The case of operating on large data sets then corresponds to a negative
> overlap.
> 
> I recall one technical review I was in where the presenter was
> displaying a spectrogram in MATLAB. He pointed out a visible feature
> and then zoomed in on it. It was a large data set, and when it finished
> drawing, the feature was no longer visible -- very strange, and
> frustrating to the presenter! I then began to wonder about the
> appropriateness of treating a spectrogram like a picture.
> 
> Not to imply that there wouldn't be anomalies like this with this
> "auto-overlap" approach, but certainly it seems (to me) like a more 
> rational approach to this signal processing operation.
> 
> So, I'm hoping to find time on my current project to implement this type
> of functionality. In my mind the FFT size would still be selected as a
> parameter, so that power-of-2 implementations are used.. Granted, there 
> would be averaging going on along the vertical axis, I just propose that 
> better 1 than 2: the number of psd's performed would correspond exactly 
> to the horizontal dimension of the drawing area, so no resampling would 
> be required along that axis.
> 
> When the axes is resized, psd's would have to be recomputed, but what is
> displayed on the screen would more closely related to the result of the
> transforms performed.
> 
> Zooming would also necessitate recomputation of the psd's. My idea for
> smooth zooming (dragging with right mouse button in pylab) was to keep
> the existing functionality, until the mouse is released, at which time
> the psd's would be recomputed, but only for the segment of the data that
> corresponds to the visible portion of the horizontal axis. Same thing
> for panning around this zoomed image: don't recalculate anything until
> the user releases the mouse button. 
> 
> This would obviously be a more complicated implementation, and I'm not
> suggesting that the current specgram implementation is useless. This
> alternate approach has served me well so far, and being able to extract
> the size of the axes will make things more efficient.
> 
> Thanks for listening and for the tip.
> 
> Best Regards,
> Glen Mabey
> 
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Hi,
I am using matplotlib with python to generate a bunch of charts. I use
IDLE, have numpy version 1.0, and pylab version 2.5 with TkAgg if that is
relavant. My
code works fine for a single iteration, which creates and saves 4
different charts. The trouble is that when I try to run it for the
entire set (about 200 items) it can run for 12 items at a time, generating
48 charts. On the
13th, I get an error from matplotlib that says it can't access data.
However, if I start the program at the point it just failed, it works
fine and will create the charts for the next 12 before failing. I
assume that I am not closing the files properly somehow or otherwise
misallocating memory. I tried just reimporting pylab each iteration,
but that didn't help. This is the function that creates a chart:
#create and save the figure
def CreateFigure(state, facility, unit, SO2, increment, year, P99):
 size = len(SO2)
 #Create Plot
 figure(1, figsize=(10,8))
 bar(range(1, size+2), SO2, width=0.1, color='k')
 grid(True)
 xlim(0,size)
 ylim(0, 1.1*SO2[-1])
 ylabel('SO2 [lb/hr]')
 heading = ConstructFigName(state, facility, unit, increment, year)
 title(heading)
 #set handles
 xticklines = getp(gca(), 'xticklines')
 xgridlines = getp(gca(), 'xgridlines')
 xticklabels = getp(gca(), 'xticklabels')
 yticklines = getp(gca(), 'yticklines')
 #set properties
 setp(xticklines, visible=False)
 setp(xgridlines, visible=False)
 setp(xticklabels, visible=False)
 setp(yticklines, visible=False)
 axhspan(P99, P99, lw=3, ec='r', fc='r')
 ax = gca()
 P99 = '%0.1f' % P99
 text(0.01, 0.95, '99th Percentile: '+P99+' lb/hr',
transform=ax.transAxes)
 figpath = ConstructFigPath(state, facility, unit, increment, year)
 savefig(figpath)
 close()
Can you see the problem? If there is nothing obviously wrong... I tried to
run the memory leak algorithm from the website, but the line:
a2 = os.popen('ps -p %d -o rss,sz' % pid).readlines()
generates an empty list rather than information about about memory, and I am
not clear on how to modify it to do something useful.
thanks,
-Lisa