matplotlib-users Mailing List for matplotlib

On 08/27/2013 09:49 AM, Chris Beaumont wrote:
> I've been burned by this before as well. MPL stores some intermediate 
> data products (for example, scaled RGB copies) at full resolution, 
> even though the final rendered image is downsampled depending on 
> screen resolution.
>
> I've used some hacky tricks to get around this, which mostly involve 
> downsampling the image on the fly based on screen resolution. One such 
> effort is at https://github.com/ChrisBeaumont/mpl-modest-image.
It looks like this wouldn't be too hard to include in matplotlib. I 
don't think we'd want to change the current behavior, because sometimes 
its tradeoff curve makes sense, but in other cases, the "modest image" 
approach also makes sense. It's just a matter of coming up with an API 
to switch between the two behaviors. Pull request?
Cheers,
Mike
>
> If you are loading your arrays from disk, you can also use 
> memory-mapped arrays -- this prevents you from loading all the data 
> into RAM, and further cuts down on the footprint.
>
> cheers,
> chris
>
>
> On Tue, Aug 27, 2013 at 6:49 AM, S(te(pán Turek 
> <ste...@se... <mailto:ste...@se...>> wrote:
>
>
> You could look at whether or not you actually need 64-bit
> precision. Often times, 8-bit precision per color channel is
> justifiable, even in grayscale. My advice is to play with the
> dtype of your array or, as you mentioned, resample.
>
>
> thanks, this helped me significantly, uint8 precision is enough.
>
> Also, is it needed to keep all images? It sounds to me like
> your application will become very resource hungry if you're
> going to be displaying several of these 2D images over each
> other (and if you don't use transparency, you won't get any
> benefit at all from plotting them together).
>
>
> Yes, I need them all .
>
> To avoid it I am thinking about merging them into one image and
> then plot it.
>
>
> Stepan
>
>
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and
> AppDynamics. Performance Central is your source for news, insights,
> analysis and resources for efficient Application Performance
> Management.
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> <mailto:Mat...@li...>
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
>
>
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and
> AppDynamics. Performance Central is your source for news, insights,
> analysis and resources for efficient Application Performance Management.
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

I've been burned by this before as well. MPL stores some intermediate data
products (for example, scaled RGB copies) at full resolution, even though
the final rendered image is downsampled depending on screen resolution.
I've used some hacky tricks to get around this, which mostly involve
downsampling the image on the fly based on screen resolution. One such
effort is at https://github.com/ChrisBeaumont/mpl-modest-image.
If you are loading your arrays from disk, you can also use memory-mapped
arrays -- this prevents you from loading all the data into RAM, and further
cuts down on the footprint.
cheers,
chris
On Tue, Aug 27, 2013 at 6:49 AM, Štěpán Turek <ste...@se...>wrote:
>
> You could look at whether or not you actually need 64-bit precision. Often
> times, 8-bit precision per color channel is justifiable, even in grayscale.
> My advice is to play with the dtype of your array or, as you mentioned,
> resample.
>
>
> thanks, this helped me significantly, uint8 precision is enough.
>
>
>
> Also, is it needed to keep all images? It sounds to me like your
> application will become very resource hungry if you're going to be
> displaying several of these 2D images over each other (and if you don't use
> transparency, you won't get any benefit at all from plotting them together).
>
>
> Yes, I need them all .
>
> To avoid it I am thinking about merging them into one image and then plot
> it.
>
>
> Stepan
>
>
>
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and
> AppDynamics. Performance Central is your source for news, insights,
> analysis and resources for efficient Application Performance Management.
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>

"
""
You could look at whether or not you actually need 64-bit precision. Often 
times, 8-bit precision per color channel is justifiable, even in grayscale. 
My advice is to play with the dtype of your array or, as you mentioned, 
resample. 
"
thanks, this helped me significantly, uint8 precision is enough.
 
"
Also, is it needed to keep all images? It sounds to me like your application
will become very resource hungry if you're going to be displaying several of
these 2D images over each other (and if you don't use transparency, you won'
t get any benefit at all from plotting them together).
"
Yes, I need them all .
To avoid it I am thinking about merging them into one image and then plot 
it. 
Stepan
"
 
"

Those numbers actually make a lot of sense.
For a 4k by 4k 2D array of 64-bit floats, you're using 128MiB of memory,
just to store them. Displaying such an array with mpl would take a copy of
that and add some objects for housekeeping (on my machine about 150MB to
display one such array together with the housekeeping objects).
You could look at whether or not you actually need 64-bit precision. Often
times, 8-bit precision per color channel is justifiable, even in grayscale.
My advice is to play with the dtype of your array or, as you mentioned,
resample.
Also, is it needed to keep all images? It sounds to me like your
application will become very resource hungry if you're going to be
displaying several of these 2D images over each other (and if you don't use
transparency, you won't get any benefit at all from plotting them together).
2013年8月27日 Štěpán Turek <ste...@se...>
> Hi,
>
>
> You could, before plotting, sum the different image arrays? Depending on
> whether you are plotting RGB(A) images or greyscale images, you could take
> the sum of the color channels, or take a weighted average.
>
>
> Yes, I will probably merge the images (RGBA) before plotting. I want to
> create more plots and even with this optimization every plot will take 300
> MB... Is there any way how to save some memory?
>
>
> Best
>
> Stepan
>
>
>

Hi,
"
You could, before plotting, sum the different image arrays? Depending on 
whether you are plotting RGB(A) images or greyscale images, you could take 
the sum of the color channels, or take a weighted average. 
"
Yes, I will probably merge the images (RGBA) before plotting. I want to 
create more plots and even with this optimization every plot will take 300 
MB... Is there any way how to save some memory?
Best
Stepan 
 

Le Lun 26 août 2013 18:21, Goyo a écrit :
> 2013年7月19日 Nicolas Mailhot <nic...@la...>:
>> Le Mer 17 juillet 2013 14:56, Michael Droettboom a écrit :
>>> Can you please provide a completely standalone example? The following
>>> code has undefined variables etc.
>>
>> Here it is, I'm afraid this testcase intent is less clear than what I
>> pasted previously (I replaced variables with precomputed values)
>>
>> As shown in the attached png, the bottom tick labels (month names) are
>> missing. It worked in matplotlib ≤ 1.2.0
>
> I can confirm the issue with 1.2.1 but it works with a recent
> development version (output attached) so it must have been fixed at
> some point.
Thank you very much for the data point, I'll try to get 1.3.0 built on RHEL 6
Regards,
-- 
Nicolas Mailhot

You could, before plotting, sum the different image arrays? Depending on
whether you are plotting RGB(A) images or greyscale images, you could take
the sum of the color channels, or take a weighted average.
The method you use here depends strongly on the image type, but it will
reduce memory consumption.
Just a thought.
2013年8月27日 Štěpán Turek <ste...@se...>
> Hi,
>
> I would like to plot multiple overlayed 4096x4096 images in one axes. If I
> run this code the plot takes 300 MB of memory:
>
> import numpy as np
> import matplotlib.pyplot as plt
>
> if __name__ == '__main__':
> img = np.zeros((4096, 4096))
> img[100: 300, 100:1500] = 200
> imgplot = plt.imshow(img)
>
> plt.show()
>
> And it takes additional 300 MB for every image with this size added into
> plot. Is there any way to reduce memory consumption without need of data
> resampling?
>
> My configuration:
> Matplotlib 1.2.1
> Numpy 1.7.1
> Ubuntu 13.04 64 bit
>
> Best
> Stepan
>
>
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and
> AppDynamics. Performance Central is your source for news, insights,
> analysis and resources for efficient Application Performance Management.
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>

Hi,
I would like to plot multiple overlayed 4096x4096 images in one axes. If I 
run this code the plot takes 300 MB of memory:
import numpy as np
import matplotlib.pyplot as plt
if __name__ == '__main__':
  img = np.zeros((4096, 4096))
  img[100: 300, 100:1500] = 200
  imgplot = plt.imshow(img)
  plt.show()
And it takes additional 300 MB for every image with this size added into 
plot. Is there any way to reduce memory consumption without need of data 
resampling?
My configuration:
Matplotlib 1.2.1
Numpy 1.7.1
Ubuntu 13.04 64 bit
Best
Stepan