Hi All,
my program lets slow down my cpu. This only appears if i plot to much
points. I am not sure how many point i need to get this, normally i plot
3*14e6 + 8e3, that is round about 50million points. My system is a
dual core 2GHz cpu with 2Gbyte Ram.
Here is my method to plot,
def drawtransientall(self,min):
self.subplot = self.figure.add_subplot(111)
self.subplot.grid(True)
list_t1,list_peaks,t2,list_samples =
self.computetransientall(min,min+self.maxitems)
offset = 0
color = ['green','red','blue','magenta','cyan']
markerPeaks = ['v','<','1','3','s']
markerSamples = ['^','>','2','4','p']
self.plots=[[],[]]
for i in range(len(self.showBands)):
self.plots[0] +=
self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i],
linestyle='None')
self.plots[1] +=
self.subplot.plot(t2,list_samples[i]+offset,color=color[i],
marker=markerSamples[i],linestyle='None')
offset +=1
self.subplot.set_xlim(t2[0]-np.abs(t2[-1]-t2[0])/100,t2[-1]+np.abs(t2[-1]-t2[0])/100)
ymax = np.amax(list_samples)
ymin = np.amin(list_samples)
self.subplot.set_ylim([ymin-np.abs(ymin)*0.1, ymax*1.2 + 2])
self.subplot.set_ylabel("abs(Sample(t)) und
abs(Peak(t)+Offset)-->",fontsize = 12)
self.subplot.set_xlabel("Zeit in Sek. -->",fontsize = 12)
Any ideas how to avoid the slow down of my cpu ?
regards Markus
A snippet of code does not help much.
Please try to post a small concise standalone example that we can run and test.
A general advise is to try to reduce the number of plot call, i.e.,
plot as may points as possible with a single plot call.
However, 50million points seems to be awful a lot.
6 inch x 6 inch figure with dpi=100 has 0.36 million number of pixels.
My guess is that it makes little sense to plot 50 million points here.
Anyhow, plotting 50million points with a single plot call dies with
some segfault error in my machine. So, I feel that matplotlib may not
be suitable for your task. But, John or others may have some insight
how to deal with.
Regards,
-JJ
On Tue, Jun 30, 2009 at 1:22 PM, Markus Feldmann<fel...@gm...> wrote:
> Hi All,
>
> my program lets slow down my cpu. This only appears if i plot to much
> points. I am not sure how many point i need to get this, normally i plot
> 3*14e6 + 8e3, that is round about 50million points. My system is a
> dual core 2GHz cpu with 2Gbyte Ram.
>
> Here is my method to plot,
> def drawtransientall(self,min):
> self.subplot = self.figure.add_subplot(111)
> self.subplot.grid(True)
> list_t1,list_peaks,t2,list_samples =
> self.computetransientall(min,min+self.maxitems)
> offset = 0
> color = ['green','red','blue','magenta','cyan']
> markerPeaks = ['v','<','1','3','s']
> markerSamples = ['^','>','2','4','p']
> self.plots=[[],[]]
> for i in range(len(self.showBands)):
> self.plots[0] +=
> self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i],
> linestyle='None')
> self.plots[1] +=
> self.subplot.plot(t2,list_samples[i]+offset,color=color[i],
>
> marker=markerSamples[i],linestyle='None')
> offset +=1
>
> self.subplot.set_xlim(t2[0]-np.abs(t2[-1]-t2[0])/100,t2[-1]+np.abs(t2[-1]-t2[0])/100)
> ymax = np.amax(list_samples)
> ymin = np.amin(list_samples)
> self.subplot.set_ylim([ymin-np.abs(ymin)*0.1, ymax*1.2 + 2])
> self.subplot.set_ylabel("abs(Sample(t)) und
> abs(Peak(t)+Offset)-->",fontsize = 12)
> self.subplot.set_xlabel("Zeit in Sek. -->",fontsize = 12)
>
> Any ideas how to avoid the slow down of my cpu ?
>
> regards Markus
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
I agree with Jae-Joon here -- try to reduce the number of points before
passing it to matplotlib.
However, I'm a little concerned about the segfault -- I'd rather
matplotlib give a MemoryError exception if that's in fact what is
happening. Jae-Joon -- can you share your test that causes the segfault?
The snippet below completely hogs my machine for a few minutes, but
then, correctly, aborts with a MemoryError.
This is on FC11 i586, Python 2.6, Numpy 1.3.
====
from matplotlib.pyplot import *
import numpy as np
points = np.random.random((50000000, 2))
plot(points)
show()
====
Mike
On 07/01/2009 01:34 PM, Jae-Joon Lee wrote:
> A snippet of code does not help much.
> Please try to post a small concise standalone example that we can run and test.
>
> A general advise is to try to reduce the number of plot call, i.e.,
> plot as may points as possible with a single plot call.
>
> However, 50million points seems to be awful a lot.
> 6 inch x 6 inch figure with dpi=100 has 0.36 million number of pixels.
> My guess is that it makes little sense to plot 50 million points here.
>
> Anyhow, plotting 50million points with a single plot call dies with
> some segfault error in my machine. So, I feel that matplotlib may not
> be suitable for your task. But, John or others may have some insight
> how to deal with.
>
> Regards,
>
> -JJ
>
>
>
> On Tue, Jun 30, 2009 at 1:22 PM, Markus Feldmann<fel...@gm...> wrote:
>
>> Hi All,
>>
>> my program lets slow down my cpu. This only appears if i plot to much
>> points. I am not sure how many point i need to get this, normally i plot
>> 3*14e6 + 8e3, that is round about 50million points. My system is a
>> dual core 2GHz cpu with 2Gbyte Ram.
>>
>> Here is my method to plot,
>> def drawtransientall(self,min):
>> self.subplot = self.figure.add_subplot(111)
>> self.subplot.grid(True)
>> list_t1,list_peaks,t2,list_samples =
>> self.computetransientall(min,min+self.maxitems)
>> offset = 0
>> color = ['green','red','blue','magenta','cyan']
>> markerPeaks = ['v','<','1','3','s']
>> markerSamples = ['^','>','2','4','p']
>> self.plots=[[],[]]
>> for i in range(len(self.showBands)):
>> self.plots[0] +=
>> self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i],
>> linestyle='None')
>> self.plots[1] +=
>> self.subplot.plot(t2,list_samples[i]+offset,color=color[i],
>>
>> marker=markerSamples[i],linestyle='None')
>> offset +=1
>>
>> self.subplot.set_xlim(t2[0]-np.abs(t2[-1]-t2[0])/100,t2[-1]+np.abs(t2[-1]-t2[0])/100)
>> ymax = np.amax(list_samples)
>> ymin = np.amin(list_samples)
>> self.subplot.set_ylim([ymin-np.abs(ymin)*0.1, ymax*1.2 + 2])
>> self.subplot.set_ylabel("abs(Sample(t)) und
>> abs(Peak(t)+Offset)-->",fontsize = 12)
>> self.subplot.set_xlabel("Zeit in Sek. -->",fontsize = 12)
>>
>> Any ideas how to avoid the slow down of my cpu ?
>>
>> regards Markus
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Matplotlib-users mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
On Wed, Jul 1, 2009 at 2:34 PM, Michael Droettboom<md...@st...> wrote:
> I agree with Jae-Joon here -- try to reduce the number of points before
> passing it to matplotlib.
>
> However, I'm a little concerned about the segfault -- I'd rather matplotlib
> give a MemoryError exception if that's in fact what is happening. Jae-Joon
> -- can you share your test that causes the segfault?
>
> The snippet below completely hogs my machine for a few minutes, but then,
> correctly, aborts with a MemoryError.
>
> This is on FC11 i586, Python 2.6, Numpy 1.3.
>
> ====
>
> from matplotlib.pyplot import *
> import numpy as np
>
> points = np.random.random((50000000, 2))
> plot(points)
> show()
>
Yes, I also got MemoryError in this case during the plot() call.
But I got segfault for the code below.
x=random(50e6)
y=random(50e6)
plt.plot(x, y)
plt.show()
In this case, plot() runs fine, but segfault during show().
The segfault happens in the _path_module::affine_transform method of
src/_path.cpp.
I wonder if you can reproduce this.
-JJ
> ====
>
> Mike
>
> On 07/01/2009 01:34 PM, Jae-Joon Lee wrote:
>
> A snippet of code does not help much.
> Please try to post a small concise standalone example that we can run and
> test.
>
> A general advise is to try to reduce the number of plot call, i.e.,
> plot as may points as possible with a single plot call.
>
> However, 50million points seems to be awful a lot.
> 6 inch x 6 inch figure with dpi=100 has 0.36 million number of pixels.
> My guess is that it makes little sense to plot 50 million points here.
>
> Anyhow, plotting 50million points with a single plot call dies with
> some segfault error in my machine. So, I feel that matplotlib may not
> be suitable for your task. But, John or others may have some insight
> how to deal with.
>
> Regards,
>
> -JJ
>
>
>
> On Tue, Jun 30, 2009 at 1:22 PM, Markus Feldmann<fel...@gm...>
> wrote:
>
>
> Hi All,
>
> my program lets slow down my cpu. This only appears if i plot to much
> points. I am not sure how many point i need to get this, normally i plot
> 3*14e6 + 8e3, that is round about 50million points. My system is a
> dual core 2GHz cpu with 2Gbyte Ram.
>
> Here is my method to plot,
> def drawtransientall(self,min):
> self.subplot = self.figure.add_subplot(111)
> self.subplot.grid(True)
> list_t1,list_peaks,t2,list_samples =
> self.computetransientall(min,min+self.maxitems)
> offset = 0
> color = ['green','red','blue','magenta','cyan']
> markerPeaks = ['v','<','1','3','s']
> markerSamples = ['^','>','2','4','p']
> self.plots=[[],[]]
> for i in range(len(self.showBands)):
> self.plots[0] +=
> self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i],
> linestyle='None')
> self.plots[1] +=
> self.subplot.plot(t2,list_samples[i]+offset,color=color[i],
>
> marker=markerSamples[i],linestyle='None')
> offset +=1
>
> self.subplot.set_xlim(t2[0]-np.abs(t2[-1]-t2[0])/100,t2[-1]+np.abs(t2[-1]-t2[0])/100)
> ymax = np.amax(list_samples)
> ymin = np.amin(list_samples)
> self.subplot.set_ylim([ymin-np.abs(ymin)*0.1, ymax*1.2 + 2])
> self.subplot.set_ylabel("abs(Sample(t)) und
> abs(Peak(t)+Offset)-->",fontsize = 12)
> self.subplot.set_xlabel("Zeit in Sek. -->",fontsize = 12)
>
> Any ideas how to avoid the slow down of my cpu ?
>
> regards Markus
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
I tracked this down do line 962 of the _path.cpp.
double* vertex_out = (double*)PyArray_DATA(result);
My guess is that PyArray_SimpleNew at line 957 returns NULL for a
memory error instead of raising an exception, which makes result=NULL
and causes a segfault at line 962.
Since I'm not an c++ expert, I'll leave it to you, Michael.
Regards,
-JJ
On Wed, Jul 1, 2009 at 3:16 PM, Jae-Joon Lee<lee...@gm...> wrote:
> On Wed, Jul 1, 2009 at 2:34 PM, Michael Droettboom<md...@st...> wrote:
>> I agree with Jae-Joon here -- try to reduce the number of points before
>> passing it to matplotlib.
>>
>> However, I'm a little concerned about the segfault -- I'd rather matplotlib
>> give a MemoryError exception if that's in fact what is happening. Jae-Joon
>> -- can you share your test that causes the segfault?
>>
>> The snippet below completely hogs my machine for a few minutes, but then,
>> correctly, aborts with a MemoryError.
>>
>> This is on FC11 i586, Python 2.6, Numpy 1.3.
>>
>> ====
>>
>> from matplotlib.pyplot import *
>> import numpy as np
>>
>> points = np.random.random((50000000, 2))
>> plot(points)
>> show()
>>
>
> Yes, I also got MemoryError in this case during the plot() call.
>
> But I got segfault for the code below.
>
> x=random(50e6)
> y=random(50e6)
> plt.plot(x, y)
> plt.show()
>
> In this case, plot() runs fine, but segfault during show().
>
> The segfault happens in the _path_module::affine_transform method of
> src/_path.cpp.
>
> I wonder if you can reproduce this.
>
> -JJ
>
>
>> ====
>>
>> Mike
>>
>> On 07/01/2009 01:34 PM, Jae-Joon Lee wrote:
>>
>> A snippet of code does not help much.
>> Please try to post a small concise standalone example that we can run and
>> test.
>>
>> A general advise is to try to reduce the number of plot call, i.e.,
>> plot as may points as possible with a single plot call.
>>
>> However, 50million points seems to be awful a lot.
>> 6 inch x 6 inch figure with dpi=100 has 0.36 million number of pixels.
>> My guess is that it makes little sense to plot 50 million points here.
>>
>> Anyhow, plotting 50million points with a single plot call dies with
>> some segfault error in my machine. So, I feel that matplotlib may not
>> be suitable for your task. But, John or others may have some insight
>> how to deal with.
>>
>> Regards,
>>
>> -JJ
>>
>>
>>
>> On Tue, Jun 30, 2009 at 1:22 PM, Markus Feldmann<fel...@gm...>
>> wrote:
>>
>>
>> Hi All,
>>
>> my program lets slow down my cpu. This only appears if i plot to much
>> points. I am not sure how many point i need to get this, normally i plot
>> 3*14e6 + 8e3, that is round about 50million points. My system is a
>> dual core 2GHz cpu with 2Gbyte Ram.
>>
>> Here is my method to plot,
>> def drawtransientall(self,min):
>> self.subplot = self.figure.add_subplot(111)
>> self.subplot.grid(True)
>> list_t1,list_peaks,t2,list_samples =
>> self.computetransientall(min,min+self.maxitems)
>> offset = 0
>> color = ['green','red','blue','magenta','cyan']
>> markerPeaks = ['v','<','1','3','s']
>> markerSamples = ['^','>','2','4','p']
>> self.plots=[[],[]]
>> for i in range(len(self.showBands)):
>> self.plots[0] +=
>> self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i],
>> linestyle='None')
>> self.plots[1] +=
>> self.subplot.plot(t2,list_samples[i]+offset,color=color[i],
>>
>> marker=markerSamples[i],linestyle='None')
>> offset +=1
>>
>> self.subplot.set_xlim(t2[0]-np.abs(t2[-1]-t2[0])/100,t2[-1]+np.abs(t2[-1]-t2[0])/100)
>> ymax = np.amax(list_samples)
>> ymin = np.amin(list_samples)
>> self.subplot.set_ylim([ymin-np.abs(ymin)*0.1, ymax*1.2 + 2])
>> self.subplot.set_ylabel("abs(Sample(t)) und
>> abs(Peak(t)+Offset)-->",fontsize = 12)
>> self.subplot.set_xlabel("Zeit in Sek. -->",fontsize = 12)
>>
>> Any ideas how to avoid the slow down of my cpu ?
>>
>> regards Markus
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Matplotlib-users mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Matplotlib-users mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>>
>
This should now be fixed on the maintenance branch and trunk. A Numpy
array allocation was not being NULL-checked in _path.cpp:affine_transform.
I know a MemoryError doesn't help the user much more than a segfault,
but it always makes me feel better to get a real Python exception rather
than exploding ;)
Mike
On 07/01/2009 03:16 PM, Jae-Joon Lee wrote:
> On Wed, Jul 1, 2009 at 2:34 PM, Michael Droettboom<md...@st...> wrote:
>
>> I agree with Jae-Joon here -- try to reduce the number of points before
>> passing it to matplotlib.
>>
>> However, I'm a little concerned about the segfault -- I'd rather matplotlib
>> give a MemoryError exception if that's in fact what is happening. Jae-Joon
>> -- can you share your test that causes the segfault?
>>
>> The snippet below completely hogs my machine for a few minutes, but then,
>> correctly, aborts with a MemoryError.
>>
>> This is on FC11 i586, Python 2.6, Numpy 1.3.
>>
>> ====
>>
>> from matplotlib.pyplot import *
>> import numpy as np
>>
>> points = np.random.random((50000000, 2))
>> plot(points)
>> show()
>>
>>
>
> Yes, I also got MemoryError in this case during the plot() call.
>
> But I got segfault for the code below.
>
> x=random(50e6)
> y=random(50e6)
> plt.plot(x, y)
> plt.show()
>
> In this case, plot() runs fine, but segfault during show().
>
> The segfault happens in the _path_module::affine_transform method of
> src/_path.cpp.
>
> I wonder if you can reproduce this.
>
> -JJ
>
>
>
>> ====
>>
>> Mike
>>
>> On 07/01/2009 01:34 PM, Jae-Joon Lee wrote:
>>
>> A snippet of code does not help much.
>> Please try to post a small concise standalone example that we can run and
>> test.
>>
>> A general advise is to try to reduce the number of plot call, i.e.,
>> plot as may points as possible with a single plot call.
>>
>> However, 50million points seems to be awful a lot.
>> 6 inch x 6 inch figure with dpi=100 has 0.36 million number of pixels.
>> My guess is that it makes little sense to plot 50 million points here.
>>
>> Anyhow, plotting 50million points with a single plot call dies with
>> some segfault error in my machine. So, I feel that matplotlib may not
>> be suitable for your task. But, John or others may have some insight
>> how to deal with.
>>
>> Regards,
>>
>> -JJ
>>
>>
>>
>> On Tue, Jun 30, 2009 at 1:22 PM, Markus Feldmann<fel...@gm...>
>> wrote:
>>
>>
>> Hi All,
>>
>> my program lets slow down my cpu. This only appears if i plot to much
>> points. I am not sure how many point i need to get this, normally i plot
>> 3*14e6 + 8e3, that is round about 50million points. My system is a
>> dual core 2GHz cpu with 2Gbyte Ram.
>>
>> Here is my method to plot,
>> def drawtransientall(self,min):
>> self.subplot = self.figure.add_subplot(111)
>> self.subplot.grid(True)
>> list_t1,list_peaks,t2,list_samples =
>> self.computetransientall(min,min+self.maxitems)
>> offset = 0
>> color = ['green','red','blue','magenta','cyan']
>> markerPeaks = ['v','<','1','3','s']
>> markerSamples = ['^','>','2','4','p']
>> self.plots=[[],[]]
>> for i in range(len(self.showBands)):
>> self.plots[0] +=
>> self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i],
>> linestyle='None')
>> self.plots[1] +=
>> self.subplot.plot(t2,list_samples[i]+offset,color=color[i],
>>
>> marker=markerSamples[i],linestyle='None')
>> offset +=1
>>
>> self.subplot.set_xlim(t2[0]-np.abs(t2[-1]-t2[0])/100,t2[-1]+np.abs(t2[-1]-t2[0])/100)
>> ymax = np.amax(list_samples)
>> ymin = np.amin(list_samples)
>> self.subplot.set_ylim([ymin-np.abs(ymin)*0.1, ymax*1.2 + 2])
>> self.subplot.set_ylabel("abs(Sample(t)) und
>> abs(Peak(t)+Offset)-->",fontsize = 12)
>> self.subplot.set_xlabel("Zeit in Sek. -->",fontsize = 12)
>>
>> Any ideas how to avoid the slow down of my cpu ?
>>
>> regards Markus
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Matplotlib-users mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Matplotlib-users mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>>
>>
Michael Droettboom schrieb: > This should now be fixed on the maintenance branch and trunk. A Numpy > array allocation was not being NULL-checked in _path.cpp:affine_transform. > > I know a MemoryError doesn't help the user much more than a segfault, > but it always makes me feel better to get a real Python exception rather > than exploding ;) > > Mike Hi All, thanks for the bugfix. I also got the segfault, but forgot to wrote this. Is there a possibility to limit the maximum points shown ? And sorry that i can not post more of my code. My program is to big. Maybe this is a feature request. :-) It would be nice to set up a maximum limit and compute a average. I doesnt know which way is the fastest. I only execute the plot command three times but every plot command plots round about 15 Mill. points. regards Markus