Hi all, Firstly, I love your software. It's great and largely it Just Works. However, I came across some weirdness pretty quickly also, without having to do anything particularly unusual. I wrote some code to generate a pair of lists about 300 elements long, and plotted them. I found that the line plot didn't have all the data points connected. When I saved to SVG, the output was correct. Also, when I tried to perform a scatter plot, I got runtime errors and no plot was drawn. These problems occur 100% of the time on my machine with my test files. I'm not sure if this list accepts files or not, but I am happy to provide the code and sample files which demonstrate this error, and/or respond to the bugfixing process. Cheers, -Tennessee
Tennessee, Maybe I missed it, but I did not see any reply to your message. We welcome simple scripts that expose bugs--and the simpler the better. If something more than a simple script is required, then instead of sending a big file to the list, you could put it on an ftp server if you have access to one, or send it to one of us off-list upon request. Tennessee Leeuwenburg wrote: [...] > I wrote some code to generate a pair of lists about 300 elements long, > and plotted them. I found that the line plot didn't have all the data > points connected. What backend were you using, and what operating system? What mpl version? Was the lack of connectedness at the level of a missing pixel, or was a whole line segment missing? > > When I saved to SVG, the output was correct. So it sounds like it must be a problem with a particular backend. > > Also, when I tried to perform a scatter plot, I got runtime errors and > no plot was drawn. These problems occur 100% of the time on my machine > with my test files. Until a recent change in SVN, scatter required an array, not a list, for its colors argument. Could this be the problem in your case? > > I'm not sure if this list accepts files or not, but I am happy to > provide the code and sample files which demonstrate this error, and/or > respond to the bugfixing process. Eric
I've sent Eric the files off-list. Anyone else who wants them, let me know. The bundle is about 400Kb -- not huge but not tiny. The script itself is small, it's just the data file and the output images which take up the space. The .png and onscreen renderer are both missing line segments, and behave in the same way. The .svg renderer behaves as expected. I'm running a very up-to-date linux, and I installed matplotlib newly about 4 days ago. Cheers, -T
Tennessee Leeuwenburg wrote: > I've sent Eric the files off-list. Anyone else who wants them, let me > know. The bundle is about 400Kb -- not huge but not tiny. The script > itself is small, it's just the data file and the output images which > take up the space. > > The .png and onscreen renderer are both missing line segments, and > behave in the same way. The .svg renderer behaves as expected. > > I'm running a very up-to-date linux, and I installed matplotlib newly > about 4 days ago. > > Cheers, > -T Tennessee, The first thing I notice is that you are trying to plot with log-log scales, but some of your y values are zero. Log(0) is undefined. The Agg renderers are doing something sensible: leaving out the 0 values. It is actually the ps and svg renderers that are incorrect, in that they seem to be substituting a value of 1 for each zero; they are bridging what *should* be gaps. Eric
On Mon, 2006年03月13日 at 20:08 -1000, Eric Firing wrote: > Tennessee Leeuwenburg wrote: > > I've sent Eric the files off-list. Anyone else who wants them, let me > > know. The bundle is about 400Kb -- not huge but not tiny. The script > > itself is small, it's just the data file and the output images which > > take up the space. > > > > The .png and onscreen renderer are both missing line segments, and > > behave in the same way. The .svg renderer behaves as expected. > > > > I'm running a very up-to-date linux, and I installed matplotlib newly > > about 4 days ago. > > > > Cheers, > > -T > > Tennessee, > > The first thing I notice is that you are trying to plot with log-log > scales, but some of your y values are zero. Log(0) is undefined. The > Agg renderers are doing something sensible: leaving out the 0 values. > It is actually the ps and svg renderers that are incorrect, in that they > seem to be substituting a value of 1 for each zero; they are bridging > what *should* be gaps. > > Eric That could certainly explain it. I don't know that I agree that it's correct -- I'm not graphing log(X), I'm graphing X, and 0 is a valid value for it. Only the scale is logarithmic, but it still contains a 0. I'll bow to a more expert opinion if there's disagreement. Cheers, -T
>>>>> "Tennessee" == Tennessee Leeuwenburg <ten...@te...> writes: Tennessee> That could certainly explain it. Tennessee> I don't know that I agree that it's correct -- I'm not Tennessee> graphing log(X), I'm graphing X, and 0 is a valid value Tennessee> for it. Only the scale is logarithmic, but it still Tennessee> contains a 0. I'll bow to a more expert opinion if Tennessee> there's disagreement. If you posst your script, we might be able to give a more informed opinion of what the desired output should be :-) I understand that there are some largish data files that you don't want to post, but if we could just read the script alone that might be enough. JDH
John Hunter wrote: >>>>>>"Tennessee" == Tennessee Leeuwenburg <ten...@te...> writes: > > Tennessee> That could certainly explain it. > > Tennessee> I don't know that I agree that it's correct -- I'm not > Tennessee> graphing log(X), I'm graphing X, and 0 is a valid value > Tennessee> for it. Only the scale is logarithmic, but it still > Tennessee> contains a 0. I'll bow to a more expert opinion if > Tennessee> there's disagreement. > > If you posst your script, we might be able to give a more informed > opinion of what the desired output should be :-) I understand that > there are some largish data files that you don't want to post, but if > we could just read the script alone that might be enough. > > JDH John, Here is a simple illustration of the problem and inconsistency, using just the first few points in Tennessee's plot and a few lines from his script. With a log scale for y, the Agg backends are leaving gaps (breaking the line) at the points where y==0; the PS and SVG backends are removing those points, but *not* breaking the line. I think the Agg behavior is much better. But maybe it would be better yet to raise an exception, forcing the user to deal with the error explicitly. For example, the user might want to change the zero values to a very small number, and then explicitly set the y range to exclude the small numbers. This is illustrated in the second subplot. I haven't looked into it, but I suspect the log(0) values are becoming nans, so what we are seeing is differences in nan-handling among the backends. Eric
>>>>> "Eric" == Eric Firing <ef...@ha...> writes:
Eric> Here is a simple illustration of the problem and
Eric> inconsistency, using just the first few points in
Eric> Tennessee's plot and a few lines from his script. With a
Eric> log scale for y, the Agg backends are leaving gaps (breaking
Eric> the line) at the points where y==0; the PS and SVG backends
Eric> are removing those points, but *not* breaking the line. I
Eric> think the Agg behavior is much better. But maybe it would
Eric> be better yet to raise an exception, forcing the user to
Eric> deal with the error explicitly. For example, the user might
Eric> want to change the zero values to a very small number, and
Eric> then explicitly set the y range to exclude the small
Eric> numbers. This is illustrated in the second subplot.
This issue has a long history, which I'll summarize here. In the old
days, we raised an exception when non-positive numbers were passed to
log plotting. Most people found this objectionable, and prefer to see
the valid points plotted rather than nothing (though a warning would
be nice). This is what matlab does. I needed to make some
architectural changes to support this, because the transformations
happened in the lines class which passed the transformed data to the
backend.
I looked into filtering the data in the lines class (basically what
you did for ma data) but did not think it would be fast enough,
because we potentially would have had to create a lot of line
segments.
So I changed the backend API and modified draw_lines to take the
transformation as an argument. Then the backend can do the following
(eg _backend_agg)
if (needNonlinear)
try {
mpltransform->nonlinear_only_api(&thisx, &thisy);
}
catch (...) {
moveto = true;
continue;
}
Ie if there is a nonlinear transformation that raises an exception,
the path state is changed from lineto to moveto. This is very fast,
at least an order of magnitude faster than trying to compute the
segments at the python level I suspect.
At the same time I introduced some optimizations for marker drawing
with a new backend method called draw_markers that also does the
transformations in the backend. This made marker drawing up to 25
times faster (we used to draw every marker as a separate function call
in lines.py)
We decided to support both old and new style APIs in the "transition
period" which made it easy for other backends to do ..... nothing. So
that is why the other backends haven't implemented this feature yet.
Steve worked on it a bit for Cairo and Darren and I both did for PS
but never got a working product. You can see the attempt in
backend_ps _draw_markers method (underscore hidden) and in
backend_cairo's draw_markers_OLD method. There was extensive
discussion on this list with the API and implementation details so
just search for draw_markers if you are interested.
So the backend is now in what I call a schizophrenic state, in two
ways.
1) some backends use the old and some use the "newstyle" API
2) Even in the newstyle API, some methods do the transformation in
the front end (draw_rectangle) and some in the backend
(draw_markers).
There is a need to simplify, unify and rationalize this API, but since
it mostly works, there has not been a lot of pressure to do so. And
it would break a lot of backends that may not be actively maintained.
A modest short term goal should be to get the newstyle draw_lines and
draw_markers working for the PS backend, both to fix this log problem
and because it is faster and potentially much smaller in terms of PS
file sizes. Something for a rainy day.
JDH
I'm playing with the new API in backend_ps today. I'm trying to make
draw_markers work:
On Tuesday 14 March 2006 1:38 pm, John Hunter wrote:
> >>>>> "Eric" == Eric Firing <ef...@ha...> writes:
>
> Eric> Here is a simple illustration of the problem and
> Eric> inconsistency, using just the first few points in
> Eric> Tennessee's plot and a few lines from his script. With a
> Eric> log scale for y, the Agg backends are leaving gaps (breaking
> Eric> the line) at the points where y==0; the PS and SVG backends
> Eric> are removing those points, but *not* breaking the line. I
> Eric> think the Agg behavior is much better. But maybe it would
> Eric> be better yet to raise an exception, forcing the user to
> Eric> deal with the error explicitly. For example, the user might
> Eric> want to change the zero values to a very small number, and
> Eric> then explicitly set the y range to exclude the small
> Eric> numbers. This is illustrated in the second subplot.
>
> This issue has a long history, which I'll summarize here. In the old
> days, we raised an exception when non-positive numbers were passed to
> log plotting. Most people found this objectionable, and prefer to see
> the valid points plotted rather than nothing (though a warning would
> be nice). This is what matlab does. I needed to make some
> architectural changes to support this, because the transformations
> happened in the lines class which passed the transformed data to the
> backend.
>
> I looked into filtering the data in the lines class (basically what
> you did for ma data) but did not think it would be fast enough,
> because we potentially would have had to create a lot of line
> segments.
>
> So I changed the backend API and modified draw_lines to take the
> transformation as an argument. Then the backend can do the following
> (eg _backend_agg)
>
> if (needNonlinear)
> try {
> mpltransform->nonlinear_only_api(&thisx, &thisy);
> }
> catch (...) {
> moveto = true;
> continue;
> }
>
> Ie if there is a nonlinear transformation that raises an exception,
> the path state is changed from lineto to moveto. This is very fast,
> at least an order of magnitude faster than trying to compute the
> segments at the python level I suspect.
For backend_ps, we want to make use of the transform's numerix methods.
if transform.need_nonlinear():
x,y,mask = transform.nonlinear_only_numerix(x, y, returnMask=1)
else:
mask = ones(x.shape)
x,y = transform.numerix_x_y(x, y)
If I do semilogy([0,1,2,3]), the mask will be [0,1,1,1]. If I do
semilogy([1,2,nan,4]), I would have expected a mask that looked like
[1,1,0,1], but it returns [1,1,1,1]. Wouldnt it make life easy if
transform.nonlinear_only_numerix and transform.numerix_x_y both returned a
mask with zeros where x or y is a nan or inf? Then I can filter out the
un-drawable markers with something like
to_draw = izip(x[start:end],y[start:end],mask[start:end])
ps = ['%1.3f %1.3f marker' % (xp, yp) for xp, yp, m in to_draw i m]
I could do something similar for draw_lines, using Johns suggestion above.
This would mask nans, infs and log(x<=0) without the need for masked arrays,
and would provide an opportunity to issue a warning about the bad values (if
that is actually desireable).
Darren
(There is a lot going on in the transforms that I'm still trying to wrap my
head around, so please feel free to correct me where my assumptions are
incorrect or naive.)
Darren Dale wrote: >For backend_ps, we want to make use of the transform's numerix methods. > > if transform.need_nonlinear(): > x,y,mask = transform.nonlinear_only_numerix(x, y, returnMask=1) > else: > mask = ones(x.shape) > x,y = transform.numerix_x_y(x, y) > >If I do semilogy([0,1,2,3]), the mask will be [0,1,1,1]. If I do >semilogy([1,2,nan,4]), I would have expected a mask that looked like >[1,1,0,1], but it returns [1,1,1,1]. > I just committed a patch that should make this work. nan support is done in a case-by-case basis at the C level. So this is another case. >Wouldnt it make life easy if >transform.nonlinear_only_numerix and transform.numerix_x_y both returned a >mask with zeros where x or y is a nan or inf? > I'm not sure this is the right thing to do for inf (or -inf) -- I can imagine transforms that accept inf and return something useful (such as a reciprocal transform). Maybe we don't use any such transforms currently, but that doesn't mean we should prevent that case in the future. Is the inf case important for you? Cheers! Andrew
On Sunday 19 March 2006 3:14 pm, Andrew Straw wrote: > Darren Dale wrote: > >For backend_ps, we want to make use of the transform's numerix methods. > > > > if transform.need_nonlinear(): > > x,y,mask =3D transform.nonlinear_only_numerix(x, y, > > returnMask=3D1) else: > > mask =3D ones(x.shape) > > x,y =3D transform.numerix_x_y(x, y) > > > >If I do semilogy([0,1,2,3]), the mask will be [0,1,1,1]. If I do > >semilogy([1,2,nan,4]), I would have expected a mask that looked like > >[1,1,0,1], but it returns [1,1,1,1]. > > I just committed a patch that should make this work. nan support is done > in a case-by-case basis at the C level. So this is another case. > > >Wouldnt it make life easy if > >transform.nonlinear_only_numerix and transform.numerix_x_y both returned= a > >mask with zeros where x or y is a nan or inf? > > I'm not sure this is the right thing to do for inf (or -inf) -- I can > imagine transforms that accept inf and return something useful (such as > a reciprocal transform). Maybe we don't use any such transforms > currently, but that doesn't mean we should prevent that case in the > future. Is the inf case important for you? I don't know. For a reciprocal transform, I suppose inf shouldn't effect th= e=20 mask, but then we can plot the results of the transform. If we can't plot t= he=20 transformed value, I think the mask should reflect it. Moving on, I've been trying to follow John's work in __draw_lines_hide (you= =20 masked that sucker twice, is it that bad?). Help me understand this, are we= =20 allowed to use transform.numerix_x_y in the new API, or do we have to use=20 transform.as_vec6_val? If I can use the former: =A0 =A0 =A0 =A0 print x,y =A0 =A0 =A0 =A0 if transform.need_nonlinear(): =A0 =A0 =A0 =A0 =A0 =A0 x,y,mask =3D transform.nonlinear_only_numerix(x, y,= returnMask=3D1) =A0 =A0 =A0 =A0 =A0 =A0 print x, y =A0 =A0 =A0 =A0 else: =A0 =A0 =A0 =A0 =A0 =A0 mask =3D ones(x.shape) =A0 =A0 =A0 =A0 x, y =3D transform.numerix_x_y(x, y) =A0 =A0 =A0 =A0 print x, y=20 that returns a "ValueError: Domain error on Transformation::numerix_x_y"=20 message when I try to do semilog plots. Linear plots look fine. If I should be using the latter, well, I'm playing with it, but I'm spillin= g=20 blue ink all over the place.
I think I actually got the new api working with draw_markers and draw_lines in
backend_ps. The transforms are done by passing the results of
transform.as_vec6_val to postscript. I would appreciate it if someone would
test it out and give me feedback. In order to do so, update from svn and
unmask backend_ps.RendererPS._draw_markers (not _draw_markers_old) by
removing the leading underscore. Try plotting things like
plot([0,1,2,3,4],'-o'); savefig('linear.eps')
plot([0,1,nan,3,4],'-o'); savefig('linear_nan.eps') # Not possible with the
old API
semilogy([0,1,2,3,4],'-o'); savefig('nonlinear.eps')
semilogy([0,1,nan,3,4],'-o'); savefig('nonlinear_nan.eps') # Not possible with
the old API
Are there other methods that need to be updated to use the new API? There are
a bunch of unused methods in RendererPS that I will clean up once things have
settled, please ignore them for now.
Darren
>>>>> "Darren" == Darren Dale <dd...@co...> writes:
Darren> I think I actually got the new api working with
Darren> draw_markers and draw_lines in backend_ps. The transforms
Darren> are done by passing the results of transform.as_vec6_val
Darren> to postscript. I would appreciate it if someone would test
Darren> it out and give me feedback. In order to do so, update
Darren> from svn and unmask backend_ps.RendererPS._draw_markers
Darren> (not _draw_markers_old) by removing the leading
Darren> underscore. Try plotting things like
I think the best way to get people to test this is to make it the
default in backend_ps -- then all svn users will be testing it
automatically and you will hear about it when there is a problem. They
don't call it "the bleeding edge" for nothing. I made this change in
svn, revision 2173.
Darren> plot([0,1,2,3,4],'-o'); savefig('linear.eps')
Darren> plot([0,1,nan,3,4],'-o'); savefig('linear_nan.eps') # Not
Darren> possible with the old API semilogy([0,1,2,3,4],'-o');
Darren> savefig('nonlinear.eps') semilogy([0,1,nan,3,4],'-o');
Darren> savefig('nonlinear_nan.eps') # Not possible with the old
Darren> API
Darren> Are there other methods that need to be updated to use the
Darren> new API? There are a bunch of unused methods in RendererPS
Darren> that I will clean up once things have settled, please
Darren> ignore them for now.
Just those two methods for now. I'd also like to add draw_paths and
remove many of the redundant ones.
Regarding a question from your previous post:
Darren> Moving on, I've been trying to follow John's work in
Darren> __draw_lines_hide (you masked that sucker twice, is it
Darren> that bad?). Help me understand this, are we allowed to use
Darren> transform.numerix_x_y in the new API, or do we have to use
Darren> transform.as_vec6_val? If I can use the former:
You are certainly allowed to use it, and any other provided method,
but for nonlinear transformations it defeats the purpose of doing the
transformation in the backend. numerix_x_y transforms the entire
array with the affine plus nonlinear transform. If it fails on any
element the entire transformation fails. To prevent this problem you
would want to do an element wise transformation and on failures set
the path state for the next element to moveto, ie skip the bad point.
We have talked in the past about writing a helper routine to generate
the postscript path in extension code if performance becomes an issue.
I ran a test script to profile this before and after, making marker
plots of various sizes. I think you'll be happy with the fruits of
your labors. Because we save the effort of setting a gc for each
marker, and thus avoid of lot of redundant function calling and state
setting in the output file, the results are much better in time and space
import time, os
from pylab import figure, close, nx
for i in 10,100,1000,10000,100000:
fig = figure(1)
ax = fig.add_subplot(111)
x,y = nx.mlab.rand(2,i)
tnow = time.time()
ax.plot(x,y, 'o')
fname = 'file%d.ps'
fig.savefig(fname)
elapsed = time.time() - tnow
fsize = os.stat(fname)[6]
print 'i=%06d time= %1.2fs size=%1.1fkb'%(i, elapsed, fsize/1e3)
close(1)
Old API:
> python test.py
i=000010 time= 0.33s size=142.4kb
i=000100 time= 0.30s size=154.9kb
i=001000 time= 0.43s size=284.1kb
i=010000 time= 1.79s size=1575.9kb
i=100000 time= 15.37s size=14495.6kb
New API:
> python test.py
i=000010 time= 0.53s size=148.1kb
i=000100 time= 0.29s size=146.5kb
i=001000 time= 0.30s size=163.6kb
i=010000 time= 0.44s size=334.5kb
i=100000 time= 2.17s size=2044.5kb
So for largish marker plots, you have about a 7x improvement in speed
and filesize. You may want to use this script to profile and see if
you can't eke out some extra gains. Maybe another 2x <wink>. This is
using the default rc file.
Now that we have a good implementation in a pure python backend that
others can follow as an example, I will shortly deprecate the old API
entirely and begin pruning the redundant methods. Considering that
both your changes and Eric's are significant enhancements, we may wait
to do this until one release cycle down the road, so that we don't
delay getting these out to the wider community.
Thanks,
JDH
On Monday 20 March 2006 11:05, John Hunter wrote: > I ran a test script to profile this before and after, making marker > plots of various sizes. I think you'll be happy with the fruits of > your labors. Because we save the effort of setting a gc for each > marker, and thus avoid of lot of redundant function calling and state > setting in the output file, the results are much better in time and space > > import time, os > from pylab import figure, close, nx > > for i in 10,100,1000,10000,100000: > fig = figure(1) > ax = fig.add_subplot(111) > x,y = nx.mlab.rand(2,i) > tnow = time.time() > ax.plot(x,y, 'o') > fname = 'file%d.ps' > fig.savefig(fname) > elapsed = time.time() - tnow > fsize = os.stat(fname)[6] > print 'i=%06d time= %1.2fs size=%1.1fkb'%(i, elapsed, fsize/1e3) > close(1) > > Old API: > > python test.py > > i=000010 time= 0.33s size=142.4kb > i=000100 time= 0.30s size=154.9kb > i=001000 time= 0.43s size=284.1kb > i=010000 time= 1.79s size=1575.9kb > i=100000 time= 15.37s size=14495.6kb > > New API: > > python test.py > > i=000010 time= 0.53s size=148.1kb > i=000100 time= 0.29s size=146.5kb > i=001000 time= 0.30s size=163.6kb > i=010000 time= 0.44s size=334.5kb > i=100000 time= 2.17s size=2044.5kb > > So for largish marker plots, you have about a 7x improvement in speed > and filesize. You may want to use this script to profile and see if > you can't eke out some extra gains. Maybe another 2x <wink>. This is > using the default rc file. > > Now that we have a good implementation in a pure python backend that > others can follow as an example, I will shortly deprecate the old API > entirely and begin pruning the redundant methods. Considering that > both your changes and Eric's are significant enhancements, we may wait > to do this until one release cycle down the road, so that we don't > delay getting these out to the wider community. I just committed a fix for a minor bug, it didnt effect the performance. My results are comparable to yours. How's this for a performance boost: set ps.useafm : true in rc: Old API: i=000010 time= 0.09s size=6.2kb i=000100 time= 0.07s size=18.0kb i=001000 time= 0.19s size=147.2kb i=010000 time= 1.43s size=1439.2kb i=100000 time= 13.99s size=14358.6kb New API: i=000010 time= 0.16s size=11.7kb i=000100 time= 0.07s size=9.6kb i=001000 time= 0.06s size=26.7kb i=010000 time= 0.13s size=197.7kb i=100000 time= 0.75s size=1907.7kb I dont see where I can make things any faster in draw_markers, we trimmed all the fat when we started on this a year ago. Here's a bug: I need isnan to create my mask. It is provided by numerix with numpy and numarray, but not Numeric. Can this be rectified? Darren
Darren Dale wrote: >Here's a bug: I need isnan to create my mask. It is provided by numerix with >numpy and numarray, but not Numeric. Can this be rectified? > > I just added the matplotlib._isnan extension module which is independent of the numerix choice (although I think it'll be better to stick with a numerix-given function, if available). Below is an example of its use. Perhaps you can modify the Numeric-flavor numerix so that isnan is exposed the same way as numarray and numpy -- I didn't do this because you'll be more familiar with the details than I am. #Example: import matplotlib._isnan as n import numpy for val in [3.2,3,numpy.nan,'adsf']: print 'val',val print n.isnan64(val) print Running displays the following: val 3.2 False val 3 False val nan True val adsf Traceback (most recent call last): File "testnan.py", line 6, in ? print n.isnan64(val) TypeError: a float is required
On Monday 20 March 2006 13:40, Andrew Straw wrote: > Darren Dale wrote: > >Here's a bug: I need isnan to create my mask. It is provided by numerix > > with numpy and numarray, but not Numeric. Can this be rectified? > > I just added the matplotlib._isnan extension module which is independent > of the numerix choice (although I think it'll be better to stick with a > numerix-given function, if available). Below is an example of its use. > Perhaps you can modify the Numeric-flavor numerix so that isnan is > exposed the same way as numarray and numpy -- I didn't do this because > you'll be more familiar with the details than I am. > > #Example: > import matplotlib._isnan as n > import numpy > > for val in [3.2,3,numpy.nan,'adsf']: > print 'val',val > print n.isnan64(val) > print Thank you, Andrew (Baker's Dozen) Straw. Does this look about right? def isnan(a): return reshape(array([isnan64(i) for i in ravel(a)],'b'), shape(a)) In [1]: a=ones((3,3,3),'d') In [2]: a[0,0,0]=array(0.0)/0 In [3]: isnan(a) Out[3]: [[[1,0,0,] [0,0,0,] [0,0,0,]] [[0,0,0,] [0,0,0,] [0,0,0,]] [[0,0,0,] [0,0,0,] [0,0,0,]]]
Darren Dale wrote: >Thank you, Andrew (Baker's Dozen) Straw. > > Hey, stop thanking me! ;) Seriously, it's me who should be thanking you for all the work you're doing on the PS and latex fronts. I'm just glad I can do a couple of minor things to help you along the way. >Does this look about right? > > >def isnan(a): > return reshape(array([isnan64(i) for i in ravel(a)],'b'), shape(a)) > > It looks fine to me -- it matches the behavior of numpy's isnan.
On Monday 20 March 2006 16:45, you wrote:
> Darren Dale wrote:
> >Thank you, Andrew (Baker's Dozen) Straw.
>
> Hey, stop thanking me! ;) Seriously, it's me who should be thanking you
> for all the work you're doing on the PS and latex fronts. I'm just glad
> I can do a couple of minor things to help you along the way.
>
> >Does this look about right?
> >
> >
> >def isnan(a):
> > return reshape(array([isnan64(i) for i in ravel(a)],'b'), shape(a))
>
> It looks fine to me -- it matches the behavior of numpy's isnan.
I modified _nc_imports.py and numerix/__init__.py. As of svn 2179, we have an
isnan for every numerix flavor (numpy and numarray use their own). That means
that the postscript backend can use nan's to mask points, if you use the new
API.
I also found a way to patch _backend_agg.cpp to make it break lines around
nan's. I havent modified extension code before, and this change would affect
all the *agg backends, so I dont want to commit before checking. Here's the
diff:
Index: src/_backend_agg.cpp
===================================================================
--- src/_backend_agg.cpp (revision 2178)
+++ src/_backend_agg.cpp (working copy)
@@ -23,6 +23,7 @@
#include "_backend_agg.h"
#include "_transforms.h"
#include "mplutils.h"
+#include "MPL_isnan.h"
#include "swig_runtime.h"
@@ -1324,6 +1325,7 @@
double thisx, thisy;
bool moveto = true;
+ bool skippoint = false;
double heightd = height;
double lastx(-2.0), lasty(-2.0);
@@ -1339,9 +1341,15 @@
}
catch (...) {
moveto = true;
+ skippoint = true;
continue;
}
-
+ else
+ if (MPL_isnan64(thisx) || MPL_isnan64(thisy)) {
+ moveto = true;
+ skippoint = true;
+ }
+
//use agg's transformer?
xytrans.transform(&thisx, &thisy);
thisy = heightd - thisy; //flipy
@@ -1367,8 +1375,9 @@
path.move_to(thisx, thisy);
else
path.line_to(thisx, thisy);
-
- moveto = false;
+ if (!skippoint)
+ moveto = false;
+ skippoint = false;
//std::cout << "draw lines " << thisx << " " << thisy << std::endl;
}
>>>>> "Darren" == Darren Dale <dd...@co...> writes: Darren> I havent modified extension code before, and this change Darren> would affect all the *agg backends, so I dont want to Darren> commit before checking. Here is a quick checklist of things to consider before committing Agg changes 1) makes a plot that you can interact with 2) passes backend_driver.py screening for Agg 3) passes unit/memleak_hawaii3.py with no appreciable memory leak 4) does something useful.... If it satisfies these, in my view it is suitable for public consumption. JDH
>>>>> "Darren" == Darren Dale <dd...@co...> writes:
Darren> double lastx(-2.0), lasty(-2.0); @@ -1339,9 +1341,15 @@
Darren> }
Darren> catch (...) { moveto = true; + skippoint = true;
Darren> continue;
Darren> }
Darren> - + else + if (MPL_isnan64(thisx) || MPL_isnan64(thisy)) {
Darren> + moveto = true; + skippoint = true;
I don't think you need skippoint. A combination of setting "moveto"
with "continue" should suffice. continue implicitly skips the point,
and setting the moveto flag indicates to agg not to connect the
previous with the next point.
catch (...) {
moveto = true;
continue;
}
else
if (MPL_isnan64(thisx) || MPL_isnan64(thisy)) {
moveto = true;
continue;
}
JDH
On Monday 20 March 2006 17:57, John Hunter wrote:
> >>>>> "Darren" == Darren Dale <dd...@co...> writes:
>
> Darren> I havent modified extension code before, and this change
> Darren> would affect all the *agg backends, so I dont want to
> Darren> commit before checking.
>
> Here is a quick checklist of things to consider before committing Agg
> changes
>
> 1) makes a plot that you can interact with
check
> 2) passes backend_driver.py screening for Agg
check
> 3) passes unit/memleak_hawaii3.py with no appreciable memory leak
Average memory consumed per loop: -0.1637k bytes (? thats odd.)
> 4) does something useful....
It helped me procrastinate, that's sort of useful.
> If it satisfies these, in my view it is suitable for public consumption.
As of svn 2181, if you use a *Agg backend or the postscript backend with the
new API, the following script will yield a gap in the line, see attached. I
guess we need to decide if this is desireable behavior in general, I think it
is pretty useful myself.
a=arange(21, dtype='d')
a[10]=nan
plot(a, '-o')
savefig('nan_masked.png')
Darren Dale wrote: .... > As of svn 2181, if you use a *Agg backend or the postscript backend with the > new API, the following script will yield a gap in the line, see attached. I > guess we need to decide if this is desireable behavior in general, I think it > is pretty useful myself. Darren, Yes, this is *very* desireable behavior. It should be in all backends. If it were, then the masked array line plotting could take advantage of it, resulting in faster and simpler code. I can't think of any possible disadvantage to this behavior. Thanks for doing it. Eric
Darren Dale wrote: > As of svn 2181, if you use a *Agg backend or the postscript backend with the > new API, the following script will yield a gap in the line, see attached. I > guess we need to decide if this is desireable behavior +1 -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no...
On 3/20/06, Darren Dale <dd...@co...> wrote:
> On Monday 20 March 2006 17:57, John Hunter wrote:
> > >>>>> "Darren" =3D=3D Darren Dale <dd...@co...> writes:
> >
> > Darren> I havent modified extension code before, and this change
> > Darren> would affect all the *agg backends, so I dont want to
> > Darren> commit before checking.
> >
> > Here is a quick checklist of things to consider before committing Agg
> > changes
> >
> > 1) makes a plot that you can interact with
>
> check
>
> > 2) passes backend_driver.py screening for Agg
>
> check
>
> > 3) passes unit/memleak_hawaii3.py with no appreciable memory leak
>
> Average memory consumed per loop: -0.1637k bytes (? thats odd.)
>
> > 4) does something useful....
>
> It helped me procrastinate, that's sort of useful.
>
> > If it satisfies these, in my view it is suitable for public consumption=
.
>
> As of svn 2181, if you use a *Agg backend or the postscript backend with =
the
> new API, the following script will yield a gap in the line, see attached.=
I
> guess we need to decide if this is desireable behavior in general, I thin=
k it
> is pretty useful myself.
>
> a=3Darange(21, dtype=3D'd')
> a[10]=3Dnan
> plot(a, '-o')
> savefig('nan_masked.png')
>
This might not be related, but since when did bar charts of zero break?
bar(arange(10), zeros(10))
I have animation code, and this has been the starter code for a long
time. In the last few days this has broke though with a
ZeroDivisionError in ticker.py.
- Charlie
>>>>> "Charlie" == Charlie Moad <cw...@gm...> writes: Charlie> This might not be related, but since when did bar charts Charlie> of zero break? Charlie> bar(arange(10), zeros(10)) I think this is related to Eric's MaxNLocator support code. JDH