matplotlib-users Mailing List for matplotlib

http://kamiasek.kx.cz/daqowrtau/669669.html

On 2012年07月07日 7:14 PM, Michiel de Hoon wrote:
> Hi,
>
> > What kind of outputs can these backends create?
>
> The Mac OS X backend can create PDFs, but it simply uses the pdf backend
> to do so, so that wouldn't help you.
> The cairo backend can create PDFs by using cairo, so that could be worth
> trying.
>
> > Could make a simple speed comparison between these backends
> > and the original script that uses the PDF backend.
>
> That would be useful, but keep in mind that there would be three options
> to compare:
> 1) The current PDF backend;
> 2) A modified PDF backend;
> 3) The cairo backend creating PDFs.
> Since we don't have 2) yet, we cannot do the full comparison yet, but
> still it would be good to know if it is faster to create PDFs by using
> cairo compared to the current PDF backend.
>
> > I am assuming the changes you mention require quite some work
> > to make the PDFbackend running faster.
>
> I think it is not so bad, since it's mainly a matter of removing the
> stuff from the PDF backend that is no longer needed. Do we have a
> maintainer for the PDF backend? Because I would rather rely on him/her
> to make the changes to this backend. Otherwise, I can give it a try, but
> probably I won't be able to find the time for it within this month.
>
It would be a good idea to enter a Github ticket for this, referring to 
this email thread.
Mike D. and Jouni S. have done most of the work on the pdf backend.
Eric
> Best,
> -Michiel.
>
>
>
> --- On *Sat, 7/7/12, Gökhan Sever /<gok...@gm...>/* wrote:
>
>
> From: Gökhan Sever <gok...@gm...>
> Subject: Re: [Matplotlib-users] Accelerating PDF saved plots
> To: "Michiel de Hoon" <mjl...@ya...>
> Cc: mat...@li...
> Date: Saturday, July 7, 2012, 9:05 PM
>
> Hi,
>
> What kind of outputs can these backends create? I don't use MAC, so
> my question is particularly for the Cairo backend. Could make a
> simple speed comparison between these backends and the original
> script that uses the PDF backend. I am assuming the changes you
> mention require quite some work to make the PDFbackend running faster.
>
> Thanks.
>
> On Sat, Jul 7, 2012 at 9:40 AM, Michiel de Hoon <mjl...@ya...
> </mc/compose?to=mjl...@ya...>> wrote:
>
> One reason behind the lengthy plot creation times is likely the
> PDF backend itself.
>
> Whereas the Mac OS X and the Cairo backends make use of new_gc
> and gc.restore to keep track of the graphics context, the PDF
> backend uses check_gc and an internal stack of graphics
> contexts. Since nowadays matplotlib has gc.restore
> functionality, I don't think that that is needed any more.
>
> See this revision for when gc.restore was added to matplotlib:
>
> http://matplotlib.svn.sourceforge.net/viewvc/matplotlib?view=revision&revision=7112
>
> In the same revision the Mac OS X and Cairo backends were
> modified to make use of gc.restore. The PDF backend (and the
> postscript backend also, btw) can be simplified in the same way
> to speed up these backends, as well as to reduce the output file
> sizes.
>
> Best,
> -Michiel.
>
> --- On *Thu, 7/5/12, Gökhan Sever /<gok...@gm...
> </mc/compose?to=gok...@gm...>>/* wrote:
>
>
> From: Gökhan Sever <gok...@gm...
> </mc/compose?to=gok...@gm...>>
> Subject: Re: [Matplotlib-users] Accelerating PDF saved plots
> To: "Benjamin Root" <ben...@ou...
> </mc/compose?to=ben...@ou...>>
> Cc: mat...@li...
> </mc/compose?to=mat...@li...>
> Date: Thursday, July 5, 2012, 2:11 PM
>
>
>
> 38 * 16 = 608
> 80 / 608 = 0.1316 seconds per plot
>
> At this point, I doubt you are going to get much more
> speed-ups. Glad to be of help!
>
> Fabrice -- Good suggestion! I should have thought of
> that given how much I use that technique in doing animation.
>
> Ben Root
>
>
> I am including profiled runs for the records --only first 10
> lines to keep e-mail shorter. Total times are longer
> comparing to the raw run -p executions. I believe profiled
> run has its own call overhead.
>
> I1 run -p test_speed.py
> 171889738 function calls (169109959 primitive calls) in
> 374.311 seconds
>
> Ordered by: internal time
>
> ncalls tottime percall cumtime percall
> filename:lineno(function)
> 4548012 34.583 0.000 34.583 0.000
> {numpy.core.multiarray.array}
> 1778401 21.012 0.000 46.227 0.000
> path.py:86(__init__)
> 521816 17.844 0.000 17.844 0.000
> artist.py:74(__init__)
> 2947090 15.432 0.000 15.432 0.000
> weakref.py:243(__init__)
> 1778401 9.515 0.000 9.515 0.000 {method 'all'
> of 'numpy.ndarray' objects}
> 13691669 8.654 0.000 8.654 0.000 {getattr}
> 1085280 8.550 0.000 17.629 0.000
> core.py:2749(_update_from)
> 1299904 7.809 0.000 76.060 0.000
> markers.py:115(_recache)
> 38 7.378 0.194 7.378 0.194 {gc.collect}
> 13564851 6.768 0.000 6.768 0.000 {isinstance}
>
>
>
>
> I1 run -p test_speed3.py
> 61658708 function calls (60685172 primitive calls) in
> 100.934 seconds
>
> Ordered by: internal time
>
> ncalls tottime percall cumtime percall
> filename:lineno(function)
> 937414 6.638 0.000 6.638 0.000
> {numpy.core.multiarray.array}
> 374227 4.377 0.000 7.500 0.000
> path.py:198(iter_segments)
> 6974613 3.866 0.000 3.866 0.000 {getattr}
> 542640 3.809 0.000 7.900 0.000
> core.py:2749(_update_from)
> 141361 3.665 0.000 7.136 0.000
> transforms.py:99(invalidate)
> 324688/161136 2.780 0.000 27.747 0.000
> transforms.py:1729(transform)
> 64448 2.753 0.000 64.921 0.001
> lines.py:463(draw)
> 231195 2.748 0.000 7.072 0.000
> path.py:86(__init__)
> 684970/679449 2.679 0.000 3.888 0.000
> backend_pdf.py:128(pdfRepr)
> 67526 2.651 0.000 7.522 0.000
> backend_pdf.py:1226(pathOperations)
>
>
>
> --
> Gökhan
>
> -----Inline Attachment Follows-----
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's
> security and
> threat landscape has changed and how IT managers can
> respond. Discussions
> will include endpoint security, mobile security and the
> latest in malware
> threats.
> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>
> -----Inline Attachment Follows-----
>
>
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> <http://mc/compose?to=Mat...@li...>
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
>
>
> --
> Gökhan
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>
>
>
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>

Michiel de Hoon <mjl...@ya...> writes:
> I think it is not so bad, since it's mainly a matter of removing the
> stuff from the PDF backend that is no longer needed. Do we have a
> maintainer for the PDF backend? Because I would rather rely on him/her
> to make the changes to this backend. 
That would be me. Can you outline what parts you think can be removed?
I'm currently travelling and don't always have an Internet connection,
or much time available, so turnaround can be slow.
-- 
Jouni K. Seppänen
http://www.iki.fi/jks

Hi,
> What kind of outputs can these backends create?
The Mac OS X backend can create PDFs, but it simply uses the pdf backend to do so, so that wouldn't help you.
The cairo backend can create PDFs by using cairo, so that could be worth trying.
> Could make a simple 
speed comparison between these backends
> and the original script that 
uses the PDF backend.
That would be useful, but keep in mind that there would be three options to compare:
1) The current PDF backend;
2) A modified PDF backend;
3) The cairo backend creating PDFs.
Since we don't have 2) yet, we cannot do the full comparison yet, but still it would be good to know if it is faster to create PDFs by using cairo compared to the current PDF backend.
> I am assuming the changes you mention require 
quite some work
> to make the PDFbackend running faster.
I think it is not so bad, since it's mainly a matter of removing the stuff from the PDF backend that is no longer needed. Do we have a maintainer for the PDF backend? Because I would rather rely on him/her to make the changes to this backend. Otherwise, I can give it a try, but probably I won't be able to find the time for it within this month.
Best,
-Michiel.
--- On Sat, 7/7/12, Gökhan Sever <gok...@gm...> wrote:
From: Gökhan Sever <gok...@gm...>
Subject: Re: [Matplotlib-users] Accelerating PDF saved plots
To: "Michiel de Hoon" <mjl...@ya...>
Cc: mat...@li...
Date: Saturday, July 7, 2012, 9:05 PM
Hi,
What kind of outputs can these backends create? I don't use MAC, so my question is particularly for the Cairo backend. Could make a simple speed comparison between these backends and the original script that uses the PDF backend. I am assuming the changes you mention require quite some work to make the PDFbackend running faster.
Thanks.
On Sat, Jul 7, 2012 at 9:40 AM, Michiel de Hoon <mjl...@ya...> wrote:
One reason behind the lengthy plot creation times is likely the PDF backend itself. 
Whereas the Mac OS X and the Cairo backends make use of new_gc and gc.restore to keep track of the graphics context, the PDF backend uses check_gc and an internal stack of graphics contexts. Since nowadays matplotlib has gc.restore functionality, I don't think that that is needed any more.
See this revision for when gc.restore was added to matplotlib:
http://matplotlib.svn.sourceforge.net/viewvc/matplotlib?view=revision&revision=7112
In the same revision the Mac OS X and Cairo backends were modified to make use of gc.restore. The PDF backend (and the postscript backend also, btw) can be simplified in the same way to speed up these backends, as well as to reduce the output file sizes.
Best,
-Michiel.
--- On Thu, 7/5/12, Gökhan
 Sever <gok...@gm...> wrote:
From: Gökhan Sever <gok...@gm...>
Subject: Re: [Matplotlib-users] Accelerating PDF saved plots
To: "Benjamin Root" <ben...@ou...>
Cc: mat...@li...
Date: Thursday, July 5, 2012, 2:11 PM
38 * 16 = 608
80 / 608 = 0.1316 seconds per plot
At this point, I doubt you are going to get much more speed-ups. Glad to be of help!
Fabrice -- Good suggestion! I should have thought of that given how much I use that technique in doing animation.
Ben Root
I am including profiled runs for the records --only first 10 lines to keep e-mail shorter. Total times are longer comparing to the raw run -p executions. I believe profiled run has its own call overhead.
I1 run -p test_speed.py
 171889738 function calls (169109959 primitive calls) in 374.311 seconds
  Ordered by: internal time
  ncalls tottime percall cumtime percall filename:lineno(function)
 4548012  34.583  0.000  34.583  0.000 {numpy.core.multiarray.array} 1778401  21.012  0.000  46.227  0.000 path.py:86(__init__)  521816  17.844  0.000  17.844  0.000 artist.py:74(__init__)
 2947090  15.432  0.000  15.432  0.000 weakref.py:243(__init__) 1778401  9.515  0.000  9.515  0.000 {method 'all' of 'numpy.ndarray' objects} 13691669  8.654  0.000  8.654  0.000 {getattr}
 1085280  8.550  0.000  17.629  0.000 core.py:2749(_update_from) 1299904  7.809  0.000  76.060  0.000 markers.py:115(_recache)    38  7.378  0.194  7.378  0.194 {gc.collect}
 13564851  6.768  0.000  6.768  0.000 {isinstance}
I1 run -p test_speed3.py 61658708 function calls (60685172 primitive calls) in 100.934 seconds
  Ordered by: internal time
  ncalls tottime percall cumtime percall filename:lineno(function)  937414  6.638  0.000  6.638  0.000 {numpy.core.multiarray.array}
  374227  4.377  0.000  7.500  0.000 path.py:198(iter_segments) 6974613  3.866  0.000  3.866  0.000 {getattr}  542640  3.809  0.000  7.900  0.000 core.py:2749(_update_from)
  141361  3.665  0.000  7.136  0.000 transforms.py:99(invalidate)324688/161136  2.780  0.000  27.747  0.000 transforms.py:1729(transform)  64448  2.753  0.000  64.921  0.001 lines.py:463(draw)
  231195  2.748  0.000  7.072  0.000 path.py:86(__init__)684970/679449  2.679  0.000  3.888  0.000 backend_pdf.py:128(pdfRepr)  67526  2.651  0.000  7.522  0.000 backend_pdf.py:1226(pathOperations)
-- 
Gökhan
-----Inline Attachment Follows-----
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
-----Inline Attachment Follows-----
_______________________________________________
Matplotlib-users mailing list
Mat...@li...
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
-- 
Gökhan

Hi,
What kind of outputs can these backends create? I don't use MAC, so my
question is particularly for the Cairo backend. Could make a simple speed
comparison between these backends and the original script that uses the PDF
backend. I am assuming the changes you mention require quite some work to
make the PDFbackend running faster.
Thanks.
On Sat, Jul 7, 2012 at 9:40 AM, Michiel de Hoon <mjl...@ya...> wrote:
> One reason behind the lengthy plot creation times is likely the PDF
> backend itself.
>
> Whereas the Mac OS X and the Cairo backends make use of new_gc and
> gc.restore to keep track of the graphics context, the PDF backend uses
> check_gc and an internal stack of graphics contexts. Since nowadays
> matplotlib has gc.restore functionality, I don't think that that is needed
> any more.
>
> See this revision for when gc.restore was added to matplotlib:
>
>
> http://matplotlib.svn.sourceforge.net/viewvc/matplotlib?view=revision&revision=7112
>
> In the same revision the Mac OS X and Cairo backends were modified to make
> use of gc.restore. The PDF backend (and the postscript backend also, btw)
> can be simplified in the same way to speed up these backends, as well as to
> reduce the output file sizes.
>
> Best,
> -Michiel.
>
> --- On *Thu, 7/5/12, Gökhan Sever <gok...@gm...>* wrote:
>
>
> From: Gökhan Sever <gok...@gm...>
> Subject: Re: [Matplotlib-users] Accelerating PDF saved plots
> To: "Benjamin Root" <ben...@ou...>
> Cc: mat...@li...
> Date: Thursday, July 5, 2012, 2:11 PM
>
>
>
> 38 * 16 = 608
> 80 / 608 = 0.1316 seconds per plot
>
> At this point, I doubt you are going to get much more speed-ups. Glad to
> be of help!
>
> Fabrice -- Good suggestion! I should have thought of that given how much
> I use that technique in doing animation.
>
> Ben Root
>
>
> I am including profiled runs for the records --only first 10 lines to keep
> e-mail shorter. Total times are longer comparing to the raw run -p
> executions. I believe profiled run has its own call overhead.
>
> I1 run -p test_speed.py
> 171889738 function calls (169109959 primitive calls) in 374.311 seconds
>
> Ordered by: internal time
>
> ncalls tottime percall cumtime percall filename:lineno(function)
> 4548012 34.583 0.000 34.583 0.000 {numpy.core.multiarray.array}
> 1778401 21.012 0.000 46.227 0.000 path.py:86(__init__)
> 521816 17.844 0.000 17.844 0.000 artist.py:74(__init__)
> 2947090 15.432 0.000 15.432 0.000 weakref.py:243(__init__)
> 1778401 9.515 0.000 9.515 0.000 {method 'all' of
> 'numpy.ndarray' objects}
> 13691669 8.654 0.000 8.654 0.000 {getattr}
> 1085280 8.550 0.000 17.629 0.000 core.py:2749(_update_from)
> 1299904 7.809 0.000 76.060 0.000 markers.py:115(_recache)
> 38 7.378 0.194 7.378 0.194 {gc.collect}
> 13564851 6.768 0.000 6.768 0.000 {isinstance}
>
>
>
>
> I1 run -p test_speed3.py
> 61658708 function calls (60685172 primitive calls) in 100.934 seconds
>
> Ordered by: internal time
>
> ncalls tottime percall cumtime percall filename:lineno(function)
> 937414 6.638 0.000 6.638 0.000 {numpy.core.multiarray.array}
> 374227 4.377 0.000 7.500 0.000 path.py:198(iter_segments)
> 6974613 3.866 0.000 3.866 0.000 {getattr}
> 542640 3.809 0.000 7.900 0.000 core.py:2749(_update_from)
> 141361 3.665 0.000 7.136 0.000 transforms.py:99(invalidate)
> 324688/161136 2.780 0.000 27.747 0.000
> transforms.py:1729(transform)
> 64448 2.753 0.000 64.921 0.001 lines.py:463(draw)
> 231195 2.748 0.000 7.072 0.000 path.py:86(__init__)
> 684970/679449 2.679 0.000 3.888 0.000
> backend_pdf.py:128(pdfRepr)
> 67526 2.651 0.000 7.522 0.000
> backend_pdf.py:1226(pathOperations)
>
>
>
> --
> Gökhan
>
> -----Inline Attachment Follows-----
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>
> -----Inline Attachment Follows-----
>
>
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...<http://mc/compose?to=Mat...@li...>
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
-- 
Gökhan