matplotlib-devel Mailing List for matplotlib

Michael Droettboom schrieb:
> Gael,
> 
> You raise a very good point about the duplication of code around. As a 
> case in point, the patches you provided no longer apply to the 
> "canonical" (or at least original) versions of the plugins that began 
> life in matplotlib. Recent versions of Sphinx have a proper extension 
> API, so that "try ... except" importing is no longer necessary.
> 
> For mathmpl.py --- I will move that into the matplotlib installed code 
> tree, so, as you suggest, "from matplotlib.sphinxext import mathmpl" 
> will work. Obviously, that won't really provide traction until the next 
> release of matplotlib. And it relies on only_directive.py, so I will 
> probably have to put that in matplotlib as well for now, though that is 
> a reasonable candidate for inclusion in Sphinx.
The "only" is so trivial that it will, in this or a more extended form,
certainly be part of 0.6, which will not come out later than Feb 31 ;)
> I have submitted inheritance_diagram.py to Sphinx already. There didn't 
> seem to be much interest, and there were some details (particularly how 
> it deals with files), that weren't "the Sphinx way", and there few 
> documents and/or examples etc. at the time about how to do it right. 
> Someone else ended up re-engineering it to use pygraphviz which felt 
> pretty heavy weight to me. So the thing kind of stalled. But it's 
> probably worth another push.
The mails are still in my "look at it again" folder. I kind of agree
that using pygraphviz is counterproductive, since many systems have
graphviz but no so many, I suspect, pygraphviz.
Is the current version of the inheritance_diagram.py the one in matplotlib?
cheers,
Georg

Gael Varoquaux schrieb:
> Hi all,
> 
> Sorry for the multiple posting, this concerns various groups, and I'd
> rather the information not be lost.
> 
> While working on getting our in-lab library ready to be merged with NiPy,
> I ran into some sort of 'sphinx extension mess' where various sphinx
> extension would have side effects on each other, and most important, the
> extensions did not work with sphinx trunk.
Which is of course kind of my fault, because I keep changing the API :)
But it must also be said that during 0.x, I tend to view cleanliness and
good code as more important than 100% backwards compatibility.
> I got the side effects to be limited by cleaning up the generated code
> from autosummary before each run: I added the following code in my
> sphinx conf.py:
> 
> ################################################################################
> # Hack: run the autosummary generation script 
> import shutil
> if os.path.exists('generated'):
> shutil.rmtree('generated')
> os.system('%s sphinxext/autosummary_generate.py -o generated *.rst' %
> sys.executable)
> ################################################################################
> 
> I am attaching a diff of all the modifications I made to get the various
> extensions to work. I hope you can use it to get your various extensions
> working on sphinx trunk quicker. For the NiPy guys, I will be committing
> these changes in the fff2 tree soon, and we can go over this at the
> sprint.
> 
> This does raise a problem: this extension code is all over the place, in
> various repository. Some of the code cannot live in the sphinx repo, as
> it introduces dependencies. However, as the extensions are not importable
> from Python (I can't do 'from matplotlib.sphinxext import mathmpl'), the
> different projects using them end up copying them in their repo, and thus
> there are several versions floating around not updated. Some of the
> extensions would do not add externa dependencies to sphinx. These should
> be pushed into sphinx, with tests. That way as sphinx evolves, they do
> not break.
I'm all for it. In the case of autosummary, I'm guilty of not getting it
in sooner. This will change soon. In other cases, I don't even know of
the extension, probably because those who write it deem it as too
project-specific to contribute it.
I don't ask for too much if an extension is contributed, so by all means
do at least post about your extensions!
Georg

Hi guys,
> You can now do:
>
> .. plot::
>
> from matplotlib.pyplot import *
> plot([1,2,3])
This is very nice - thank you for doing that.
But, thinking about the online tutorials, you often want to do
something as you can do with the sourcecode directive, as in:
.. testcode::
 import numpy as np
 print np.inf
Some text then
.. testoutput::
 inf
More text
.. testcode::
 # I still have the context from the blocks above
 print np.nan
More text
.. testoutput::
 nan
In that way, I can build up a tutorial, setting and calculating
variables, doing plots as I go, without having to recreate the
calculations at each step.
Is it possible to make the ..plot directive pick up the context in the same way?
Thanks a lot,
Matthew

On Tue, Feb 17, 2009 at 12:17:17AM +0100, Georg Brandl wrote:
> I'm all for it. In the case of autosummary, I'm guilty of not getting it
> in sooner. This will change soon. In other cases, I don't even know of
> the extension, probably because those who write it deem it as too
> project-specific to contribute it.
> I don't ask for too much if an extension is contributed, so by all means
> do at least post about your extensions!
I am not blaming anyone, just pointing out a non ideal situation. It has
already improved a lot with the matplotlib guys and the scipy guys
merging some changes in extensions and publishing the extensions in an
importable part of their source tree.
It is true that I'll be happier when I will be able to import the
only_directive, and the auto_summary from sphinx :).
Thanks for your great work,
Gaël

2009年2月16日 13:36:07 -0600, Robert Kern wrote:
[clip]
> At one point, some of us had a plan to keep all of these "scientific"
> extensions collected here:
> 
> http://sphinx.googlecode.com/svn/contrib/trunk/numpyext/
> 
> SVN-using projects could use svn:externals to include these in their
> projects without diverging the code. I really don't know why this plan
> changed. Pauli?
I think my reasons for moving the extensions to Scipy SVN was mainly that
(i) it seemed clearer to keep the code where it was mainly used,
also for bug tracking and mailing list purposes,
(ii) Sphinx's google-code SVN seemed quite dead and obscure, and was 
difficult to find,
(iii) svn:externals would work also against Numpy's repository.
A better thing to do might have been to rally people to put also their 
extensions there and press for better advertisement. We can of course 
still do this and revive the idea; it would be good to get a slightly 
bigger starting mass, though.
-- 
Pauli Virtanen

Sorry -- Ignore my last comment. I see the Scipy version already has 
the covered -- it's just in a different place. No problem.
Cheers,
Mike
Michael Droettboom wrote:
> Pauli Virtanen wrote:
> 
>> 2009年2月16日 13:26:40 -0500, Michael Droettboom wrote:
>> [clip]
>> 
>> 
>>> Anyway, the current version in matplotlib handles files in a way that
>>> behaves well with Sphinx (which I see is a TODO list item in the numpy
>>> version). It also uses the Sphinx extension API rather than the old and
>>> brittle way of defining directives etc. So those things will need to be
>>> merged with the other changes made on the Numpy side.
>>> 
>>> 
>> Ok, I'll try to get these merged now, starting from the Scipy
>> version.
>>
>> What was new:
>>
>> * Detect doctest format automatically.
>> * Matplotlib rcparameters settable in conf.py
>> * :include-source: also as conf.py option
>> * Preamble code in conf.py, to run in every plot
>> * Used Sphinx API to register the directive.
>> * Slightly modified template for the inserted RST, to make multiple
>> output figures floatable side-by-side in HTML.
>>
>> So I think what needs to be merged is the output file handling,
>> and to check that the output is similar as previously for the matplotlib
>> docs.
>> 
>> 
> Those all look like great features to have.
>
> Thanks for volunteering to do the merge.
>
> I would also add that it should be ported to the Sphinx extension API 
> (basically the stuff in the last section in the file). It's a lot 
> easier now, and should be more stable between docutils/Sphinx versions:
>
> http://sphinx.pocoo.org/ext/appapi.html
>
> Cheers,
> Mike
>
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Pauli Virtanen wrote:
> 2009年2月16日 13:26:40 -0500, Michael Droettboom wrote:
> [clip]
> 
>> Anyway, the current version in matplotlib handles files in a way that
>> behaves well with Sphinx (which I see is a TODO list item in the numpy
>> version). It also uses the Sphinx extension API rather than the old and
>> brittle way of defining directives etc. So those things will need to be
>> merged with the other changes made on the Numpy side.
>> 
>
> Ok, I'll try to get these merged now, starting from the Scipy
> version.
>
> What was new:
>
> * Detect doctest format automatically.
> * Matplotlib rcparameters settable in conf.py
> * :include-source: also as conf.py option
> * Preamble code in conf.py, to run in every plot
> * Used Sphinx API to register the directive.
> * Slightly modified template for the inserted RST, to make multiple
> output figures floatable side-by-side in HTML.
>
> So I think what needs to be merged is the output file handling,
> and to check that the output is similar as previously for the matplotlib
> docs.
> 
Those all look like great features to have.
Thanks for volunteering to do the merge.
I would also add that it should be ported to the Sphinx extension API 
(basically the stuff in the last section in the file). It's a lot 
easier now, and should be more stable between docutils/Sphinx versions:
http://sphinx.pocoo.org/ext/appapi.html
Cheers,
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Sun, Feb 15, 2009 at 08:04, Gael Varoquaux
<gae...@no...> wrote:
> Hi all,
>
> Sorry for the multiple posting, this concerns various groups, and I'd
> rather the information not be lost.
>
> While working on getting our in-lab library ready to be merged with NiPy,
> I ran into some sort of 'sphinx extension mess' where various sphinx
> extension would have side effects on each other, and most important, the
> extensions did not work with sphinx trunk.
At one point, some of us had a plan to keep all of these "scientific"
extensions collected here:
 http://sphinx.googlecode.com/svn/contrib/trunk/numpyext/
SVN-using projects could use svn:externals to include these in their
projects without diverging the code. I really don't know why this plan
changed. Pauli?
-- 
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco

2009年2月16日 13:26:40 -0500, Michael Droettboom wrote:
[clip]
> Anyway, the current version in matplotlib handles files in a way that
> behaves well with Sphinx (which I see is a TODO list item in the numpy
> version). It also uses the Sphinx extension API rather than the old and
> brittle way of defining directives etc. So those things will need to be
> merged with the other changes made on the Numpy side.
Ok, I'll try to get these merged now, starting from the Scipy
version.
What was new:
* Detect doctest format automatically.
* Matplotlib rcparameters settable in conf.py
* :include-source: also as conf.py option
* Preamble code in conf.py, to run in every plot
* Used Sphinx API to register the directive.
* Slightly modified template for the inserted RST, to make multiple
 output figures floatable side-by-side in HTML.
So I think what needs to be merged is the output file handling,
and to check that the output is similar as previously for the matplotlib
docs.
-- 
Pauli Virtanen

On Mon, Feb 16, 2009 at 01:26:40PM -0500, Michael Droettboom wrote:
> > The "official" version of the plot directive should IMO end up either
> > in Sphinx or matplotlib repository. It's probably OK to require matplotlib
> > SVN version to build Scipy docs for a while...
> I think it makes the most sense for this to be part of matplotlib, since 
> it fundamentally requires matplotlib -- that prevents Sphinx from 
> growing a matplotlib dependency.
Yeah, I am +1 for that too. 
Thank you for exposing the plot_directive in the MPL importable tree,
Michael. It is going to take a little while for a new MPL release to
trickle down to enough installed computers for people to be abe to throw
away their local copy, but this is clearly the good solution.
Gaël

Pauli Virtanen wrote:
> 2009年2月16日 10:30:47 -0500, Michael Droettboom wrote:
>
> 
>> A preliminary version of this is committed on the branch and trunk.
>>
>> You can now do:
>>
>> .. plot::
>>
>> from matplotlib.pyplot import *
>> plot([1,2,3])
>>
>> One open API question is whether to implicitly "from matplotlib.pyplot
>> import *" or "from matplotlib import pyplot as plt" or nothing. I
>> generally don't like "import *" or implicit things, so I've decided to
>> do none of the above. But I could see how it might be nice to reduce
>> the verbosity of these little code examples to import something be
>> default. Let me know if you feel strongly about changing it.
>>
>> It has also been moved to the installed source directory, so other
>> projects no longer need to copy it. Just change the entry in the
>> extensions list in the Sphinx conf.py from 'plot_directive' to
>> 'matplotlib.sphinxext.plot_directive' and remove your local copy of the
>> extension.
>> 
>
> Scipy's plot directive has had this feature for some time:
> 	
> 	http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/plot_directive.py
>
> plus some extra features. I think it should contain all the features in
> matplotlib's version; but OTOH, I haven't tried to build matplotlib docs
> using it.
> I think we really should merge it back *now*!
> (Re: Gael's thread; sorry, I've been lazy pushing the changes back...)
> 
We've clearly all been bad about forking this stuff all over the place. 
I was completely unaware it was being used for Scipy. It's unfortunate 
the suggestion to install this stuff as part of matplotlib wasn't made 
earlier. 
Anyway, the current version in matplotlib handles files in a way that 
behaves well with Sphinx (which I see is a TODO list item in the numpy 
version). It also uses the Sphinx extension API rather than the old and 
brittle way of defining directives etc. So those things will need to be 
merged with the other changes made on the Numpy side.
> The "official" version of the plot directive should IMO end up either
> in Sphinx or matplotlib repository. It's probably OK to require matplotlib
> SVN version to build Scipy docs for a while...
> 
I think it makes the most sense for this to be part of matplotlib, since 
it fundamentally requires matplotlib -- that prevents Sphinx from 
growing a matplotlib dependency.
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

2009年2月16日 10:30:47 -0500, Michael Droettboom wrote:
> A preliminary version of this is committed on the branch and trunk.
> 
> You can now do:
> 
> .. plot::
> 
> from matplotlib.pyplot import *
> plot([1,2,3])
> 
> One open API question is whether to implicitly "from matplotlib.pyplot
> import *" or "from matplotlib import pyplot as plt" or nothing. I
> generally don't like "import *" or implicit things, so I've decided to
> do none of the above. But I could see how it might be nice to reduce
> the verbosity of these little code examples to import something be
> default. Let me know if you feel strongly about changing it.
> 
> It has also been moved to the installed source directory, so other
> projects no longer need to copy it. Just change the entry in the
> extensions list in the Sphinx conf.py from 'plot_directive' to
> 'matplotlib.sphinxext.plot_directive' and remove your local copy of the
> extension.
Scipy's plot directive has had this feature for some time:
	
	http://svn.scipy.org/svn/numpy/trunk/doc/sphinxext/plot_directive.py
plus some extra features. I think it should contain all the features in
matplotlib's version; but OTOH, I haven't tried to build matplotlib docs
using it.
I think we really should merge it back *now*!
(Re: Gael's thread; sorry, I've been lazy pushing the changes back...)
The "official" version of the plot directive should IMO end up either
in Sphinx or matplotlib repository. It's probably OK to require matplotlib
SVN version to build Scipy docs for a while...
What do you think?
-- 
Pauli Virtanen

A preliminary version of this is committed on the branch and trunk.
You can now do:
.. plot::
 from matplotlib.pyplot import *
 plot([1,2,3])
One open API question is whether to implicitly "from matplotlib.pyplot 
import *" or "from matplotlib import pyplot as plt" or nothing. I 
generally don't like "import *" or implicit things, so I've decided to 
do none of the above. But I could see how it might be nice to reduce 
the verbosity of these little code examples to import something be 
default. Let me know if you feel strongly about changing it.
It has also been moved to the installed source directory, so other 
projects no longer need to copy it. Just change the entry in the 
extensions list in the Sphinx conf.py from 'plot_directive' to 
'matplotlib.sphinxext.plot_directive' and remove your local copy of the 
extension.
Mike
Michael Droettboom wrote:
> Has anyone else had a chance to look at this? It seems fairly 
> straightforward and I may have a go at this this morning if no one else 
> has already.
>
> Mike
>
> Fernando Perez wrote:
> 
>> Howdy,
>>
>> recently, Matthew Brett pointed out that reST supports a mode that's
>> very handy for writing tutorial-like documents that contain code
>> blocks including their output, and they can even be run as tests.
>> Here's a simple example with its corresponding source:
>>
>> http://neuroimaging.scipy.org/site/doc/manual/html/users/analysis_pipeline.html
>> http://neuroimaging.scipy.org/site/doc/manual/html/_sources/users/analysis_pipeline.txt
>>
>> and they can even use the MPL math support, as seen here:
>>
>> http://neuroimaging.scipy.org/site/doc/manual/html/users/glm_spec.html
>> http://neuroimaging.scipy.org/site/doc/manual/html/_sources/users/glm_spec.txt
>>
>> But we were thinking it would be great to have also plot support for
>> this, without being forced to use standalone scripts like in mpl's
>> current form of the plot directive. I unfortunately have to go now
>> and will be mostly offline for a week, but I just had a chat about
>> this with John, so I want to leave some context in here in case this
>> is of interest to you guys. If there's a discussion on the API, I'll
>> do my best to keep up, but I'm also cc'ing the nipy list so those
>> interested can pitch in (though we should keep the conversation to the
>> MPL list, where the plot machinery is implemented).
>>
>> Cheers,
>>
>> f
>>
>> ------------------------------------------------------------------------------
>> Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
>> software. With Adobe AIR, Ajax developers can use existing skills and code to
>> build responsive, highly engaging applications that combine the power of local
>> resources and data with the reach of the web. Download the Adobe AIR SDK and
>> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>> 
>> 
>
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Has anyone else had a chance to look at this? It seems fairly 
straightforward and I may have a go at this this morning if no one else 
has already.
Mike
Fernando Perez wrote:
> Howdy,
>
> recently, Matthew Brett pointed out that reST supports a mode that's
> very handy for writing tutorial-like documents that contain code
> blocks including their output, and they can even be run as tests.
> Here's a simple example with its corresponding source:
>
> http://neuroimaging.scipy.org/site/doc/manual/html/users/analysis_pipeline.html
> http://neuroimaging.scipy.org/site/doc/manual/html/_sources/users/analysis_pipeline.txt
>
> and they can even use the MPL math support, as seen here:
>
> http://neuroimaging.scipy.org/site/doc/manual/html/users/glm_spec.html
> http://neuroimaging.scipy.org/site/doc/manual/html/_sources/users/glm_spec.txt
>
> But we were thinking it would be great to have also plot support for
> this, without being forced to use standalone scripts like in mpl's
> current form of the plot directive. I unfortunately have to go now
> and will be mostly offline for a week, but I just had a chat about
> this with John, so I want to leave some context in here in case this
> is of interest to you guys. If there's a discussion on the API, I'll
> do my best to keep up, but I'm also cc'ing the nipy list so those
> interested can pitch in (though we should keep the conversation to the
> MPL list, where the plot machinery is implemented).
>
> Cheers,
>
> f
>
> ------------------------------------------------------------------------------
> Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and code to
> build responsive, highly engaging applications that combine the power of local
> resources and data with the reach of the web. Download the Adobe AIR SDK and
> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Gael,
You raise a very good point about the duplication of code around. As a 
case in point, the patches you provided no longer apply to the 
"canonical" (or at least original) versions of the plugins that began 
life in matplotlib. Recent versions of Sphinx have a proper extension 
API, so that "try ... except" importing is no longer necessary.
For mathmpl.py --- I will move that into the matplotlib installed code 
tree, so, as you suggest, "from matplotlib.sphinxext import mathmpl" 
will work. Obviously, that won't really provide traction until the next 
release of matplotlib. And it relies on only_directive.py, so I will 
probably have to put that in matplotlib as well for now, though that is 
a reasonable candidate for inclusion in Sphinx.
I have submitted inheritance_diagram.py to Sphinx already. There didn't 
seem to be much interest, and there were some details (particularly how 
it deals with files), that weren't "the Sphinx way", and there few 
documents and/or examples etc. at the time about how to do it right. 
Someone else ended up re-engineering it to use pygraphviz which felt 
pretty heavy weight to me. So the thing kind of stalled. But it's 
probably worth another push.
Mike
Gael Varoquaux wrote:
> Hi all,
>
> Sorry for the multiple posting, this concerns various groups, and I'd
> rather the information not be lost.
>
> While working on getting our in-lab library ready to be merged with NiPy,
> I ran into some sort of 'sphinx extension mess' where various sphinx
> extension would have side effects on each other, and most important, the
> extensions did not work with sphinx trunk.
>
> I got the side effects to be limited by cleaning up the generated code
> from autosummary before each run: I added the following code in my
> sphinx conf.py:
>
> ################################################################################
> # Hack: run the autosummary generation script 
> import shutil
> if os.path.exists('generated'):
> shutil.rmtree('generated')
> os.system('%s sphinxext/autosummary_generate.py -o generated *.rst' %
> sys.executable)
> ################################################################################
>
> I am attaching a diff of all the modifications I made to get the various
> extensions to work. I hope you can use it to get your various extensions
> working on sphinx trunk quicker. For the NiPy guys, I will be committing
> these changes in the fff2 tree soon, and we can go over this at the
> sprint.
>
> This does raise a problem: this extension code is all over the place, in
> various repository. Some of the code cannot live in the sphinx repo, as
> it introduces dependencies. However, as the extensions are not importable
> from Python (I can't do 'from matplotlib.sphinxext import mathmpl'), the
> different projects using them end up copying them in their repo, and thus
> there are several versions floating around not updated. Some of the
> extensions would do not add externa dependencies to sphinx. These should
> be pushed into sphinx, with tests. That way as sphinx evolves, they do
> not break.
>
> Gaël
> 
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a 600ドル discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> ------------------------------------------------------------------------
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

a writes:
> Michael Droettboom <mdroe@...> writes:
> >
> > Thanks for the pointers.
> >
> > The original simplification code was written by John Hunter (I believe),
> > and I don't know if it was designed by him also or is a replication of
> > something published elsewhere. So I take no credit for and have little
> > knowledge of its original goals.
>
> I'm not sure on everything it does, but it seems to do clipping and removes
> line segments where the change in slope is less than some limit. There are
> probably better algorithms out there, but this one works surprisingly well
> and is fast and simple. I think it should be a requirement that it returns
> points which are a subset of the original points- with the change you've
> made it does this, right?
Oh Hey! I'm the one who originally wrote the path simplification code. I'd
have thought it would be gone by now, but I am very happy it turned out to
be useful. I made it up in order to plot a very large set of noisy data I
had.
The goal was to simplify two types of plots at once: Smooth curves, as
well as very noisy data where many lines are 'on top' of each other. (eg
plot(rand(100000)) ). I noticed both could be taken care of by checking
for changes in slope.
An important goal (for me) was making sure that the min/max span of the
points plotted was preserved. (so that eg plot(rand(1000)) spans from the
lowest to highest point in the data (ie ~ 0 to 1) for any zoom factor).
I'm not sure if this property survived...: If you do plot(rand(1000)) with
the latest matplotlib and gradually zoom out on the x axis, you can see
the top/bottom tips of the plotted line flickering in height, which is
what I was trying to avoid. I forget whether I actually got it as I wanted
it though, maybe I gave up.
Allan