matplotlib-devel Mailing List for matplotlib (Page 3)

On 2010年12月09日 16:44:37 -0500, Ben Gamari <bg...@gm...> wrote:
> rcdefaults()'s implementation appears to implement the latter, updating
> rcParams from rcParamsDefault in rcsetup.py, which appears to describe
> the factory default values. Perhaps we should
> rcParamsDefault.update(rcParams) after loading matplotlibrc?
> 
As expected, doing the update of rcParamsDefault proposed above (patch
below) allows the examples.download setting to persist throughout the
documentation build. It seems like either the documentation build
process or rcParamsDefault has been badly broken for a very long
time. Is rcParamsDefault really supposed to be the factory defaults or
is this just a bug? If the former, we will need to introduce a variant
of matplotlib.rcdefaults() to reset the configuration to that specified
in matplotlibrc.
- Ben
diff --git a/lib/matplotlib/__init__.py b/lib/matplotlib/__init__.py
--- a/lib/matplotlib/__init__.py
+++ b/lib/matplotlib/__init__.py
@@ -764,13 +772,13 @@ Please do not ask for support with these customizations active.
 # this is the instance used by the matplotlib classes
 rcParams = rc_params()
-
-rcParamsDefault = RcParams([ (key, default) for key, (default, converter) in \
- defaultParams.iteritems() ])
-
 rcParams['ps.usedistiller'] = checkdep_ps_distiller(rcParams['ps.usedistiller'])
 rcParams['text.usetex'] = checkdep_usetex(rcParams['text.usetex'])
+rcParamsDefault = RcParams([ (key, default) for key, (default, converter) in \
+ defaultParams.iteritems() ])
+rcParamsDefault.update(rcParams)
+
 def rc(group, **kwargs):
 """
 Set the current rc params. Group is the grouping for the rc, eg.

Ben Gamari <bg...@gm...> writes:
> It seems that matplotlib's sphinx extension calls
> matplotlib.rcdefaults() after rendering each figure, thus resetting
> examples.download to True (as set in rcsetup.py).
Right, and it sets figure.figsize after it. I guess you will want to set
examples.download in the same function.
> The documentation claims that this function will "Restore the default rc
> params - the ones that were created at matplotlib load time."
> rcdefaults()'s implementation appears to implement the latter, updating
> rcParams from rcParamsDefault in rcsetup.py, which appears to describe
> the factory default values. Perhaps we should
> rcParamsDefault.update(rcParams) after loading matplotlibrc?
I think you're better off modifying the sphinx extension. rcdefaults has
a useful purpose, and the extension is trying to use it for that
purpose: to avoid having the user's configuration mess up the
documentation. If you were to update rcParamsDefault with the
matplotlibrc parameters, you might accidentally change the example
output.
-- 
Jouni K. Seppänen
http://www.iki.fi/jks

On 2010年12月09日 15:42:18 -0500, Ben Gamari <bg...@gm...> wrote:
> To try identifying where this happens I have tried putting some debug
> output in RcParams.__setitem__(). Unfortunately this doesn't show anyone
> setting examples.download after the initial loading of matplotlibrc. Any
> ideas how/where this might get overridden?
> 
Found it!
It seems that matplotlib's sphinx extension calls
matplotlib.rcdefaults() after rendering each figure, thus resetting
examples.download to True (as set in rcsetup.py).
The documentation claims that this function will "Restore the default rc
params - the ones that were created at matplotlib load time." Despite
the apparent attempt at clarification in this docstring, it's still not
clear what the "default rc params" actually are. Is this supposed to be
the parameters loaded from matplotlibrc (which seems to be what the
sphinx extension expects) or the factory default values?
rcdefaults()'s implementation appears to implement the latter, updating
rcParams from rcParamsDefault in rcsetup.py, which appears to describe
the factory default values. Perhaps we should
rcParamsDefault.update(rcParams) after loading matplotlibrc?
- Ben

This patch is for make.osx. Note it also changes the python version to 2.7.
https://gist.github.com/735337
Uri
...................................................................................
Uri Laserson
Graduate Student, Biomedical Engineering
Harvard-MIT Division of Health Sciences and Technology
M +1 917 742 8019
las...@mi...

On Thu, 9 Dec 2010 14:09:41 -0600, John Hunter <jd...@gm...> wrote:
> On Thu, Dec 9, 2010 at 1:52 PM, Ben Gamari <bg...@gm...> wrote:
> 
> > Any idea why my debugging messages are ineffective? Or, even better, any
> > idea why setting the examples.download in doc/matplotlibrc isn't being
> > reflected in rc_params? I'll examine this more closely in about 30
> > minutes once I make it to the office.
> 
> I am not sure. Perhaps it is working but in the confusion you never
> had everything set right at the same time. Try setting in
> doc/matplotlibrc
> 
> examples.download : False
> examples.directory : somedir/relative/to/doc
>
Already done.
 
> Then flush the sphinx build directory and try again. Perhaps stick
> some debug print in matplotlib.matplotlib_fname and
> cbook.get_sample_data and let us know.
I have already done this. This is how I know that rc_examples.download
is set to True in get_sample_data(). It turns out the debugging messages
I made reference to in my last message were being output and I just
missed them in the noise. Right after matplotlib is initialized with
matplotlib.rc_params() it seems that rcParams['examples.download'] is
set to False as expected. By the time execution makes it to
get_sample_data(), however, this value has somehow been set to True.
To try identifying where this happens I have tried putting some debug
output in RcParams.__setitem__(). Unfortunately this doesn't show anyone
setting examples.download after the initial loading of matplotlibrc. Any
ideas how/where this might get overridden?
- Ben

On Thu, Dec 9, 2010 at 1:52 PM, Ben Gamari <bg...@gm...> wrote:
> Any idea why my debugging messages are ineffective? Or, even better, any
> idea why setting the examples.download in doc/matplotlibrc isn't being
> reflected in rc_params? I'll examine this more closely in about 30
> minutes once I make it to the office.
I am not sure. Perhaps it is working but in the confusion you never
had everything set right at the same time. Try setting in
doc/matplotlibrc
 examples.download : False
 examples.directory : somedir/relative/to/doc
Then flush the sphinx build directory and try again. Perhaps stick
some debug print in matplotlib.matplotlib_fname and
cbook.get_sample_data and let us know.
Good luck!
JDH

On Thu, 9 Dec 2010 13:39:41 -0600, John Hunter <jd...@gm...> wrote:
> On Thu, Dec 9, 2010 at 1:12 PM, Ben Gamari <bg...@gm...> wrote:
> 
> > It seems that this may have been broken from the beginning. After
> > noticing that my changes were not taking effect, I added some debug
> > output and found that sphinx was changing the current working directory
> > (every example gets built with its own directory being the working
> > directory). I fail to see how doc/matplotlibrc could be currently used
> > given most examples are not built from doc/.
> 
> Well, the matplotlibrc file is checked for when the python process is
> started, and the current directory when matplotlib is imported is
> used. Since we launch the build process in the doc directory next to
> make.py, doc is the current working directory when the plot directive
> extension is imported and this is used to build the examples. So it
> is not the directory that the example is in that is important, but the
> current directory when matplotlib is imported.
> 
Ahh, so matplotlib is not reimported with every example? I was under the
impression sphinx operated more like make (1 process per target). I
suppose that explains it then.
> On import the matplotlib.rc_params function is called to set
> matplotlib.rcParams. The rc_params function in turn calls
> matplotlib_fname, which does the following search
> 
Any idea why my debugging messages are ineffective? Or, even better, any
idea why setting the examples.download in doc/matplotlibrc isn't being
reflected in rc_params? I'll examine this more closely in about 30
minutes once I make it to the office.
Cheers,
- Ben

On Thu, Dec 9, 2010 at 1:12 PM, Ben Gamari <bg...@gm...> wrote:
> It seems that this may have been broken from the beginning. After
> noticing that my changes were not taking effect, I added some debug
> output and found that sphinx was changing the current working directory
> (every example gets built with its own directory being the working
> directory). I fail to see how doc/matplotlibrc could be currently used
> given most examples are not built from doc/.
Well, the matplotlibrc file is checked for when the python process is
started, and the current directory when matplotlib is imported is
used. Since we launch the build process in the doc directory next to
make.py, doc is the current working directory when the plot directive
extension is imported and this is used to build the examples. So it
is not the directory that the example is in that is important, but the
current directory when matplotlib is imported.
On import the matplotlib.rc_params function is called to set
matplotlib.rcParams. The rc_params function in turn calls
matplotlib_fname, which does the following search
 * current working dir
 * environ var MATPLOTLIBRC
 * HOME/.matplotlib/matplotlibrc
 * MATPLOTLIBDATA/matplotlibrc
Hope this helps!
JDH

On Thu, 9 Dec 2010 12:57:49 -0600, John Hunter <jd...@gm...> wrote:
> On Thu, Dec 9, 2010 at 12:34 PM, Ben Gamari <bg...@gm...> wrote:
> 
> > Regardless, after making the necessary changes to use absolute paths
> > (after painfully realizing that the MATPLOTLIB environment variable
> > expects the directory of the desired matplotlibrc, not the file path;
> > documentation would have been nice here) the configuration is _still_
> > having no effect. While matplotlib.matplotlib_fname() seems to be
> > reporting the correct matplotlibrc path, examples.download is still True
> > despite setting this to False in the file.
> 
> Sorry to be joining this a bit late. Why do you need to set the
> MATPLOTLIBRC variable. The file doc/matplotlibrc is used by the doc
> build process, so if you patch in your changes there, you should be
> good.
It seems that this may have been broken from the beginning. After
noticing that my changes were not taking effect, I added some debug
output and found that sphinx was changing the current working directory
(every example gets built with its own directory being the working
directory). I fail to see how doc/matplotlibrc could be currently used
given most examples are not built from doc/.
> and these are picked up at build time since the rest table formatting
> of the API docs depends on the latter setting.
> 
Are you certain this configuration isn't leaking in from some other
source?
> And yes, the relative paths in sphinx doc builds are a major headache.
Are you referring to the fact that sphinx's above-mentioned chdir
behavior?
Cheers,
- Ben

On Thu, Dec 9, 2010 at 12:34 PM, Ben Gamari <bg...@gm...> wrote:
> Regardless, after making the necessary changes to use absolute paths
> (after painfully realizing that the MATPLOTLIB environment variable
> expects the directory of the desired matplotlibrc, not the file path;
> documentation would have been nice here) the configuration is _still_
> having no effect. While matplotlib.matplotlib_fname() seems to be
> reporting the correct matplotlibrc path, examples.download is still True
> despite setting this to False in the file.
Sorry to be joining this a bit late. Why do you need to set the
MATPLOTLIBRC variable. The file doc/matplotlibrc is used by the doc
build process, so if you patch in your changes there, you should be
good. Specifically, we set the parameters
 backend : Agg
 docstring.hardcopy : True
and these are picked up at build time since the rest table formatting
of the API docs depends on the latter setting.
And yes, the relative paths in sphinx doc builds are a major headache.
 Sorry you are having so much trouble. We struggled mightily against
them in trying to get the plot directive working. In my experience,
setting relative paths with respect to the doc root (where conf.py
lives) works in most cases.

On 2010年12月09日 20:00:39 +0200, Jouni K. Seppänen <jk...@ik...> wrote:
> Ben Gamari <bg...@gm...> writes:
> 
> > An absolute path is necesary because the current working directory
> > changes for each example.
> 
> Oh, right, the examples are in a lot of different subdirectories.
> 
Precisely, this is the problem.
Regardless, after making the necessary changes to use absolute paths
(after painfully realizing that the MATPLOTLIB environment variable
expects the directory of the desired matplotlibrc, not the file path;
documentation would have been nice here) the configuration is _still_
having no effect. While matplotlib.matplotlib_fname() seems to be
reporting the correct matplotlibrc path, examples.download is still True
despite setting this to False in the file.
I've placed a debug message writing to sys.stderr at the beginning of
matplotlib.rc_params(), but unfortunately I have seen no output from
it. Considering the rest of my debugging output works just fine, I'm
quite confused. Is this function not invoked?
Cheers,
- Ben

Ben Gamari <bg...@gm...> writes:
> An absolute path is necesary because the current working directory
> changes for each example.
Oh, right, the examples are in a lot of different subdirectories.
> I'm very confused how you found that specifying a relative path in
> download.path worked. If the cwd changes as seen above then the
> documentation was entirely correct in claiming an absolute path is
> necessary.
I just tested that the relative path works fine by making the path
relative and trying cbook.get_sample_data; it's just not much use if
your current working directory keeps changing.
-- 
Jouni K. Seppänen
http://www.iki.fi/jks

On 2010年12月09日 18:37:34 +0200, Jouni K. Seppänen <jk...@ik...> wrote:
> Ben Gamari <bga...@gm...> writes:
> 
> > That being said, I've tracked down the reason for the configuration in
> > doc/matplotlibrc not being used. It seems that sphinx runs the examples
> > in their current directory (I suppose this is to be expected).
> >
> > It seems that there is a (as far as I can tell undocumented, please fix
> > this) MATPLOTLIBRC environment variable which can be used to override
> > the location of the matplotlibrc used. This could be useful in
> > specifying the matplotlibrc used during the documentation build process
> > but unfortunately this would require an absolute path which, again,
> > complicates the packaging (i.e. I don't know the best way to accomplish
> > this). Suggestions?
> 
> I don't see why the path must be absolute, though I haven't tried. Is
> there some problem having a different global matplotlibrc
> $HOME/.matplotlib/matplotlibrc or $MATPLOTLIBDATA/matplotlibrc,
> possibly setting $HOME or $MATPLOTLIBDATA from your build script?
An absolute path is necesary because the current working directory
changes for each example. To check this I added,
 import sys
 sys.stderr.write('SAMPLE cwd=%s\n'%os.getcwd())
 sys.stderr.write('SAMPLE matplotlibrc=%s\n'%matplotlib.matplotlib_fname())
to the beginning of cbook.get_sample_data(). During the documentation
build I then saw output like,
reading sources... [ 12%] examples/api/date_demo
SAMPLE cwd=/home/bgamari/trees/matplotlib/matplotlib-1.0.1/examples/api
SAMPLE matplotlibrc=/home/bgamari/.matplotlib/matplotlibrc
reading sources... [ 12%] examples/api/date_index_formatter
SAMPLE cwd=/home/bgamari/trees/matplotlib/matplotlib-1.0.1/examples/api
SAMPLE matplotlibrc=/home/bgamari/.matplotlib/matplotlibrc
I suppose the only way around this is to patch the absolute path into
doc/matplotlibrc with sed, and then invoke doc/make.py with
MATPLOTLIBRC="$(CURDIR)/doc/matplotlibrc". I'm very confused how you
found that specifying a relative path in download.path worked. If the
cwd changes as seen above then the documentation was entirely correct in
claiming an absolute path is necessary. I really do wish there as a way
to prevent spinx from changing the cwd.
- Ben

Ben Gamari <bga...@gm...> writes:
> That being said, I've tracked down the reason for the configuration in
> doc/matplotlibrc not being used. It seems that sphinx runs the examples
> in their current directory (I suppose this is to be expected).
>
> It seems that there is a (as far as I can tell undocumented, please fix
> this) MATPLOTLIBRC environment variable which can be used to override
> the location of the matplotlibrc used. This could be useful in
> specifying the matplotlibrc used during the documentation build process
> but unfortunately this would require an absolute path which, again,
> complicates the packaging (i.e. I don't know the best way to accomplish
> this). Suggestions?
I don't see why the path must be absolute, though I haven't tried. Is
there some problem having a different global matplotlibrc
$HOME/.matplotlib/matplotlibrc or $MATPLOTLIBDATA/matplotlibrc,
possibly setting $HOME or $MATPLOTLIBDATA from your build script?
-- 
Jouni K. Seppänen
http://www.iki.fi/jks

Sorry to the Benjamin Drung, Scott Kitterman, and Sandro Tosi for not
including you in this thread earlier. I am trying to bring the
matplotlib 1.0.1 packaging into compliance with the debian packaging
requirements.
Unfortunately, matplotlib now includes out-of-tree sample data which is
downloaded during the documentation build process. To avoid this we need
to override a setting in matplotlibrc during the documentation build and
package these data in a separate source tarball. Most of this work has
been done (see my PPA[1]) but unfortunately the configuration changes
are non-trivial.
On Sun, 5 Dec 2010 17:16:44 -0600, John Hunter <jd...@gm...> wrote:
> On Sun, Dec 5, 2010 at 3:14 PM, Ben Gamari <bga...@gm...> wrote:
> >> I am not intimately familiar with the doc build system, but the way that
> >> the "html" function in make.py copies matplotlibrc from mpl-data to
> >> _static, I suspect that it is trying to ensure that that particular file
> >> is used.
> >>
> > Anyone else have further guidance?
> 
> No, that is there so that the link from the customizing page
> 
> http://matplotlib.sourceforge.net/users/customizing.html
> 
> to the "download matplotlibrc" link
> 
> http://matplotlib.sourceforge.net/_static/matplotlibrc
> 
> works and gives an update matplotlibrc. It is not used in the doc
> build to customize the settings of the docs. The files in _static are
> just copied up to the web server but can be linked to from withing
> rest.
> 
I'm glad to hear that this file isn't used. This should make packaging
much easier.
That being said, I've tracked down the reason for the configuration in
doc/matplotlibrc not being used. It seems that sphinx runs the examples
in their current directory (I suppose this is to be expected).
It seems that there is a (as far as I can tell undocumented, please fix
this) MATPLOTLIBRC environment variable which can be used to override
the location of the matplotlibrc used. This could be useful in
specifying the matplotlibrc used during the documentation build process
but unfortunately this would require an absolute path which, again,
complicates the packaging (i.e. I don't know the best way to accomplish
this). Suggestions?
Thanks,
- Ben
[1] https://launchpad.net/~bgamari/+archive/matplotlib-unofficial

This command in the make.osx file:
python2.7 -c 'import urllib; urllib.urlretrieve("
http://sourceforge.net/projects/libpng/files/libpng-stable/1.2.39/libpng-1.2.39.tar.gz/download",
"libpng-1.2.39.tar.gz")'
returns a bunch of html.
I'm not sure what the new URL is, but it should be a simple fix.
...................................................................................
Uri Laserson
Graduate Student, Biomedical Engineering
Harvard-MIT Division of Health Sciences and Technology
M +1 917 742 8019
las...@mi...

On Sun, Dec 5, 2010 at 3:14 PM, Ben Gamari <bga...@gm...> wrote:
>> I am not intimately familiar with the doc build system, but the way that
>> the "html" function in make.py copies matplotlibrc from mpl-data to
>> _static, I suspect that it is trying to ensure that that particular file
>> is used.
>>
> Anyone else have further guidance?
No, that is there so that the link from the customizing page
 http://matplotlib.sourceforge.net/users/customizing.html
to the "download matplotlibrc" link
 http://matplotlib.sourceforge.net/_static/matplotlibrc
works and gives an update matplotlibrc. It is not used in the doc
build to customize the settings of the docs. The files in _static are
just copied up to the web server but can be linked to from withing
rest.
JDH

On 2010年12月05日 23:03:02 +0200, Jouni K. Seppänen <jk...@ik...> wrote:
> Ben Gamari <bga...@gm...>
> writes:
> 
> > In practice, however, I have run into a few issues. First, the comment
> > regarding examples.directory indicates that this path must be
> > absolute. Is this really true? 
> 
> The comment is wrong: a relative path will work (I just tried this).
> 
Awesome. Perhaps this should be corrected in the tree so it doesn't look
like I'm blatantly violating the documentation?
> > Secondly, it seems that the doc/matplotlibrc file is completely
> > ignored.
> 
> I am not intimately familiar with the doc build system, but the way that
> the "html" function in make.py copies matplotlibrc from mpl-data to
> _static, I suspect that it is trying to ensure that that particular file
> is used.
>
Anyone else have further guidance?
- Ben

On 2010年12月05日 23:03:02 +0200, Jouni K. Seppänen <jk...@ik...> wrote:
> Ben Gamari <bga...@gm...>
> writes:
> 
> > In practice, however, I have run into a few issues. First, the comment
> > regarding examples.directory indicates that this path must be
> > absolute. Is this really true? 
> 
> The comment is wrong: a relative path will work (I just tried this).
> 
Awesome. Perhaps this should be corrected in the tree so it doesn't look
like I'm blatantly violating the documentation?
> > Secondly, it seems that the doc/matplotlibrc file is completely
> > ignored.
> 
> I am not intimately familiar with the doc build system, but the way that
> the "html" function in make.py copies matplotlibrc from mpl-data to
> _static, I suspect that it is trying to ensure that that particular file
> is used.
>
Anyone else have further guidance?
- Ben

Ben Gamari <bga...@gm...>
writes:
> In practice, however, I have run into a few issues. First, the comment
> regarding examples.directory indicates that this path must be
> absolute. Is this really true? 
The comment is wrong: a relative path will work (I just tried this).
> Secondly, it seems that the doc/matplotlibrc file is completely
> ignored.
I am not intimately familiar with the doc build system, but the way that
the "html" function in make.py copies matplotlibrc from mpl-data to
_static, I suspect that it is trying to ensure that that particular file
is used.
-- 
Jouni K. Seppänen
http://www.iki.fi/jks

On Mon, 8 Nov 2010 14:01:26 -0600, John Hunter <jd...@gm...> wrote:
> I will try and get to the release ASAP and set the sample_data up this way...
> 
Any news on this? A release would be very nice. I did the bulk of the
work[1] to get the debian packaging working with the 1.0 series but I
can't really do much else until there is an actual release to work
against.
- Ben
[1] https://launchpad.net/~bgamari/+archive/matplotlib-unofficial
 Although this isn't quite working yet. More about this in another
 thread.

This weekend I put a bit of time into bringing the Debian packaging[1]
for the matplotlib 1.0 series up to date.
In the tree it seems there is a matplotlibrc in the /doc
directory. Judging from the documentation, it looks like matplotlib will
first use a matplotlibrc in the current directory. This works very well
for the case of packaging for Debian, since I can simply override the
examples.* options here to avoid downloading the samples while building
the documentation.
In practice, however, I have run into a few issues. First, the comment
regarding examples.directory indicates that this path must be
absolute. Is this really true? For obvious reasons we don't know the
absolute path of the samples until we actually build the package. I
suppose an 'echo' in the packaging makefile could dynamically patch this
path in, but a relative path would be far simpler.
Secondly, it seems that the doc/matplotlibrc file is completely ignored.
When I tried building with "examples.directory=../sampledata", I still
saw multiple failed download attempts in the package build log. Indeed,
even putting an invalid line in doc/matplotlibrc doesn't result in an
error.
For the record, the documentation build process is invoked with,
 cd doc ; MATPLOTLIBDATA=../lib/matplotlib/mpl-data/ \
 PYTHONPATH=../build/lib.$(PY_PLATFORM)-$(DEFPY) ./make.py --small all
Any suggestions would be greatly appreciated.
- Ben
[1] https://launchpad.net/~bgamari/+archive/matplotlib-unofficial