matplotlib-devel Mailing List for matplotlib

On Thu, Jan 6, 2011 at 1:24 PM, Benjamin Root <ben...@ou...> wrote:
> Ah, all the more reason to apply abspath() or realpath(). To decide which
> to use, let's consider the case of someone (like a developer) having
> multiple builds of matplotlib in separate directories, and uses a symlink to
> point to whichever he wants to use at the moment.
>
> The question is, in this use-case, would we want the symbolic link pathname,
> or the absolute pathname? I don't mess around with docs enough to know
> which I would want.
>
> I have attached a modified patch (which uses realpath(), but could easily be
> changed to abspath()). I also included some comments to more fully document
> what is going on and the rational for the logic being taken.
OK, I incorporated your changes and committed. Thanks.
> P.S. - Just to make sure, I noticed that rcParamsOrig is only in the
> maintenance branch. It was intended to leave the development branch
> "broken" for now until we get this working properly?
That's correct, but I just did a big merge of all the branch changes
so the trunk is fixed as well now.
JDH

On Thu, Jan 6, 2011 at 12:32 PM, John Hunter <jd...@gm...> wrote:
> On Thu, Jan 6, 2011 at 12:22 PM, John Hunter <jd...@gm...> wrote:
>
> > matplotlib_fname() always returns absolute path. I have not used
> > realpath, but if you think there is a use for it here, feel free to
> > post an amended patch.
>
> There is an exception to this -- if MATPLOTLIBRC or MPLCONFIGDIR are
> relative paths, then matplotlib_fname will return a relative path too.
>
Ah, all the more reason to apply abspath() or realpath(). To decide which
to use, let's consider the case of someone (like a developer) having
multiple builds of matplotlib in separate directories, and uses a symlink to
point to whichever he wants to use at the moment.
The question is, in this use-case, would we want the symbolic link pathname,
or the absolute pathname? I don't mess around with docs enough to know
which I would want.
I have attached a modified patch (which uses realpath(), but could easily be
changed to abspath()). I also included some comments to more fully document
what is going on and the rational for the logic being taken.
Ben Root
P.S. - Just to make sure, I noticed that rcParamsOrig is only in the
maintenance branch. It was intended to leave the development branch
"broken" for now until we get this working properly?

On Thu, Jan 6, 2011 at 12:22 PM, John Hunter <jd...@gm...> wrote:
> matplotlib_fname() always returns absolute path. I have not used
> realpath, but if you think there is a use for it here, feel free to
> post an amended patch.
There is an exception to this -- if MATPLOTLIBRC or MPLCONFIGDIR are
relative paths, then matplotlib_fname will return a relative path too.

On Thu, Jan 6, 2011 at 12:15 PM, Benjamin Root <ben...@ou...> wrote:
> A comment on the patch. Unless matplotlib_fname() is guaranteed to return
> an absolute filename, then we need to use realpath() on _basedir so that the
> final joined filename will also be absolute. The reason for using
> realpath() instead of abspath() is that in case there are any symbolic links
> in the name from matplotlib_fname(), these will be qualified so that the
> filepath matplotlib sees is the same filepath that spinx sees.
>
> Also, a little oddity that I discovered in reading the docs for
> os.path.join(), turns out that if any element is already an absolute path,
> then it ignores all elements prior to that. In this patch, everything is ok
> because we already check for isabs(). Just thought I ought to point that
> out because I never knew that and that would certainly be a nasty bug to try
> and track down..
matplotlib_fname() always returns absolute path. I have not used
realpath, but if you think there is a use for it here, feel free to
post an amended patch.
JDH

On Thu, Jan 6, 2011 at 11:50 AM, John Hunter <jd...@gm...> wrote:
> On Thu, Jan 6, 2011 at 11:36 AM, Benjamin Root <ben...@ou...> wrote:
> > On Thu, Jan 6, 2011 at 11:15 AM, John Hunter <jd...@gm...> wrote:
> >>
> >> On Thu, Jan 6, 2011 at 10:58 AM, Benjamin Root <ben...@ou...> wrote:
> >>
> >> > I thought that problem was already addressed in a previous patch?
> John,
> >> > could this be another "gremlin"? Or did I mis-understand the whole
> >> > issue
> >> > absolute path problem?
> >>
> >> My understanding is that because sphinx switches the current working
> >> directory during execution, this path must be absolute. We could get
> >> around this by processing a relative path internally in mpl at rc load
> >> time, and converting it into an absolute path assuming the path is
> >> relative to the directory containing the rc file.
> >>
> >> JDH
> >
> >
> > I actually think that would be a better solution. Python's os.path
> module
> > is very powerful with functions like isabs(), abspath(), expanduser(),
> > expandvars() and realpath(). Of course, one could easily go overboard
> with
> > this, but I am sure we could probably allow for something real simple
> like
> > expandvars() so that packagers could utilize environment variables for
> the
> > build process?
>
> Here is a candidate patch...this will be processed once on module load
> time. I considered doing it in the validate method of the rc params
> in rcsetup, but I am not sure this is better because if users are
> overriding the variable at runtime, we should probably just let them
> do what they want.
>
> I am not sure if this qualifies as a "one time hack" that makes mpl so
> difficult to maintain -- Sandro and Ben can weigh in -- but at least
> we are doing it to make their lives easier :-)
>
> Index: lib/matplotlib/__init__.py
> ===================================================================
> --- lib/matplotlib/__init__.py (revision 8898)
> +++ lib/matplotlib/__init__.py (working copy)
> @@ -762,6 +762,13 @@
>
> # this is the instance used by the matplotlib classes
> rcParams = rc_params()
> +
> +if rcParams['examples.directory']:
> + if not os.path.isabs(rcParams['examples.directory']):
> + _basedir, _fname = os.path.split(matplotlib_fname())
> + _fullpath = os.path.join(_basedir, rcParams['examples.directory'])
> + rcParams['examples.directory'] = _fullpath
> +
> rcParamsOrig = rcParams.copy()
>
> rcParamsDefault = RcParams([ (key, default) for key, (default, converter)
> in \
> @@ -770,6 +777,8 @@
> rcParams['ps.usedistiller'] =
> checkdep_ps_distiller(rcParams['ps.usedistiller'])
> rcParams['text.usetex'] = checkdep_usetex(rcParams['text.usetex'])
>
> +
> +
> def rc(group, **kwargs):
> """
> Set the current rc params. Group is the grouping for the rc, eg.
>
I like the idea. Obviously we would need to make it very clear exactly what
relative filepaths are going to be relative to.
A comment on the patch. Unless matplotlib_fname() is guaranteed to return
an absolute filename, then we need to use realpath() on _basedir so that the
final joined filename will also be absolute. The reason for using
realpath() instead of abspath() is that in case there are any symbolic links
in the name from matplotlib_fname(), these will be qualified so that the
filepath matplotlib sees is the same filepath that spinx sees.
Also, a little oddity that I discovered in reading the docs for
os.path.join(), turns out that if any element is already an absolute path,
then it ignores all elements prior to that. In this patch, everything is ok
because we already check for isabs(). Just thought I ought to point that
out because I never knew that and that would certainly be a nasty bug to try
and track down...

On Thu, Jan 6, 2011 at 11:36 AM, Benjamin Root <ben...@ou...> wrote:
> On Thu, Jan 6, 2011 at 11:15 AM, John Hunter <jd...@gm...> wrote:
>>
>> On Thu, Jan 6, 2011 at 10:58 AM, Benjamin Root <ben...@ou...> wrote:
>>
>> > I thought that problem was already addressed in a previous patch? John,
>> > could this be another "gremlin"? Or did I mis-understand the whole
>> > issue
>> > absolute path problem?
>>
>> My understanding is that because sphinx switches the current working
>> directory during execution, this path must be absolute. We could get
>> around this by processing a relative path internally in mpl at rc load
>> time, and converting it into an absolute path assuming the path is
>> relative to the directory containing the rc file.
>>
>> JDH
>
>
> I actually think that would be a better solution. Python's os.path module
> is very powerful with functions like isabs(), abspath(), expanduser(),
> expandvars() and realpath(). Of course, one could easily go overboard with
> this, but I am sure we could probably allow for something real simple like
> expandvars() so that packagers could utilize environment variables for the
> build process?
Here is a candidate patch...this will be processed once on module load
time. I considered doing it in the validate method of the rc params
in rcsetup, but I am not sure this is better because if users are
overriding the variable at runtime, we should probably just let them
do what they want.
I am not sure if this qualifies as a "one time hack" that makes mpl so
difficult to maintain -- Sandro and Ben can weigh in -- but at least
we are doing it to make their lives easier :-)
Index: lib/matplotlib/__init__.py
===================================================================
--- lib/matplotlib/__init__.py (revision 8898)
+++ lib/matplotlib/__init__.py (working copy)
@@ -762,6 +762,13 @@
 # this is the instance used by the matplotlib classes
 rcParams = rc_params()
+
+if rcParams['examples.directory']:
+ if not os.path.isabs(rcParams['examples.directory']):
+ _basedir, _fname = os.path.split(matplotlib_fname())
+ _fullpath = os.path.join(_basedir, rcParams['examples.directory'])
+ rcParams['examples.directory'] = _fullpath
+
 rcParamsOrig = rcParams.copy()
 rcParamsDefault = RcParams([ (key, default) for key, (default, converter) in \
@@ -770,6 +777,8 @@
 rcParams['ps.usedistiller'] =
checkdep_ps_distiller(rcParams['ps.usedistiller'])
 rcParams['text.usetex'] = checkdep_usetex(rcParams['text.usetex'])
+
+
 def rc(group, **kwargs):
 """
 Set the current rc params. Group is the grouping for the rc, eg.

On Thu, Jan 6, 2011 at 11:15 AM, John Hunter <jd...@gm...> wrote:
> On Thu, Jan 6, 2011 at 10:58 AM, Benjamin Root <ben...@ou...> wrote:
>
> > I thought that problem was already addressed in a previous patch? John,
> > could this be another "gremlin"? Or did I mis-understand the whole issue
> > absolute path problem?
>
> My understanding is that because sphinx switches the current working
> directory during execution, this path must be absolute. We could get
> around this by processing a relative path internally in mpl at rc load
> time, and converting it into an absolute path assuming the path is
> relative to the directory containing the rc file.
>
> JDH
>
I actually think that would be a better solution. Python's os.path module
is very powerful with functions like isabs(), abspath(), expanduser(),
expandvars() and realpath(). Of course, one could easily go overboard with
this, but I am sure we could probably allow for something real simple like
expandvars() so that packagers could utilize environment variables for the
build process?
Ben Root

On Thu, Jan 6, 2011 at 10:58 AM, Benjamin Root <ben...@ou...> wrote:
> I thought that problem was already addressed in a previous patch? John,
> could this be another "gremlin"? Or did I mis-understand the whole issue
> absolute path problem?
My understanding is that because sphinx switches the current working
directory during execution, this path must be absolute. We could get
around this by processing a relative path internally in mpl at rc load
time, and converting it into an absolute path assuming the path is
relative to the directory containing the rc file.
JDH

On Wed, Jan 5, 2011 at 10:42 PM, Sandro Tosi <mo...@de...> wrote:
> On Thu, Jan 6, 2011 at 02:31, John Hunter <jd...@gm...> wrote:
> > mpl1> grep rc_file lib/matplotlib/sphinxext/plot_directive.py
> >
> > Surely I have it right now!
>
> Indeed, and it works fine!!! the only thing to notice is that
> examples.directory must be an absolute path, but that set, the doc
> build process gets files from the location specified.
>
>
I thought that problem was already addressed in a previous patch? John,
could this be another "gremlin"? Or did I mis-understand the whole issue
absolute path problem?
Ben Root

2011年1月4日 Russell E. Owen <ro...@uw...>:
> I'm not sure what to do about 64-bit Python 2.7. It does not even
> support Mac OS X 10.5 due to tcl/tk issues that I think were resolved
> too late for python 2.7.1. In my opinion a matplotlib built against
> ActiveState's Python 2.7 (which is 64-bit and supports 10.5 and 10.6)
> might be of more use. Opinions?
Here's the state of the art for numpy:
http://article.gmane.org/gmane.comp.python.numeric.general/41799.
So from my point of view mpl Mac OS X py2.7 32+64bit should be built
on Mac OS X 10.6 and should not support < 10.6.
Friedrich

On Thu, Jan 6, 2011 at 02:31, John Hunter <jd...@gm...> wrote:
> mpl1> grep rc_file lib/matplotlib/sphinxext/plot_directive.py
>
> Surely I have it right now!
Indeed, and it works fine!!! the only thing to notice is that
examples.directory must be an absolute path, but that set, the doc
build process gets files from the location specified.
Thanks a lot!!
Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

On Wed, Jan 5, 2011 at 5:53 PM, Sandro Tosi <mo...@de...> wrote:
> On Wed, Jan 5, 2011 at 23:22, John Hunter <jd...@gm...> wrote:
>> OK, I found the problem. Somehow my edit to sphinxext.plot_directive
>> to actually call the new function was not saved and committed. I just
>> fixed this, so the branch *should* work. The branch
>> matlpotlib/sphinxext/plot_directive.py should call rc_file_defaults
>> rather than rcdefaults. After an svn up, confirm that this is so
>> before testing.
>
> $ grep -c rc_file_defaults lib/matplotlib/sphinxext/plot_directive.py
> 0
>
> so it's not there :(
>
> and the only file containing rc_file_defaults is lib/matplotlib/__init__.py
>
> of course, examples are stil stuck if no network to SF is available
>
> Oh i think I got it, lib/matplotlib/sphinxext/plot_directive.py was
> updated on trunk instead of the v1.0 branch.
I swear there is a gremlin in my computer. I checked and rechecked
this on my box because of all the confusion we were having, but
apparently managed to commit on the trunk (which doesn't even have the
function defined because I haven't merged) and not on the branch. We
need to get the buildbot going again, because the doc buildbot would
have caught this. I have fixed both the trunk and the branch: the
branch calls the new function and has the desired "no internet"
behavior. The trunk calls the old function which restores rc internal
defaults rather than file defaults and so is broken in the way it has
always been broken, but will be fixed when we merge.
In the command below, mpl1 is my branch (1.0.1 release) directory:
 mpl1> grep rc_file lib/matplotlib/sphinxext/plot_directive.py
Surely I have it right now!
JDH

On Wed, 5 Jan 2011 16:00:40 -0600, John Hunter <jd...@gm...> wrote:
> Strange -- I'll try and test tonight with a machine disconnected from
> the internet. Out of curiosity, what command/tool are you using to
> monitor the internet requests? Ben, are you seeing the same problems?
>
It took me a long time to finally figure out how to get matplotlib to
respect the wishes of the packaging. Ultimately I believe I got it
working but unfortunately, it's been a while, so I'm not entirely sure
what was necessary. If it helps, I believe the package in my PPA builds
correctly. Sorry I don't have more time to investigate; things are quite
hectic at the moment.
Cheers,
- Ben