matplotlib-devel Mailing List for matplotlib

Michael Droettboom wrote:
> Cleaning the docs first seems to have fixed it.
>
> Is there a way to download the build products (i.e. the PDF file 
> produced)?
That, and testing for doc build failures, is the point, although I 
managed to screw up the uploading until now. However, I believe I have 
fixed this and we should now have auto-built docs from svn trunk at:
http://matplotlib.sourceforge.net/trunk-docs
http://matplotlib.sourceforge.net/trunk-docs/Matplotlib.pdf
Perhaps we should link these from somewhere. And the URLs can certainly 
be changed without much difficulty.
Thanks for finding the clean step issue -- I thought the buildbot 
software automatically cleaned the working directory on every build, but 
obviously I thought wrong.
-Andrew

Cleaning the docs first seems to have fixed it.
Is there a way to download the build products (i.e. the PDF file produced)?
Mike
Michael Droettboom wrote:
> Hmm... I can't reproduce this locally. But it looks like it's using 
> doctree files cached from a previous run.
>
> I'll try calling "python make.py clean" before "python make.py all" in 
> the _buildbot_doc.sh (at least temporarily), to try to get a complete 
> build log which might offer more clues.
>
> Mike
>
> Andrew Straw wrote:
> 
>> Hi,
>>
>> With the recent regression fixed in Pygments, the doc auto-builder is
>> closer to completing successfully. However, there's a new bug. The build
>> ends with:
>>
>> LaTeX Warning: File `/home/mpl-chslave/slave-py25/build_docs/build/doc/build/pl
>> ot_directive/mpl_examples/pylab_examples/barb_demo.pdf' not found on input line
>> 28650.
>>
>>
>> !pdfTeX error: pdflatex (file /home/mpl-chslave/slave-py25/build_docs/build/doc
>> /build/plot_directive/mpl_examples/pylab_examples/barb_demo.pdf): cannot find i
>> mage file
>> ==> Fatal error occurred, no output PDF file produced!
>>
>>
>> Anyone have a clue what is going on here?
>>
>> Feel free to check in the attempted solution and the buildbot will
>> automatically try it. (See the "
>> <http://mpl-buildbot.code.astraw.com/builders/Ubuntu%208.04%2C%20Python%202.5%2C%20amd64>Ubuntu
>> 8.04, Python 2.5, amd64 (docs)" column at
>> http://mpl-buildbot.code.astraw.com/waterfall )
>>
>> Waiting for this will take up to 20 minutes before the build is
>> triggered (10 minute polling on the svn repo, and a 10 minute timeout
>> before any build is started in case another commit comes in). So you can
>> also force a build by clicking the column title ("Ubuntu 8.04, Python
>> 2.5, amd64 (docs)" and then clicking the "Force Build" button.
>>
>> It will be great to get the buildbot automatically building the svn docs
>> successfully.
>>
>> -Andrew
>>
>> ------------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Verizon Developer Community
>> Take advantage of Verizon's best-in-class app development support
>> A streamlined, 14 day to market process makes app distribution fast and easy
>> Join now and get one step closer to millions of Verizon customers
>> http://p.sf.net/sfu/verizon-dev2dev 
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>> 
>> 
>
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Hmm... I can't reproduce this locally. But it looks like it's using 
doctree files cached from a previous run.
I'll try calling "python make.py clean" before "python make.py all" in 
the _buildbot_doc.sh (at least temporarily), to try to get a complete 
build log which might offer more clues.
Mike
Andrew Straw wrote:
> Hi,
>
> With the recent regression fixed in Pygments, the doc auto-builder is
> closer to completing successfully. However, there's a new bug. The build
> ends with:
>
> LaTeX Warning: File `/home/mpl-chslave/slave-py25/build_docs/build/doc/build/pl
> ot_directive/mpl_examples/pylab_examples/barb_demo.pdf' not found on input line
> 28650.
>
>
> !pdfTeX error: pdflatex (file /home/mpl-chslave/slave-py25/build_docs/build/doc
> /build/plot_directive/mpl_examples/pylab_examples/barb_demo.pdf): cannot find i
> mage file
> ==> Fatal error occurred, no output PDF file produced!
>
>
> Anyone have a clue what is going on here?
>
> Feel free to check in the attempted solution and the buildbot will
> automatically try it. (See the "
> <http://mpl-buildbot.code.astraw.com/builders/Ubuntu%208.04%2C%20Python%202.5%2C%20amd64>Ubuntu
> 8.04, Python 2.5, amd64 (docs)" column at
> http://mpl-buildbot.code.astraw.com/waterfall )
>
> Waiting for this will take up to 20 minutes before the build is
> triggered (10 minute polling on the svn repo, and a 10 minute timeout
> before any build is started in case another commit comes in). So you can
> also force a build by clicking the column title ("Ubuntu 8.04, Python
> 2.5, amd64 (docs)" and then clicking the "Force Build" button.
>
> It will be great to get the buildbot automatically building the svn docs
> successfully.
>
> -Andrew
>
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev 
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Eric Firing wrote:
> John Hunter wrote:
> 
>> On Sat, Jan 2, 2010 at 12:34 AM, Eric Firing <ef...@ha...> wrote:
>> 
>>> Jae-Joon Lee wrote:
>>> 
>>>> Maybe we need something like "python make.py clean"?
>>>> 
It already does have "clean", but from the looks of it, it is currently 
broken (I just fixed this in SVN). It used to call out to svn-clean, 
but it was pointed out that only works if you're building from SVN, not 
from a tarball (which is a problem for the Linux distros). So there is 
explicit code in make.py to delete all of the generated files.
> Do you know why the plot directive (I assume that is the cause of the 
> problem) cannot put all generated output in build? Or is this a more 
> general Sphinx wart?
> 
The problem is not in the plot_directive (though I suppose it could be 
more helpful when the file isn't found) -- it does in fact put 
everything under build. The problem is in gen_rst.py, which generates 
pages for all of the examples, and puts them in doc/examples. At the 
time, it wasn't clear how to generate these pages under the build tree 
and then have Sphinx pick them up. That may have predated all of the 
"hooks" that Sphinx now has, however, and it may now be possible to 
solve that problem.
Mike
-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

On Mon, Jan 4, 2010 at 5:48 PM, Dag Sverre Seljebotn
<da...@st...> wrote:
>
> Rolling this into the Python package distribution scheme seems backwards
> though, since a lot of binary packages that have nothing to do with Python
> are used as well
Yep, exactly.
>
> To solve the exact problem you (and me) have I think the best solution is
> to integrate the tools mentioned above with what David is planning (SciPI
> etc.). Or if that isn't good enough, find generic "userland package
> manager" that has nothing to do with Python (I'm sure a dozen
> half-finished ones must have been written but didn't look), finish it, and
> connect it to SciPI.
You have 0install, autopackage and klik, to cite the ones I know
about. I wish people had looked at those before rolling toy solutions
to complex problems.
>
> What David does (I think) is seperate the concerns.
Exactly - you've describe this better than I did
David

Nathaniel Smith <nj...@po...> wrote:
> On Sun, Jan 3, 2010 at 4:23 AM, David Cournapeau <cou...@gm...>
> wrote:
>> Another way is to provide our own repository for a few major
>> distributions, with automatically built packages. This is how most
>> open source providers work. Miguel de Icaza explains this well:
>>
>> http://tirania.org/blog/archive/2007/Jan-26.html
>>
>> I hope we will be able to reuse much of the opensuse build service
>> infrastructure.
>
> Sure, I'm aware of the opensuse build service, have built third-party
> packages for my projects, etc. It's a good attempt, but also has a lot
> of problems, and when talking about scientific software it's totally
> useless to me :-). First, I don't have root on our compute cluster.
I use Sage for this very reason, and others use EPD or FEMHub or
Python(x,y) for the same reasons.
Rolling this into the Python package distribution scheme seems backwards
though, since a lot of binary packages that have nothing to do with Python
are used as well -- the Python stuff is simply thin wrappers around what
should ideally be located in /usr/lib or similar (but are nowadays
compiled into the Python extension .so because of distribution problems).
To solve the exact problem you (and me) have I think the best solution is
to integrate the tools mentioned above with what David is planning (SciPI
etc.). Or if that isn't good enough, find generic "userland package
manager" that has nothing to do with Python (I'm sure a dozen
half-finished ones must have been written but didn't look), finish it, and
connect it to SciPI.
What David does (I think) is seperate the concerns. This makes the task
feasible, and also has the advantage of convenience for the people that
*do* want to use Ubuntu, Red Hat or whatever to roll out scientific
software on hundreds of clients.
Dag Sverre

On Mon, Jan 4, 2010 at 8:42 AM, Nathaniel Smith <nj...@po...> wrote:
> On Sun, Jan 3, 2010 at 4:23 AM, David Cournapeau <cou...@gm...> wrote:
>> On Sun, Jan 3, 2010 at 8:05 PM, Nathaniel Smith <nj...@po...> wrote:
>>> What I do -- and documented for people in my lab to do -- is set up
>>> one virtualenv in my user account, and use it as my default python. (I
>>> 'activate' it from my login scripts.) The advantage of this is that
>>> easy_install (or pip) just works, without any hassle about permissions
>>> etc.
>>
>> It just works if you happen to be able to build everything from
>> sources. That alone means you ignore the majority of users I intend to
>> target.
>>
>> No other community (except maybe Ruby) push those isolated install
>> solutions as a general deployment solutions. If it were such a great
>> idea, other people would have picked up those solutions.
>
> AFAICT, R works more-or-less identically (once I convinced it to use a
> per-user library directory); install.packages() builds from source,
> and doesn't automatically pull in and build random C library
> dependencies.
As mentioned by Robert, this is different from the usual virtualenv
approach. Per-user app installation is certainly a useful (and
uncontroversial) feature.
And R does support automatically-built binary installers.
>
> Sure, I'm aware of the opensuse build service, have built third-party
> packages for my projects, etc. It's a good attempt, but also has a lot
> of problems, and when talking about scientific software it's totally
> useless to me :-). First, I don't have root on our compute cluster.
True, non-root install is a problem. Nothing *prevents* dpkg to run in
non root environment in principle if the packages itself does not
require it, but it is not really supported by the tools ATM.
> Second, even if I did I'd be very leery about installing third-party
> packages because there is no guarantee that the version numbering will
> be consistent between the third-party repo and the real distro repo --
> suppose that the distro packages 0.1, then the third party packages
> 0.2, then the distro packages 0.3, will upgrades be seamless? What if
> the third party screws up the version numbering at some point? Debian
> has "epochs" to deal with this, but third-parties can't use them and
> maintain compatibility.
Actually, at least with .deb-based distributions, this issue has a
solution. As packages has their own version in addition to the
upstream version, PPA-built packages have their own versions.
https://help.launchpad.net/Packaging/PPA/BuildingASourcePackage
Of course, this assumes a simple versioning scheme in the first place,
instead of the cluster-fck that versioning has became within python
packages (again, the scheme used in python is much more complicated
than everyone else, and it seems that nobody has ever stopped and
thought 5 minutes about the consequences, and whether this complexity
was a good idea in the first place).
> What if the person making the third party
> packages is not an expert on these random distros that they don't even
> use?
I think simple rules/conventions + build farms would solve most
issues. The problem is if you allow total flexibility as input, then
automatic and simple solutions become impossible. Certainly, PPA and
the build service provides for a much better experience than anything
pypi has ever given to me.
> Third, while we shouldn't advocate that people screw up backwards
> compatibility, version skew is a real issue. If I need one version of
> a package and my lab-mate needs another and we have submissions due
> tomorrow, then filing bugs is a great idea but not a solution.
Nothing prevents you from using virtualenv in that case (I may sound
dismissive of those tools, but I am really not. I use them myselves.
What I strongly react to is when those are pushed as the de-facto,
standard method).
> Fourth,
> even if we had expert maintainers taking care of all these third-party
> packages and all my concerns were answered, I couldn't convince our
> sysadmin of that; he's the one who'd have to clean up if something
> went wrong we don't have a big budget for overtime.
I am not advocating using only packaged, binary installers. I am
advocating using them as much as possible where it makes sense - on
windows and mac os x in particular.
Toydist also aims at making it easier to build, customize installs.
Although not yet implemented, --user-like scheme would be quite simple
to implement, because toydist installer internally uses autoconf-like
directories description (of which --user is a special case).
If you need sandboxed installs, customized installs, toydist will not
prevent it. It is certainly my intention to make it possible to use
virtualenv and co (you already can by building eggs, actually). I hope
that by having our own "SciPi", we can actually have a more reliable
approach. For example, the static dependency description + mandated
metadata would make this much easier and more robust, as there would
not be a need to run a setup.py to get the dependencies.
If you look at hackageDB
(http://hackage.haskell.org/packages/hackage.html), they have a very
simple index structure, which makes it easy to download it entirely,
and reuse this locally to avoid any internet access.
> Let's be honest -- scientists, on the whole, suck at IT
> infrastructure, and small individual packages are not going to be very
> expertly put together. IMHO any real solution should take this into
> account, keep them sandboxed from the rest of the system, and focus on
> providing the most friendly and seamless sandbox possible.
I agree packages will not always be well put together - but I don't
see why this would be worse than the current situation. I also
strongly disagree about the sandboxing as the solution of choice. For
most users, having only one install of most packages is the typical
use-case. Once you start sandboxing, you create artificial barriers
between the sandboxes, and this becomes too complicated for most users
IMHO.
>
> Maybe I was unclear -- proper build directory handling is nice,
> Cython/Pyrex's distutils integration get it wrong (not their fault,
> distutils is just impossible to do anything sensible with, as you've
> said), and I've never found build directories hard to implement
It is simple if you have a good infrastructure in place (node
abstraction, etc...), but that infrastructure is hard to get right.
> But what I'm really talking about is
> having a "pre-build" step that integrates properly with the source and
> binary packaging stages, and that's not something waf or scons have
> any particular support for, AFAIK.
Could you explain with a concrete example what a pre-build stage would
look like ? I don't think I understand what you want,
cheers,
David

John Hunter wrote:
> On Sun, Jan 3, 2010 at 3:54 PM, Eric Firing <ef...@ha...> wrote:
>> The only problem is that lines.color is the default for LineCollection and
>> Line2D, both of which are fairly separate from Axes, so having them default
>> to rcParams['axes.color_cycle'][0] seems a little odd.
> 
> Yes, I was worrying about the same thing while cooking dinner :-) A
> line can be added to a Figure or an Axes, so perhaps the default color
> should not come from there. What about a module level attribute
> matplotlib.colors.cycle by an rc param "colors.cycle" and then the
> default Line2D and LineCollection color can default to
> matplotlib.colors.cycle[0].
We could, but I don't see any advantage over leaving it the way I made 
it a few minutes ago, which leaves lines.color independent of the 
axes-level cycle. It seems to me that color-cycling is quite different 
from setting a default color for something like Line2D, and there is no 
reason they should be connected via the first element of a list. Maybe 
this would need to be revisited as part of the API layer redesign that 
you sketched out once, but that we still haven't gotten around to advancing.
The idea of a rename to colors.cycle has some appeal, except that as it 
is used, it really is Axes-specific, and appears uniquely at the Axes level.
Eric

On Sun, Jan 3, 2010 at 3:54 PM, Eric Firing <ef...@ha...> wrote:
> The only problem is that lines.color is the default for LineCollection and
> Line2D, both of which are fairly separate from Axes, so having them default
> to rcParams['axes.color_cycle'][0] seems a little odd.
Yes, I was worrying about the same thing while cooking dinner :-) A
line can be added to a Figure or an Axes, so perhaps the default color
should not come from there. What about a module level attribute
matplotlib.colors.cycle by an rc param "colors.cycle" and then the
default Line2D and LineCollection color can default to
matplotlib.colors.cycle[0].

Hi,
With the recent regression fixed in Pygments, the doc auto-builder is
closer to completing successfully. However, there's a new bug. The build
ends with:
LaTeX Warning: File `/home/mpl-chslave/slave-py25/build_docs/build/doc/build/pl
ot_directive/mpl_examples/pylab_examples/barb_demo.pdf' not found on input line
 28650.
!pdfTeX error: pdflatex (file /home/mpl-chslave/slave-py25/build_docs/build/doc
/build/plot_directive/mpl_examples/pylab_examples/barb_demo.pdf): cannot find i
mage file
 ==> Fatal error occurred, no output PDF file produced!
Anyone have a clue what is going on here?
Feel free to check in the attempted solution and the buildbot will
automatically try it. (See the "
<http://mpl-buildbot.code.astraw.com/builders/Ubuntu%208.04%2C%20Python%202.5%2C%20amd64>Ubuntu
8.04, Python 2.5, amd64 (docs)" column at
http://mpl-buildbot.code.astraw.com/waterfall )
Waiting for this will take up to 20 minutes before the build is
triggered (10 minute polling on the svn repo, and a 10 minute timeout
before any build is started in case another commit comes in). So you can
also force a build by clicking the column title ("Ubuntu 8.04, Python
2.5, amd64 (docs)" and then clicking the "Force Build" button.
It will be great to get the buildbot automatically building the svn docs
successfully.
-Andrew