matplotlib-devel Mailing List for matplotlib

On Mon, Dec 28, 2009 at 02:29:24PM -0500, Neal Becker wrote:
> Perhaps this could be useful:
> http://checkinstall.izto.org/
Yes, checkinstall is really cool. However, I tend to prefer things with
no magic that I don't have to sandbox to know what they are doing. This
is why I am also happy to hear about toydist.
Gaël

David Cournapeau wrote:
> The idea would be that for a few major distributions at least, you
> would have .rpm available on the repository. If you install from
> sources, there would be a few mechanisms to avoid your exact issue
> (like maybe defaulting to --user kind of installs). Of course, it can
> only be dealt up to a point.
> 
> David
> 
Perhaps this could be useful:
http://checkinstall.izto.org/

> On Tue, Dec 29, 2009 at 3:49 AM, Dag Sverre Seljebotn
> <da...@st...> wrote:
>
>>
>> Do you here mean automatic generation of Ubuntu debs, Debian debs,
>> Windows
>> MSI installer, Windows EXE installer, and so on? (If so then great!)
>
> Yes (although this is not yet implemented). In particular on windows,
> I want to implement a scheme so that you can convert from eggs to .exe
> and vice et versa, so people can still install as exe (or msi), even
> though the method would default to eggs.
>
>> If this is the goal, I wonder if one looks outside of Python-land one
>> might find something that already does this -- there's a lot of
>> different
>> package format, "Linux meta-distributions", "install everywhere
>> packages"
>> and so on.
>
> Yes, there are things like 0install or autopackage. I think those are
> deemed to fail, as long as it is not supported thoroughly by the
> distribution. Instead, my goal here is much simpler: producing
> rpm/deb. It does not solve every issue (install by non root, multiple
> // versions), but one has to be realistic :)
>
> I think automatically built rpm/deb, easy integration with native
> method can solve a lot of issues already.
>
>>
>> - Currently I'm making a Sage SPKG for CHOLMOD. This essentially gets
>> the
>> job done by not bothering about the problem, not even using the
>> OS-installed Python.
>>
>> Something that would spit out both Sage SPKGs, Ubuntu debs, Windows
>> installers, both with Python code and C/Fortran code or a mix (and put
>> both in the place preferred by the system in question), seems ideal. Of
>> course one would still need to make sure that the code builds properly
>> everywhere, but just solving the distribution part of this would be a
>> huge
>> step ahead.
>
> On windows, this issue may be solved using eggs: enstaller has a
> feature where dll put in a special location of an egg are installed in
> python such as they are found by the OS loader. One could have
> mechanisms based on $ORIGIN + rpath on linux to solve this issue for
> local installs on Linux, etc...
>
> But again, one has to be realistic on the goals. With toydist, I want
> to remove all the pile of magic, hacks built on top of distutils so
> that people can again hack their own solutions, as it should have been
> from the start (that's a big plus of python in general). It won't
> magically solve every issue out there, but it would hopefully help
> people to make their own.
>
> Bundling solutions like SAGE, EPD, etc... are still the most robust
> ways to deal with those issues in general, and I do not intended to
> replace those.
>
>> What I'm saying is that this is a software distribution problem in
>> general, and I'm afraid that Python-specific solutions are too narrow.
>
> Distribution is a hard problem. Instead of pushing a very narrow (and
> mostly ill-funded) view of how people should do things like
> distutils/setuptools/pip/buildout do, I want people to be able to be
> able to build their own solutions. No more "use this magic stick v
> 4.0.3.3.14svn1234, trust me it work you don't have to understand"
> which is too prevalant with those tools, which has always felt deeply
> unpythonic to me.
Thanks, this cleared things up, and I like the direction this is heading.
Thanks a lot for doing this!
Dag Sverre

On Tue, Dec 29, 2009 at 3:49 AM, Dag Sverre Seljebotn
<da...@st...> wrote:
>
> Do you here mean automatic generation of Ubuntu debs, Debian debs, Windows
> MSI installer, Windows EXE installer, and so on? (If so then great!)
Yes (although this is not yet implemented). In particular on windows,
I want to implement a scheme so that you can convert from eggs to .exe
and vice et versa, so people can still install as exe (or msi), even
though the method would default to eggs.
> If this is the goal, I wonder if one looks outside of Python-land one
> might find something that already does this -- there's a lot of different
> package format, "Linux meta-distributions", "install everywhere packages"
> and so on.
Yes, there are things like 0install or autopackage. I think those are
deemed to fail, as long as it is not supported thoroughly by the
distribution. Instead, my goal here is much simpler: producing
rpm/deb. It does not solve every issue (install by non root, multiple
// versions), but one has to be realistic :)
I think automatically built rpm/deb, easy integration with native
method can solve a lot of issues already.
>
> - Currently I'm making a Sage SPKG for CHOLMOD. This essentially gets the
> job done by not bothering about the problem, not even using the
> OS-installed Python.
>
> Something that would spit out both Sage SPKGs, Ubuntu debs, Windows
> installers, both with Python code and C/Fortran code or a mix (and put
> both in the place preferred by the system in question), seems ideal. Of
> course one would still need to make sure that the code builds properly
> everywhere, but just solving the distribution part of this would be a huge
> step ahead.
On windows, this issue may be solved using eggs: enstaller has a
feature where dll put in a special location of an egg are installed in
python such as they are found by the OS loader. One could have
mechanisms based on $ORIGIN + rpath on linux to solve this issue for
local installs on Linux, etc...
But again, one has to be realistic on the goals. With toydist, I want
to remove all the pile of magic, hacks built on top of distutils so
that people can again hack their own solutions, as it should have been
from the start (that's a big plus of python in general). It won't
magically solve every issue out there, but it would hopefully help
people to make their own.
Bundling solutions like SAGE, EPD, etc... are still the most robust
ways to deal with those issues in general, and I do not intended to
replace those.
> What I'm saying is that this is a software distribution problem in
> general, and I'm afraid that Python-specific solutions are too narrow.
Distribution is a hard problem. Instead of pushing a very narrow (and
mostly ill-funded) view of how people should do things like
distutils/setuptools/pip/buildout do, I want people to be able to be
able to build their own solutions. No more "use this magic stick v
4.0.3.3.14svn1234, trust me it work you don't have to understand"
which is too prevalant with those tools, which has always felt deeply
unpythonic to me.
David

David wrote:
> Repository
> ========
>
> The goal here is to have something like CRAN
> (http://cran.r-project.org/web/views/), ideally with a build farm so
> that whenever anyone submits a package to our repository, it would
> automatically be checked, and built for windows/mac os x and maybe a
> few major linux distributions. One could investigate the build service
> from open suse to that end (http://en.opensuse.org/Build_Service),
> which is based on xen VM to build installers in a reproducible way.
Do you here mean automatic generation of Ubuntu debs, Debian debs, Windows
MSI installer, Windows EXE installer, and so on? (If so then great!)
If this is the goal, I wonder if one looks outside of Python-land one
might find something that already does this -- there's a lot of different
package format, "Linux meta-distributions", "install everywhere packages"
and so on.
Of course, toydist could have such any such tool as a backend/in a pipeline.
> What's next ?
> ==========
>
> At this point, I would like to ask for help and comments, in particular:
> - Does all this make sense, or hopelessly intractable ?
> - Besides the points I have mentioned, what else do you think is needed ?
Hmm. What I miss is the discussion of other native libraries which the
Python libraries need to bundle. Is it assumed that one want to continue
linking C and Fortran code directly into Python .so modules, like the
scipy library currently does?
Let me take CHOLMOD (sparse Cholesky) as an example.
 - The Python package cvxopt use it, simply by linking about 20 C files
directly into the Python-loadable module (.so) which goes into the Python
site-packages (or wherever). This makes sure it just works. But, it
doesn't feel like the right way at all.
 - scikits.sparse.cholmod OTOH simple specifies libraries=["cholmod"], and
leave it up to the end-user to make sure it is installed. Linux users
with root access can simply apt-get, but it is a pain for everybody else
(Windows, Mac, non-root Linux).
 - Currently I'm making a Sage SPKG for CHOLMOD. This essentially gets the
job done by not bothering about the problem, not even using the
OS-installed Python.
Something that would spit out both Sage SPKGs, Ubuntu debs, Windows
installers, both with Python code and C/Fortran code or a mix (and put
both in the place preferred by the system in question), seems ideal. Of
course one would still need to make sure that the code builds properly
everywhere, but just solving the distribution part of this would be a huge
step ahead.
What I'm saying is that this is a software distribution problem in
general, and I'm afraid that Python-specific solutions are too narrow.
Dag Sverre

On Tue, Dec 29, 2009 at 3:03 AM, Neal Becker <ndb...@gm...> wrote:
> David Cournapeau wrote:
>
>> On Mon, Dec 28, 2009 at 11:47 PM, Stefan Schwarzburg
>> <ste...@go...> wrote:
>>> Hi,
>>> I would like to add a comment from the user perspective:
>>>
>>> - the main reason why I'm not satisfied with pypi/distutils/etc. and why
>>> I will not be satisfied with toydist (with the features you listed), is
>>> that they break my installation (debian/ubuntu).
>>
>> Toydist (or distutils) does not break anything as is. It would be like
>> saying make breaks debian - it does not make much sense. As stated,
>> one of the goal of giving up distutils is to make packaging by os
>> vendors easier. In particular, by allowing to follow the FHS, and
>> making things more consistent. It should be possible to automatically
>> convert most packages to .deb (or .rpm) relatively easily. When you
>> look at the numpy .deb package, most of the issues are distutils
>> issues, and almost everything else can be done automatically.
>>
>> Note that even ignoring the windows problem, there are systems to do
>> the kind of things I am talking about for linux-only systems (the
>> opensuse build service), because distributions are not always really
>> good at tracking fast changing softwares. IOW, traditional linux
>> packaging has some issues as well. And anyway, nothing prevents debian
>> or other OS vendors to package things as they want (as they do for R
>> packages).
>>
>> David
>
> I think the breakage that is referred to I can describe on my favorite
> system, fedora.
>
> I can install the fedora numpy rpm using yum. I could also use
> easy_install. Unfortunately:
> 1) Each one knows nothing about the other
> 2) They may install things into conflicting paths. In particular, on fedora
> arch-dependent things go in /usr/lib64/python<version>/site-packages while
> arch-independent goes into /usr/lib/python<version>... If you mix yum with
> easy_install (or setuptools), you many times wind up with 2 versions and a
> lot of confusion.
>
> This is NOT unusual. Let's say I have numpy-1.3.0 installed from rpms. I
> see the announcement of numpy-1.4.0, and decide I want it, before the rpm is
> available, so I use easy_install. Now numpy-1.4.0 shows up as a standard
> rpm, and a subsequent update (which could be automatic!) could produce a
> broken system.
Several points:
 - First, this is caused by distutils misfeature of defaulting to
/usr. This is a mistake. It should default to /usr/local, as does
every other install method from sources.
 - A lot of instructions start by sudo easy_install... This is a very
bad advice, especially given the previous issue.
> I don't really know what could be done about it. Perhaps a design that
> attempts to use native backends for installation where available?
The idea would be that for a few major distributions at least, you
would have .rpm available on the repository. If you install from
sources, there would be a few mechanisms to avoid your exact issue
(like maybe defaulting to --user kind of installs). Of course, it can
only be dealt up to a point.
David

Stefan Schwarzburg wrote:
> Hi,
> I would like to add a comment from the user perspective:
>
> - the main reason why I'm not satisfied with pypi/distutils/etc. and
> why I will not be satisfied with toydist (with the features you
> listed), is that they break my installation (debian/ubuntu). The main
> advantage of these kinds of linux is their packaging system. So even
> if there is a way to query if a python package is installed, I will
> need three or more commands to check if it really is there (toydist,
> distutils, aptitude and other packages installed by hand).
I am interested in adapting stdeb ( http://github.com/astraw/stdeb ) to
handle toydist distributions. Nevertheless, the goals of toydist seem
mostly unrelated to your issue with the exception that toydist will
hopefully make generating .deb packages easier and more robust.
See http://github.com/astraw/stdeb/issues#issue/16 for a wishlist item
that I think would solve your issue. In that ticket, I give the steps
required to solve the issue -- I don't think it looks too hard, and
patches will be reviewed with a goal of producing something that gets
merged.
-Andrew

On Mon, Dec 28, 2009 at 11:47 PM, Stefan Schwarzburg
<ste...@go...> wrote:
> Hi,
> I would like to add a comment from the user perspective:
>
> - the main reason why I'm not satisfied with pypi/distutils/etc. and why I
> will not be satisfied with toydist (with the features you listed), is that
> they break my installation (debian/ubuntu).
Toydist (or distutils) does not break anything as is. It would be like
saying make breaks debian - it does not make much sense. As stated,
one of the goal of giving up distutils is to make packaging by os
vendors easier. In particular, by allowing to follow the FHS, and
making things more consistent. It should be possible to automatically
convert most packages to .deb (or .rpm) relatively easily. When you
look at the numpy .deb package, most of the issues are distutils
issues, and almost everything else can be done automatically.
Note that even ignoring the windows problem, there are systems to do
the kind of things I am talking about for linux-only systems (the
opensuse build service), because distributions are not always really
good at tracking fast changing softwares. IOW, traditional linux
packaging has some issues as well. And anyway, nothing prevents debian
or other OS vendors to package things as they want (as they do for R
packages).
David

Hi,
I would like to add a comment from the user perspective:
- the main reason why I'm not satisfied with pypi/distutils/etc. and why I
will not be satisfied with toydist (with the features you listed), is that
they break my installation (debian/ubuntu). The main advantage of these
kinds of linux is their packaging system. So even if there is a way to query
if a python package is installed, I will need three or more commands to
check if it really is there (toydist, distutils, aptitude and other packages
installed by hand).
I know you don't want to hear this, but my suggestion (as a user) would be:
use the debian package system. If you introduce a new system anyway, why not
use the best there is and remove at least one incompatibility with at least
one system?
It shouldn't make a big difference on windows, because they would have to
install/learn/use your new system anyway. The same is true for OS X, BSDs
and RPM based linuxes. So they won't have any disadvantage when debs are
used. But deb based Linuxes would have a big advantage: their system
wouldn't be broken.
And with debs, you would have:
- http-based package repository ala debian/ubuntu, which would be easy to
mirror and backup (through rsync-like tools)
- decoupling the building, packaging and distribution of code and data
- reliable install/uninstall/query of what is installed locally
- making the life of OS vendors (Linux, *BSD, etc...) easier
You wouldn't get automatic compiling for windows and I'm not sure if the
tools are as easy to understand as your toydist would be. But they are
mighty, ready to use, heavily tested, and maintained even if you don't find
the time to contribute.
I'm aware, that there might be issues that I did not mention here, and this
is meant as an idea and not as a complete solution...
Cheers,
Stefan
On Mon, Dec 28, 2009 at 15:03, David Cournapeau <cou...@gm...> wrote:
> (warning, long post)
>
> Hi there,
>
> As some of you already know, the packaging and distributions of
> scientific python packages have been a constant source of frustration.
> Open source is about making it easy for anyone to use software how
> they see fit, and I think python packaging infrastructure has not been
> very successfull for people not intimately familiar with python. A few
> weeks ago, after Guido visited Berkeley and was told how those issues
> were still there for the scientific community, he wrote an email
> asking whether current efforts on distutils-sig will be enough (see
> http://aspn.activestate.com/ASPN/Mail/Message/distutils-sig/3775972).
>
> Several of us have been participating to this discussion, but I feel
> like the divide between current efforts on distutils-sig and us (the
> SciPy community) is not getting smaller. At best, their efforts will
> be more work for us to track the new distribute fork, and more likely,
> it will be all for nothing as it won't solve any deep issue. To be
> honest, most of what is considered on distutils-sig sounds like
> anti-goals to me.
>
> Instead of keeping up with the frustrating process of "improving"
> distutils, I think we have enough smart people and manpower in the
> scientific community to go with our own solution. I am convinced it is
> doable because R or haskell, with a much smaller community than
> python, managed to pull out something with is miles ahead compared to
> pypi. The SciPy community is hopefully big enough so that a
> SciPy-specific solution may reach critical mass. Ideally, I wish we
> had something with the following capabilities:
>
> - easy to understand tools
> - http-based package repository ala CRAN, which would be easy to
> mirror and backup (through rsync-like tools)
> - decoupling the building, packaging and distribution of code and data
> - reliable install/uninstall/query of what is installed locally
> - facilities for building windows/max os x binaries
> - making the life of OS vendors (Linux, *BSD, etc...) easier
>
> The packaging part
> ==============
>
> Speaking is easy, so I started coding part of this toolset, called
> toydist (temporary name), which I presented at Scipy India a few days
> ago:
>
> http://github.com/cournape/toydist/
>
> Toydist is more or less a rip off of cabal
> (http://www.haskell.org/cabal/), and consist of three parts:
> - a core which builds a package description from a declarative file
> similar to cabal files. The file is almost purely declarative, and can
> be parsed so that no arbitrary code is executed, thus making it easy
> to sandbox packages builds (e.g. on a build farm).
> - a set of command line tools to configure, build, install, build
> installers (egg only for now) etc... from the declarative file
> - backward compatibility tools: a tool to convert existing setup.py
> to the new format has been written, and a tool to use distutils
> through the new format for backward compatibility with complex
> distutils extensions should be relatively easy.
>
> The core idea is to make the format just rich enough to describe most
> packages out there, but simple enough so interfacing it with external
> tools is possible and reliable. As a regular contributor to scons, I
> am all too aware that a build tool is a very complex beast to get
> right, and repeating their efforts does not make sense. Typically, I
> envision that complex packages such as numpy, scipy or matplotlib
> would use make/waf/scons for the build - in a sense, toydist is
> written so that writing something like numscons would be easier. OTOH,
> most if not all scikits should be buildable from a purely declarative
> file.
>
> To give you a feel of the format, here is a snippet for the grin
> package from Robert K. (automatically converted):
>
> Name: grin
> Version: 1.1.1
> Summary: A grep program configured the way I like it.
> Description:
> ====
> grin
> ====
>
> I wrote grin to help me search directories full of source code.
> The venerable
> GNU grep_ and find_ are great tools, but they fall just a little
> short for my
> normal use cases.
>
> <snip>
> License: BSD
> Platforms: UNKNOWN
> Classifiers:
> License :: OSI Approved :: BSD License,
> Development Status :: 5 - Production/Stable,
> Environment :: Console,
> Intended Audience :: Developers,
> Operating System :: OS Independent,
> Programming Language :: Python,
> Topic :: Utilities,
>
> ExtraSourceFiles:
> README.txt,
> setup.cfg,
> setup.py,
>
> Library:
> InstallDepends:
> argparse,
> Modules:
> grin,
>
> Executable: grin
> module: grin
> function: grin_main
>
> Executable: grind
> module: grin
> function: grind_main
>
> Although still very much experimental at this point, toydist already
> makes some things much easier than with distutils/setuptools:
> - path customization for any target can be done easily: you can
> easily add an option in the file so that configure --mynewdir=value
> works and is accessible at every step.
> - making packages FHS compliant is not a PITA anymore, and the scheme
> can be adapted to any OS, be it traditional FHS-like unix, mac os x,
> windows, etc...
> - All the options are accessible at every step (no more distutils
> commands nonsense)
> - data files can finally be handled correctly and consistently,
> instead of the 5 or 6 magics methods currently available in
> distutils/setuptools/numpy.distutils
> - building eggs does not involve setuptools anymore
> - not much coupling between package description and build
> infrastructure (building extensions is actually done through distutils
> ATM).
>
> Repository
> ========
>
> The goal here is to have something like CRAN
> (http://cran.r-project.org/web/views/), ideally with a build farm so
> that whenever anyone submits a package to our repository, it would
> automatically be checked, and built for windows/mac os x and maybe a
> few major linux distributions. One could investigate the build service
> from open suse to that end (http://en.opensuse.org/Build_Service),
> which is based on xen VM to build installers in a reproducible way.
>
> Installed package db
> ===============
>
> I believe that the current open source enstaller package from
> Enthought can be a good starting point. It is based on eggs, but eggs
> are only used as a distribution format (eggs are never installed as
> eggs AFAIK). You can easily remove packages, query installed versions,
> etc... Since toydist produces eggs, interoperation between toydist and
> enstaller should not be too difficult.
>
> What's next ?
> ==========
>
> At this point, I would like to ask for help and comments, in particular:
> - Does all this make sense, or hopelessly intractable ?
> - Besides the points I have mentioned, what else do you think is needed ?
> - There has already been some work for the scikits webportal, but I
> think we should bypass pypi entirely (the current philosophy of not
> enforcing consistent metadata does not make much sense to me, and is
> at the opposite of most other similar system out there).
> - I think a build farm for at least windows packages would be a
> killer feature, and enough incentive to push some people to use our
> new infrastructure. It would be good to have a windows guy familiar
> with windows sandboxing/virtualization to do something there. The
> people working on the opensuse build service have started working on
> windows support
> - I think being able to automatically convert most of scientific
> packages is a significant feature, and needs to be more robust - so
> anyone is welcomed to try converting existing setup.py with toydist
> (see toydist readme).
>
> thanks,
>
> David
>
>
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and
> easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

(warning, long post)
Hi there,
 As some of you already know, the packaging and distributions of
scientific python packages have been a constant source of frustration.
Open source is about making it easy for anyone to use software how
they see fit, and I think python packaging infrastructure has not been
very successfull for people not intimately familiar with python. A few
weeks ago, after Guido visited Berkeley and was told how those issues
were still there for the scientific community, he wrote an email
asking whether current efforts on distutils-sig will be enough (see
http://aspn.activestate.com/ASPN/Mail/Message/distutils-sig/3775972).
Several of us have been participating to this discussion, but I feel
like the divide between current efforts on distutils-sig and us (the
SciPy community) is not getting smaller. At best, their efforts will
be more work for us to track the new distribute fork, and more likely,
it will be all for nothing as it won't solve any deep issue. To be
honest, most of what is considered on distutils-sig sounds like
anti-goals to me.
Instead of keeping up with the frustrating process of "improving"
distutils, I think we have enough smart people and manpower in the
scientific community to go with our own solution. I am convinced it is
doable because R or haskell, with a much smaller community than
python, managed to pull out something with is miles ahead compared to
pypi. The SciPy community is hopefully big enough so that a
SciPy-specific solution may reach critical mass. Ideally, I wish we
had something with the following capabilities:
 - easy to understand tools
 - http-based package repository ala CRAN, which would be easy to
mirror and backup (through rsync-like tools)
 - decoupling the building, packaging and distribution of code and data
 - reliable install/uninstall/query of what is installed locally
 - facilities for building windows/max os x binaries
 - making the life of OS vendors (Linux, *BSD, etc...) easier
The packaging part
==============
Speaking is easy, so I started coding part of this toolset, called
toydist (temporary name), which I presented at Scipy India a few days
ago:
http://github.com/cournape/toydist/
Toydist is more or less a rip off of cabal
(http://www.haskell.org/cabal/), and consist of three parts:
 - a core which builds a package description from a declarative file
similar to cabal files. The file is almost purely declarative, and can
be parsed so that no arbitrary code is executed, thus making it easy
to sandbox packages builds (e.g. on a build farm).
 - a set of command line tools to configure, build, install, build
installers (egg only for now) etc... from the declarative file
 - backward compatibility tools: a tool to convert existing setup.py
to the new format has been written, and a tool to use distutils
through the new format for backward compatibility with complex
distutils extensions should be relatively easy.
The core idea is to make the format just rich enough to describe most
packages out there, but simple enough so interfacing it with external
tools is possible and reliable. As a regular contributor to scons, I
am all too aware that a build tool is a very complex beast to get
right, and repeating their efforts does not make sense. Typically, I
envision that complex packages such as numpy, scipy or matplotlib
would use make/waf/scons for the build - in a sense, toydist is
written so that writing something like numscons would be easier. OTOH,
most if not all scikits should be buildable from a purely declarative
file.
To give you a feel of the format, here is a snippet for the grin
package from Robert K. (automatically converted):
Name: grin
Version: 1.1.1
Summary: A grep program configured the way I like it.
Description:
 ====
 grin
 ====
 I wrote grin to help me search directories full of source code.
The venerable
 GNU grep_ and find_ are great tools, but they fall just a little
short for my
 normal use cases.
 <snip>
License: BSD
Platforms: UNKNOWN
Classifiers:
 License :: OSI Approved :: BSD License,
 Development Status :: 5 - Production/Stable,
 Environment :: Console,
 Intended Audience :: Developers,
 Operating System :: OS Independent,
 Programming Language :: Python,
 Topic :: Utilities,
ExtraSourceFiles:
 README.txt,
 setup.cfg,
 setup.py,
Library:
 InstallDepends:
 argparse,
 Modules:
 grin,
Executable: grin
 module: grin
 function: grin_main
Executable: grind
 module: grin
 function: grind_main
Although still very much experimental at this point, toydist already
makes some things much easier than with distutils/setuptools:
 - path customization for any target can be done easily: you can
easily add an option in the file so that configure --mynewdir=value
works and is accessible at every step.
 - making packages FHS compliant is not a PITA anymore, and the scheme
can be adapted to any OS, be it traditional FHS-like unix, mac os x,
windows, etc...
 - All the options are accessible at every step (no more distutils
commands nonsense)
 - data files can finally be handled correctly and consistently,
instead of the 5 or 6 magics methods currently available in
distutils/setuptools/numpy.distutils
 - building eggs does not involve setuptools anymore
 - not much coupling between package description and build
infrastructure (building extensions is actually done through distutils
ATM).
Repository
========
The goal here is to have something like CRAN
(http://cran.r-project.org/web/views/), ideally with a build farm so
that whenever anyone submits a package to our repository, it would
automatically be checked, and built for windows/mac os x and maybe a
few major linux distributions. One could investigate the build service
from open suse to that end (http://en.opensuse.org/Build_Service),
which is based on xen VM to build installers in a reproducible way.
Installed package db
===============
I believe that the current open source enstaller package from
Enthought can be a good starting point. It is based on eggs, but eggs
are only used as a distribution format (eggs are never installed as
eggs AFAIK). You can easily remove packages, query installed versions,
etc... Since toydist produces eggs, interoperation between toydist and
enstaller should not be too difficult.
What's next ?
==========
At this point, I would like to ask for help and comments, in particular:
 - Does all this make sense, or hopelessly intractable ?
 - Besides the points I have mentioned, what else do you think is needed ?
 - There has already been some work for the scikits webportal, but I
think we should bypass pypi entirely (the current philosophy of not
enforcing consistent metadata does not make much sense to me, and is
at the opposite of most other similar system out there).
 - I think a build farm for at least windows packages would be a
killer feature, and enough incentive to push some people to use our
new infrastructure. It would be good to have a windows guy familiar
with windows sandboxing/virtualization to do something there. The
people working on the opensuse build service have started working on
windows support
 - I think being able to automatically convert most of scientific
packages is a significant feature, and needs to be more robust - so
anyone is welcomed to try converting existing setup.py with toydist
(see toydist readme).
thanks,
David