[Python-Dev] PEP 365 (Adding the pkg_resources module)

Thu Mar 20 05:18:47 CET 2008

On Mar 19, 2008, at 3:23 PM, Guido van Rossum wrote:
> If other people want to chime in please do so; if this is just a
> dialog between Phillip and me I might incorrectly assume
> that nobody besides Phillip really cares.

I really care. I've used setuptools, easy_install, eggs, and 
pkg_resources extensively for the past year or so (and contributed a 
few small patches). There have been plenty of problems, but I find 
them to be overall useful tools.
It is a great boon to a programming community to lower the costs of 
re-using other people's code. The Python community will benefit 
greatly once a way to do that becomes widely enough accepted to reach 
a tipping point and become ubiquitous. Setuptools is already the de 
facto standard, but it hasn't become ubiquitous, possibly in part 
because of "egg hatred", about which more below.
I've interviewed several successful Python hackers who "hate eggs" in 
order to understand what they hate about them, and I've taken notes 
from some of these interviews. (The list includes MvL, whose name 
was invoked earlier in this thread.)
After filtering out yer basic complaining about bugs (which 
complaints are of course legitimate, but which don't indict 
setuptools as worse than other software of comparable scope and 
maturity), their objections seem to fall into two categories:
1. "The very notion of package dependency resolution and 
programmable or command-line installation of packages at the language 
level is a bad notion."
This can't really be the case. If the existence of such 
functionality at the programming language level were an inherently 
bad notion, then we would be hearing some complaints from the Ruby 
folks, where the Gems system is standard and ubiquitous. We hear no 
complaints -- only murmurs of satisfaction. One person recently 
reported to me that while there are more packages in Python, he finds 
himself re-using other people's code more often when he works in 
Ruby, because almost all Ruby software is Gemified, but only a 
fraction of Python software is Eggified.
Often this complaint comes with the idea that eggs conflict with 
their system-level package management tools. (These are usually 
Debian/Ubuntu users.)
Note that Ruby software is not too hard to include in operating 
system packaging schemes -- my Ubuntu Hardy apt-cache shows plenty of 
Ruby software. A sufficiently mature and widely supported setuptools 
could actually make it easier to integrate Python software into 
Debian -- see stdeb [1].
2. "Setuptools/eggs give me grief."
What can really be the case is that setuptools causes a host of 
small, unnecessary problems for people who prefer to do things 
differently than PJE does. Personally, I prefer to use GNU stow, and 
setuptools causes unnecessary, but avoidable, problems for me. Many 
people object (rightly enough) to a "./setup.py install" 
automatically fetching new software over the Internet by default. 
The fact that easy_install creates a site.py that changes the 
semantics of PYTHONPATH is probably the most widely and deservedly 
hated example of this kind of thing [2]. I could go on with a few 
other common technical complaints of this kind.
These type-2 problems can be fixed by changing setuptools or they can 
be grudgingly accepted by users, while retaining compatibility with 
the large and growing ecosystem of eggy software. Certainly fixing 
setuptools to play better with others is a more likely path to 
success than setting out to invent a non-egg-compatible alternative. 
Such a project might never be implemented well enough to serve, and 
if it were it would probably never overtake eggs's lead in the Python 
ecosystem, and if it did it would probably not turn out to be a 
better tool.
So, since you asked for my chime, I advise you to publically bless 
eggs, setuptools, and easy_install as plausible future standards and 
solicit patches which address the complaints. For that matter, 
soliciting specific complaints would be a good start. I've done so 
in private many times with only partial success as to the "specific" 
part. One promising approach is to request objections in the form of 
automated tests that setuptools fails, e.g. [3].
Regards,
Zooko O'Whielacronx
[1] http://stdeb.python-hosting.com/
[2] http://www.rittau.org/blog/20070726-02
 And no, PJE's suggested "trivial fix" does not satisfy the 
objectors, as it can't support the use case of "cd somepkg ; python ./ 
setup.py install ; cd .. ; python -c 'import somepkg'".
[3] http://twistedmatrix.com/trac/ticket/2308#comment:5