homepage

Here’s a patch adding Lib/packaging subdirectories to Makefile.pre.in. Because of our extensive tests for pypi-related code, the list is huge. Maybe it should use a call to find(1) instead of using a fragile manually-maintained list, so I’m asking here.

Listing all those directories to the Makefile does seem very brittle and will certainly be a maintenance hassle. OTOH, there is something to be said for being explicit about what gets installed. I suppose you could add the find step to ./configure and pass the list into the Makefile. Dunno if that is really is a good idea, though.
In any case, if a reminder isn't already there, perhaps one should be added to the devguide or packaging docs about this when adding test directories.
Also you may want to add some of the packaging test directories to those excluded in the calls to compileall.py a little further down in the libinstall target.

> Also you may want to add some of the packaging test directories to
> those excluded in the calls to compileall.py a little further down in
> the libinstall target.
Ah, good one. I guess doing this could have prevented http://hg.python.org/cpython/rev/9041520be581 

Nice to see my search didn't find this bug. ;)
I already committed a change to install packaging, but you may want to revert that and commit the patch in this issue. I'll leave that for you to decide!

> Nice to see my search didn't find this bug. ;)
Do you remember what search terms you tried?
> I already committed a change to install packaging, but you may want to
> revert that and commit the patch in this issue.
I honestly don’t know. Do we have a policy about using intermediary variables (like XMLLIBSUBDIRS) or not? Can we assume find(1) exists on the system and use it?

It's not packaging-related, but other directories are also missing from LIBSUBDIRS:
test/test_email test/test_email/data
IMO it's not ideal to use find(1), as in theory a developer may have some directories with matching names in the source build that they don't want installed - or would that be a no-no?
Ideally at least one buildbot should be set up to install and then test the new installation - I presume this error wasn't caught on the buildbots as they all test source builds?

Fixing the missing dirs for email has been on my todo list for a while but obviously hasn't gotten done yet. I talked with Antoine about setting up a buildbot that tested an installed version, but I'm not sure when I'll get around to trying *that* either. And in any case, even that would not have caught the test_email omission, since rergrtest uses filename-based discovery to select the tests to run, so it would not have noticed that test_email was missing. I suppose that is a disadvantage of having regrtest recognize test packages.

Perhaps not in the general case, but I found the missing test_email and test_email/data through running regrtest using an installed Python 3.3.

Ah. I wouldn't have expected that. For my information, what was the error you got?

> R. David Murray <rdmurray@bitdance.com> added the comment:
> 
> Ah. I wouldn't have expected that. For my information, what was the error 
>you got?
[ 99/356] test_email
test test_email crashed -- Traceback (most recent call last):
 File "regrtest.py", line 1036, in runtest_inner
ImportError: No module named 'test_email'
From the full test results at
https://gist.github.com/1022705/45fa6992c25197a3c9f6281fc645da94dcc5feae
N.B. I have now fixed the error in my fork, you can see the change I made here:
https://bitbucket.org/vinay.sajip/pythonv/compare/..mirror/cpython#chg-Makefile.pre.in
Regards,
Vinay Sajip

OK, now I'm really confused. I tried to reproduce this by installing from my checkout into a work dir, and then running regrtest using that installed python, and discovered that test_email and its data directory got copied by the installation process. (There were test failures, so I'll have to work on that...) This is without any mention of test_email in the Makefile I used to do the 'make install'.

> OK, now I'm really confused. I tried to reproduce this by installing from my 
>checkout into a work dir, and then running regrtest using that installed 
>python, and discovered that test_email and its data directory got copied by the 
>installation process. (There were test failures, so I'll have to work on 
>that...) This is without any mention of test_email in the Makefile I used to 
>do the 'make install'.
That does seem odd. On my pythonv fork (which follows the cpython repo pretty 
closely), the only current failures are test_lib2to3, test_packaging and 
test_sysconfig - all of which failures are already the subject of tickets on the 
tracker. On my system at least, test_email doesn't show up any failures. See the 
latest version of the test log from the Gist link I posted earlier - the comment 
also references the failure tickets.

New changeset c9d27c63b45c by R David Murray in branch 'default':
#12313: update Makefile.pre.in to account for email tests moving to 'test' dir
http://hg.python.org/cpython/rev/c9d27c63b45c 

Well, regardless of whether or not I understand what's going on, clearly those directories in the makefile needed updating, so I did it.

The changes have been checked in by Barry and David, so I'm closing this issue.