homepage

NFS on certain platforms (most notably AIX, Solaris) use .nfsXXXXX files that appear and disappear automagically. distutils can get confused by that once a .nfsXXXXX file was earlier seen with listdir and then removed by the OS before its copied.
Simply ignore .nfsXXXXX files as workaround.

Here is the applicable patch against py3k branch.

Thanks for the report and patch (I think SilentGhost’s version is better). Do you have any pointer about those .nfs* files? Are there other (build) tools that ignore them? Is it always safe to skip .nfs* files, or only .nfs????? (i.e. six characters)?
Aside from those questions, a patch requires a test and doc update. See guidelines at http://wiki.python.org/moin/Distutils/FixingBugs 

On Tue, Feb 8, 2011 at 23:40, Éric Araujo <report@bugs.python.org> wrote:
>
> Éric Araujo <merwok@netwok.org> added the comment:
>
> Do you have any pointer about those .nfs* files? Are there other (build) tools that ignore them? Is it always safe to skip .nfs* files, or only .nfs????? (i.e. six characters)?
Just replying for this part: .nfs* files are created by the nfs server
when on of its client removes a file while another client has the very
same file opened. the nfs server keeps the .nfs* file around until the
last client closes the file, after that it removes the .nfs* file.
About the number of digits after '.nfs' I'm not sure there's a
standard for it, but I saw files with more that 6 chars after that.
Regards,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi 

Review of both patches (python-2.5.1-distutils-aixnfs.patch and dir_util.py.diff) as they are essentially the same:
I think that the test is in wrong place: we would want to ignore those .nfs* files always, not just when checking for symlinks. A separate test
at the top of the loop would be better:
 for n in names:
+ if n.startswith('.nfs'):
+ continue
 src_name = os.path.join(src, n)
 dst_name = os.path.join(dst, n)
 if preserve_symlinks and os.path.islink(src_name):
BTW: under linux 2.6.20 I see files like: .nfs000000000592413900000036, always of this length, with hexadecimal digits after .nfs).

Thanks to Sandro’s explanation and a passage in the Unix Haters Handbook, I now think that ignoring .nfs* files is a safe change. If should not be hard to add a test for this in test_sdist.

I've updated SilentGhost's patch to include a test and documentation.

Jeff,
the patch LGTM but Eric indicated he'd like to have a higher level test inside Lib/distutils/tests/test_sdist.py. Possibly as part of a bigger test like test_prune_file_list or test_add_defaults; no need to remove the old test though I guess.
P.S. There's an extra empty line inside of apiref.rst. You may want to strip it from your next patch.

Jeff, are you still interested in updating your patch or would you prefer if someone else tackled it?

Wouldn't it be better to add an 'ignore' option to the copy_tree() method with an optional list of patterns to ignore instead of hardcoding '.nsfXXX' files?
This would make it possible to also skip '.hg', 'CVS' artifacts.

No improvements are done in distutils, only the smallest possible changes to fix bugs. In distutils2 we use shutil.copytree.

Hynek,
I would indeed like to update this patch. I will make the updates you have suggested.

Attached the updated patch. I have added a .nfs file to test_prune_file_list in test_sdist. I've also cleaned up the documentation to read a little better (IMO).

Thanks, will apply.

SilentGhost, could you submit a contributor agreement to the PSF for your patches? See http://www.python.org/psf/contrib/ . Thanks.

New changeset a56cebff113a by Éric Araujo in branch '2.7':
Ignore .nfs* files in distutils (#7719).
http://hg.python.org/cpython/rev/a56cebff113a 

New changeset d978828bfd0e by Éric Araujo in branch '3.2':
Ignore .nfs* files in distutils (#7719).
http://hg.python.org/cpython/rev/d978828bfd0e
New changeset 10ab746f55fb by Éric Araujo in branch '3.3':
Merge fixes for #13614, #13512 and #7719 from 3.2
http://hg.python.org/cpython/rev/10ab746f55fb
New changeset b10c1c6f869f by Éric Araujo in branch 'default':
Merge fixes for #13614, #13512 and #7719 from 3.3
http://hg.python.org/cpython/rev/b10c1c6f869f 

Thanks all!