homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: OSError exception in multiprocessing module when using os.remove() on NFS
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.2
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: jjardon, neologix
Priority: normal Keywords:

Created on 2012年02月09日 18:24 by jjardon, last changed 2022年04月11日 14:57 by admin. This issue is now closed.

Messages (3)
msg152972 - (view) Author: Javier Jardón (jjardon) Date: 2012年02月09日 18:24
I have this little test case:
import multiprocessing
 
manager = multiprocessing.Manager()
del manager
and I get this:
Traceback (most recent call last):
 File "/usr/lib/python2.7/multiprocessing/util.py", line 261, in _run_finalizers
 finalizer()
 File "/usr/lib/python2.7/multiprocessing/util.py", line 200, in __call__
 res = self._callback(*self._args, **self._kwargs)
 File "/usr/lib/python2.7/shutil.py", line 249, in rmtree
 onerror(os.remove, fullname, sys.exc_info())
 File "/usr/lib/python2.7/shutil.py", line 247, in rmtree
 os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/nfs/tmp/pymp-f7R9S6/.nfs00000000e039692f00000236'
(the TMDIR directory is in a nfs server)
msg153071 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2012年02月10日 18:42
"""
OSError: [Errno 16] Device or resource busy: '/nfs/tmp/pymp-f7R9S6/.nfs00000000e039692f00000236'
"""
That's because the temporary directory is removed while a file inside is still open.
And that's really likely the unix socket used by the server's listener.
What happens if you do this instead:
"""
import multiprocessing
 
manager = multiprocessing.Manager()
manager.shutdown()
del manager
"""
You should shutdown the manager before it gets garbage collected.
> (the TMDIR directory is in a nfs server)
Bad idea, for the following reasons:
- you're actually lucky Linux allows binding unix sockets over NFS filesystems, some Unix flavors don't
- you're likely to run into similar problems, because code which removes a temporary directory while having still an open FD is quite common, and will break with a tmp dir over NFS
- mkstemp() and friends use O_EXCL to create a temporary file securely, and some NFS implementations of O_EXCL are unsafe (should be OK with NFSv3 and later though)
- it's not a good idea performance wise
msg153640 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2012年02月18日 11:56
I'm closing, since the manager should be shutdown (and TMPDIR on NFS isn't a good idea).
History
Date User Action Args
2022年04月11日 14:57:26adminsetgithub: 58186
2012年02月18日 11:56:29neologixsetstatus: open -> closed
type: crash -> behavior
messages: + msg153640

resolution: not a bug
stage: resolved
2012年02月10日 18:42:52neologixsetnosy: + neologix
messages: + msg153071
2012年02月09日 18:24:08jjardoncreate

AltStyle によって変換されたページ (->オリジナル) /