homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: mmap.mmap() allocates a file descriptor that isn't CLOEXEC
Type: resource usage Stage: needs patch
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Python 2.7: make private file descriptors non inheritable
View: 26769
Assigned To: Nosy List: Arfrever, christian.heimes, docs@python, martin.panter, neologix, rfm, vstinner
Priority: normal Keywords:

Created on 2013年10月30日 05:19 by rfm, last changed 2022年04月11日 14:57 by admin. This issue is now closed.

Messages (9)
msg201709 - (view) Author: Robert Merrill (rfm) Date: 2013年10月30日 05:19
Same code but different problem as this issue: http://bugs.python.org/issue10897
The above-linked issue was closed as WONTFIX, but there is a secondary problem: the file descriptor that mmap.mmap() allocates is not set to close-on-exec.
This means that any process you spawn using os.system() and the like, its children, its children's children etc will inherit this file descriptor.
This is unexpected, unnecessary behavior, and potentially a huge pain if you mmap() a file and then spawn, say, inetd or something that does similar stuff, because now you have to kill every process that spawned from that in order to unmount filesystems or remove kernel modules that are being blocked as a result of the fds being open.
msg201710 - (view) Author: Robert Merrill (rfm) Date: 2013年10月30日 05:25
I should add a caveat: the fd that is created by mmap /will/ be set to close-on-exec if the fd you passed in was. But even if it's not, I see no reason why mmap should not be setting it anyway.
At the very least, the documentation should bring the user's attention to this potential stumbling block. Currently it makes no mention of the fact that mmap duplicates file descriptors at all, much less how the flags are set.
msg201718 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013年10月30日 09:23
Can you suggest a documentation update?
msg201720 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013年10月30日 09:33
"the file descriptor that mmap.mmap() allocates is not set to close-on-exec"
In Python 3.4, the file descriptor is now non-inheritable, as a side effect of the PEP 446.
http://www.python.org/dev/peps/pep-0446/ 
msg201722 - (view) Author: Robert Merrill (rfm) Date: 2013年10月30日 09:45
I'm adding Library again because I think the current behavior is a bug and should be fixed in the 2.7 tree. Perhaps the documentation in older versions should be updated 
mmap.mmap should always set the FD_CLOEXEC flag on the descriptor that it gets from dup(). I don't see why we wouldn't do this, because if we are calling exec(), we are blowing away all python state, and only the mmap object should be using this file descriptor (since its existence is hidden from the user).
Users should not be depending on the old behavior, because the interface which is presented to them gives them no indication that this fd even exists. Users are probably expecting the proposed behavior, because anyone who has worked with unix file descriptors in C would not expect:
fd = os.open(...)
mmap = mmap.mmap(fd, size)
# do some stuff
os.close(fd)
To result in an extra inheritable fd still being there. Nothing in the documentation indicates that this would happen, either.
msg201726 - (view) Author: Robert Merrill (rfm) Date: 2013年10月30日 10:34
Sorry, I correct my earlier statement: even if the fd you pass to mmap.mmap() is set to FD_CLOEXEC, the dup'd fd /will not be/
So this is a REALLY bad bug because users cannot workaround it except by just not using mmap
msg201755 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013年10月30日 17:47
I agree this should be fixed.
Robert, want to submit a patch?
msg201854 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013年10月31日 22:24
> Robert, want to submit a patch?
If someone wants to work on a patch, here is the code of Python 3.4 to duplicate a file descriptor and make it non-inheritable.
 http://hg.python.org/cpython/file/e97d9926da83/Python/fileutils.c#l978
and
 http://hg.python.org/cpython/file/e97d9926da83/Python/fileutils.c#l618
The code is complex because it has optimized versions for each operating system. Only supporting Windows and fcntl() is enough for Python 2.
msg265050 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016年05月07日 08:13
Closing because Victor has proposed a patch in Issue 26769.
History
Date User Action Args
2022年04月11日 14:57:52adminsetgithub: 63643
2016年05月07日 08:13:57martin.pantersetstatus: open -> closed

nosy: + martin.panter
messages: + msg265050

superseder: Python 2.7: make private file descriptors non inheritable
resolution: duplicate
2016年04月15日 09:16:22berker.peksagsetassignee: docs@python ->
components: - Documentation
versions: - Python 3.3
2013年10月31日 22:24:11vstinnersetmessages: + msg201854
2013年10月30日 23:11:54Arfreversetnosy: + Arfrever
2013年10月30日 17:47:07neologixsetmessages: + msg201755
2013年10月30日 10:34:27rfmsetmessages: + msg201726
2013年10月30日 09:45:36rfmsetmessages: + msg201722
components: + Library (Lib)
versions: - Python 3.4
2013年10月30日 09:35:24vstinnersetnosy: + neologix
2013年10月30日 09:33:43vstinnersetmessages: + msg201720
versions: + Python 3.3, Python 3.4, - Python 3.2
2013年10月30日 09:23:44christian.heimessetassignee: docs@python
components: + Documentation, - Library (Lib)
versions: + Python 3.2
nosy: + christian.heimes, docs@python

messages: + msg201718
stage: needs patch
2013年10月30日 09:12:06pitrousetnosy: + vstinner
2013年10月30日 05:25:24rfmsetmessages: + msg201710
2013年10月30日 05:19:09rfmcreate

AltStyle によって変換されたページ (->オリジナル) /