homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: gdbm reorganize() leaves hanging file descriptor
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: jcea, sam-s, themoken
Priority: normal Keywords:

Created on 2012年02月06日 04:13 by themoken, last changed 2022年04月11日 14:57 by admin. This issue is now closed.

Messages (7)
msg152719 - (view) Author: Jack Miller (themoken) Date: 2012年02月06日 04:13
I've discovered that using either 2.7.2 or 3.2.2 (from Arch) reorganizing gdbm databases leave hanging file descriptors that will cause EAGAIN errors trying to reopen it until the process is killed. For example:
Python 2.7.2 (default, Jan 31 2012, 13:19:49) 
[GCC 4.6.2 20120120 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gdbm
>>> o = gdbm.open("test27", "c")
>>> o.reorganize()
>>> o.close()
>>> o = gdbm.open("test27", "w")
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
gdbm.error: (11, 'Resource temporarily unavailable')
>>>
By using lsof you can see that after reorganize() test27 has two open file descriptors, and after close() one is still open with no (obvious) way to close it. This happens with freshly created and populated databases and regardless of open mode flags.
I also tested this on 3.2.2 with dbm.gnu and got identical behavior.
msg152720 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2012年02月06日 04:48
I can not reproduce the issue with GDBM 1.9.1, neither GDBM 1.8.3, neither GDBM 1.10, under Solaris.
I don't see the file descriptor leak either.
According to http://sunsite.ualberta.ca/Documentation/Gnu/gdbm-1.8.0/html_node/gdbm_10.html, all file descriptor management is done inside GDBM.
What GDBM are you using?. Could you possibly upgrade?. Beware, 1.8 and 1.9/1.10 fileformats are *NOT* compatible.
msg152721 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2012年02月06日 04:52
Can't reproduce either in Ubuntu 10.04, 64 bits, with GDBM 1.8.3.
Closing bug.
Reporter, please indicate your GDBM version. Can you possibly upgrade it?.
msg152722 - (view) Author: Jack Miller (themoken) Date: 2012年02月06日 05:22
I am using gdbm 1.10, the latest available.
pacman says:
core/gdbm 1.10-1 [installed]
 GNU database library
Which also contains the file (libgdbm.so.4) that Python's gdbm.so is linked to (just in case there was some version trickery going on here).
I guess it's possible this is Arch brokenness, but the only patch applied to the 1.10 code in the PKGBUILD is malloc -> calloc for a known bug.
msg153050 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2012年02月10日 14:46
I can't reproduce. Sorry.
Reading the GDBM documentation, all filedescriptors management seem to be inside gdbm, not Python. Anyway, if I can't reproduce, I can not diagnose neither patch. Sorry.
msg339398 - (view) Author: sds (sam-s) Date: 2019年04月03日 15:58
I observe the same behavior on Linux 4.18.0-17-generic #18-Ubuntu
with Python 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0].
gdbm 1.14.1-6:
after
db = dbm.gnu.open("asdf","n")
db["a"] = "124"
db.reorganize()
db.close()
I see
$ lsof ~/asdf 
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python3 22071 sds mem-W REG 8,2 16384 4328326 /home/sds/asdf
and
>>> db = dbm.gnu.open("asdf","r")
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/sds/src/pyapp/tests/fileconv_test.py", line 1053, in <module>
_gdbm.error: [Errno 11] Resource temporarily unavailable
msg339668 - (view) Author: sds (sam-s) Date: 2019年04月08日 19:39
Upstream claims that this is fixed by http://git.gnu.org.ua/cgit/gdbm.git/commit/?id=a0d6c1a8 
History
Date User Action Args
2022年04月11日 14:57:26adminsetgithub: 58155
2019年04月08日 19:39:39sam-ssetmessages: + msg339668
2019年04月03日 15:59:33sam-ssetversions: + Python 3.6, - Python 3.7
2019年04月03日 15:59:05sam-ssettype: behavior
2019年04月03日 15:58:44sam-ssetcomponents: + Library (Lib)
versions: + Python 3.7, - Python 2.7, Python 3.2
2019年04月03日 15:58:04sam-ssetnosy: + sam-s
messages: + msg339398
2012年02月10日 14:46:47jceasetmessages: + msg153050
2012年02月06日 05:22:13themokensetmessages: + msg152722
2012年02月06日 04:52:03jceasetmessages: + msg152721
stage: resolved
2012年02月06日 04:48:24jceasetstatus: open -> closed

nosy: + jcea
messages: + msg152720

resolution: works for me
2012年02月06日 04:13:35themokencreate

AltStyle によって変換されたページ (->オリジナル) /