This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2016年04月26日 13:47 by vocdetnojz, last changed 2022年04月11日 14:58 by admin. This issue is now closed.
| Messages (6) | |||
|---|---|---|---|
| msg264286 - (view) | Author: Vukasin Felbab (vocdetnojz) | Date: 2016年04月26日 13:47 | |
shutil.copyfile() doesn't close the opened files, so it is vulnerable to IO Error 24: too many files open actually, the src and dst files should be closed after copy |
|||
| msg264294 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2016年04月26日 14:33 | |
Could you provide an example? |
|||
| msg282042 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2016年11月29日 20:32 | |
As I see, shutil.copyfile() uses the "with" statements and closes files just after copying. |
|||
| msg282051 - (view) | Author: Josh Rosenberg (josh.r) * (Python triager) | Date: 2016年11月29日 22:55 | |
Agreed. 2.7 source is definitely using with: https://hg.python.org/cpython/file/2.7/Lib/shutil.py#l82 |
|||
| msg282232 - (view) | Author: Vukasin Felbab (vocdetnojz) | Date: 2016年12月02日 09:10 | |
Okay, so the problem is that I have a command library instance which is containing a connection to a server and functions can be called to execute some queries. As I pass this instance as a parameter in a recursive chain, after a while the open files limit is reached, I got it a few times an error on file opening. It is sure that I don't leave connections and open files before calling the next recursion, only thing this command library has the connection open. Additionally, when I store the instance in a global variable and call it from there then there are no IO errors. So, it means that the python is not giving parameters by reference, but by value? Because according to the Python, if I pass a mutable object as an argument, an if I use that for querying and passing the same instance to the next recursion level it should work fine, because in this case it should pass by reference, but it is not obviously, it passes by value, the connections remain open and the io limit is reached. |
|||
| msg282238 - (view) | Author: Josh Rosenberg (josh.r) * (Python triager) | Date: 2016年12月02日 13:52 | |
You're going to need to provide a real repro; your description is effectively useless. For the record, Python is not precisely pass by reference (it's roughly equivalent to passing a pointer in C, binding the local name to the same pointer, so if you reassign the local name, you lose the reference, but otherwise it behaves like pass-by-reference). Are you sure the command library doesn't open new connections as needed? In a recursive call scenario, if it goes deep enough, even if it clean up with the recursive call eventually returns, you could still hit the open files limit. The problem is not copyfile, even if you happen to see copyfile in the traceback. You've got too many open files, and copyfile is the straw that broke the camel's back, but unless you're running 1000 threads doing copyfile simultaneously, it's not responsible for the excess open file handles. But this is all speculation; the only thing I'm confident of is that copyfile isn't ultimately responsible. We have no idea what's wrong, so you need to provide some sort of minimal repro that exhibits the problem. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:58:30 | admin | set | github: 71048 |
| 2017年10月23日 14:55:37 | serhiy.storchaka | set | status: open -> closed resolution: not a bug stage: resolved |
| 2016年12月02日 13:52:24 | josh.r | set | messages: + msg282238 |
| 2016年12月02日 09:10:05 | vocdetnojz | set | status: pending -> open messages: + msg282232 |
| 2016年12月02日 06:03:08 | serhiy.storchaka | set | status: open -> pending |
| 2016年11月29日 22:55:39 | josh.r | set | status: pending -> open nosy: + josh.r messages: + msg282051 |
| 2016年11月29日 20:32:13 | serhiy.storchaka | set | status: open -> pending messages: + msg282042 |
| 2016年04月26日 14:33:48 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages: + msg264294 |
| 2016年04月26日 13:47:15 | vocdetnojz | create | |