homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: [doc] os.walk is limited by python's recursion limit
Type: Stage: resolved
Components: Documentation Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Elena.Oat, Sergey Chvalyuk, Thomas.Waldmann, docs@python, iritkatriel, rhettinger, rmast, slateny
Priority: low Keywords: easy

Created on 2016年03月12日 00:59 by Thomas.Waldmann, last changed 2022年04月11日 14:58 by admin. This issue is now closed.

Messages (7)
msg261626 - (view) Author: Thomas Waldmann (Thomas.Waldmann) Date: 2016年03月12日 00:58
os.walk calls itself recursively and that limits the directory depth it can "walk" into.
On Linux, one can create directory hierarchies deeper than that.
For some more details see there:
https://github.com/borgbackup/borg/issues/380 
msg261627 - (view) Author: Thomas Waldmann (Thomas.Waldmann) Date: 2016年03月12日 01:07
Note: similar issues can be seen in other stdlib recursive filesystem-related functions also.
msg261681 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016年03月13日 07:22
I think the code should be left as-is (we've used some variation of recursion for walking a very long time with no reported real-world problems). Perhaps a documentation note can be added to the effect that the there is a recursion limit and that it can be changed by the user if needed.
msg262064 - (view) Author: Elena Oat (Elena.Oat) * Date: 2016年03月20日 10:07
I've actually tried to test the recursion limit on my Mac OS X. It seems that I cannot create any subdirectories after around 500 subdir depth. I guess it's related to the allowed path length (<1024 chars?).
msg268116 - (view) Author: Sergey Chvalyuk (Sergey Chvalyuk) Date: 2016年06月10日 11:56
There is another limitation with os.walk
it also cannot go very deep inside tree because of scandir can throw 
OSError(36, 'File name too long')
I have wrote tests for both cases:
https://gist.github.com/grubberr/3367f1d605e292103b576a17a46ef3c3
tested on linux only
on MacOS it throw 'File name too long' for 1-st test too
set sys.settrecursionlimit to lower values
msg414334 - (view) Author: Stanley (slateny) * Date: 2022年03月02日 09:43
I'm not too sure about documenting the recursive limit here. There's a few other recursive functions in the os library (such as makedirs()) and if we note the recursive limit for os.walk then all of them should be noted too, but that doesn't seem quite right to me.
msg415330 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022年03月16日 09:17
I agree with Stanley. The documentation for os is clear that recursion is used and the documentation for RecursionError links to getrecursionlimit(). This seems sufficient.
History
Date User Action Args
2022年04月11日 14:58:28adminsetgithub: 70732
2022年03月16日 09:17:27iritkatrielsetstatus: open -> closed

nosy: + iritkatriel
messages: + msg415330

resolution: wont fix
stage: resolved
2022年03月02日 09:43:54slatenysetnosy: + slateny
messages: + msg414334
2021年08月18日 06:30:22rmastsetnosy: + rmast
2021年06月22日 13:42:59iritkatrielsetkeywords: + easy
title: os.walk is limited by python's recursion limit -> [doc] os.walk is limited by python's recursion limit
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.5
2016年06月10日 11:56:12Sergey Chvalyuksetnosy: + Sergey Chvalyuk
messages: + msg268116
2016年03月20日 10:07:38Elena.Oatsetnosy: + Elena.Oat
messages: + msg262064
2016年03月13日 07:22:35rhettingersetpriority: normal -> low

nosy: + rhettinger, docs@python
messages: + msg261681

assignee: docs@python
components: + Documentation, - Library (Lib)
2016年03月12日 01:07:20Thomas.Waldmannsetmessages: + msg261627
2016年03月12日 00:59:00Thomas.Waldmanncreate

AltStyle によって変換されたページ (->オリジナル) /