homepage

When format a traceback of MemoryError, linecache tries to read all lines of source file and likely raises a MemoryError. For example:
http://buildbot.python.org/all/builders/x86%20OpenIndiana%203.x/builds/9712/steps/test/logs/stdio
Traceback (most recent call last):
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 1280, in runtest_inner
 test_runner()
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/test_socket.py", line 5340, in test_main
 support.run_unittest(*tests)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/support/__init__.py", line 1808, in run_unittest
 _run_suite(suite)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/support/__init__.py", line 1774, in _run_suite
 result = runner.run(suite)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/runner.py", line 176, in run
 test(result)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/suite.py", line 84, in __call__
 return self.run(*args, **kwds)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/suite.py", line 122, in run
 test(result)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/suite.py", line 84, in __call__
 return self.run(*args, **kwds)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/suite.py", line 122, in run
 test(result)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/case.py", line 625, in __call__
 return self.run(*args, **kwds)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/case.py", line 585, in run
 self._feedErrorsToResult(result, outcome.errors)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/case.py", line 518, in _feedErrorsToResult
 result.addError(test, exc_info)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/runner.py", line 67, in addError
 super(TextTestResult, self).addError(test, err)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/result.py", line 17, in inner
 return method(self, *args, **kw)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/result.py", line 115, in addError
 self.errors.append((test, self._exc_info_to_string(err, test)))
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/unittest/result.py", line 186, in _exc_info_to_string
 exctype, value, tb, limit=length, capture_locals=self.tb_locals)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 456, in __init__
 capture_locals=capture_locals)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 341, in extract
 f.line
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 270, in line
 self._line = linecache.getline(self.filename, self.lineno).strip()
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 16, in getline
 lines = getlines(filename, module_globals)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 44, in getlines
 return updatecache(filename, module_globals)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 134, in updatecache
 lines = fp.readlines()
MemoryError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 532, in main
 result = runtest(*args, **kwargs)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 967, in runtest
 display_failure=False)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 1304, in runtest_inner
 msg = traceback.format_exc()
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 161, in format_exc
 return "".join(format_exception(*sys.exc_info(), limit=limit, chain=chain))
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 115, in format_exception
 type(value), value, tb, limit=limit).format(chain=chain))
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 456, in __init__
 capture_locals=capture_locals)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 341, in extract
 f.line
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/traceback.py", line 270, in line
 self._line = linecache.getline(self.filename, self.lineno).strip()
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 16, in getline
 lines = getlines(filename, module_globals)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 44, in getlines
 return updatecache(filename, module_globals)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/linecache.py", line 134, in updatecache
 lines = fp.readlines()
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/codecs.py", line 321, in decode
 (result, consumed) = self._buffer_decode(data, self.errors, final)
MemoryError
Traceback (most recent call last):
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/runpy.py", line 170, in _run_module_as_main
 "__main__", mod_spec)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/runpy.py", line 85, in _run_code
 exec(code, run_globals)
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/__main__.py", line 3, in <module>
 regrtest.main_in_temp_cwd()
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 1564, in main_in_temp_cwd
 main()
 File "/export/home/buildbot/32bits/3.x.cea-indiana-x86/build/Lib/test/regrtest.py", line 738, in main
 raise Exception("Child error on {}: {}".format(test, result[1]))
Exception: Child error on test_socket: 
make: *** [buildbottest] Error 1
program finished with exit code 2
Proposed patch clears the cache and repeats an attempt to read source file, and returns empty result if the second attempt fails (as when the source file can't be read).
I hesitate how to classify this issue. On one hand, it looks as new feature, on other hand it is related to testing on all versions.

> Proposed patch clears the cache and repeats an attempt to read source file, and returns empty result if the second attempt fails (as when the source file can't be read).
IMO it's a bad idea. You should not retry an operation on MemoryError but only do you best to limit errors.
I don't think that it's correct to return an empty list for getlines() on MemoryError. So you maybe chose the wrong module: you should handle MemoryError in the traceback module, not in the linecache module.
It's just fine to return an empty string for FrameSummary.line. It's common to not being able to display a line: because the .py was removed (only the .pyc is available), invalid line number, invalid filename ("<string>" or "<stdin>"), etc.
If you are motivated to make the traceback module more reliable, you should maybe also enhance _Py_DisplaySourceLine() in Python/traceback.c. You can probably ignore almost all errors in _Py_DisplaySourceLine() ("except (Exception, MemoryError): pass").

Serhiy: you look to be interested in fixing Python to not fail badly under low memory condition. You may take a look at #19817 which proposes a function to arbitrary limit the memory allocated to Python. I fixed a lot of issues found by failmalloc, but they are still serious issues. For example, unit tests crash on subinterpreter tests with failmalloc, and there is also the memory allocation failure when raising MemoryError: see #19835.

getlines() already returns an empty list on OSError. Clearing the cache on MemoryError will help to avoid other MemoryErrors. And after this there is a large chance the repeated reading will be successful. In any case it wouldn't make worse.
If don't change linecache, then we should patch all places where linecache is used: traceback, idlelib, warnings, trace, debugger, etc. I intentionally made the fix at the low level, so it fixes all issues in one place.

2015年04月01日 14:02 GMT+02:00 Serhiy Storchaka <report@bugs.python.org>:
> getlines() already returns an empty list on OSError.
Oh, I misread getlines(), you're right.
> Clearing the cache on MemoryError will help to avoid other MemoryErrors. And after this there is a large chance
I agree, but please avoid retrying on MemoryError. IMO clearing the
cache and return [] is enough.
> If don't change linecache, then we should patch all places where linecache is used: traceback, idlelib, warnings, trace, debugger, etc. I intentionally made the fix at the low level, so it fixes all issues in one place.
I agree.

OK, here is a patch that just clear the cache and returns an empty result.
It is hard to write reliable test, but I think we will see the result on OpenIndiana buildbot.
The question is left: should the patch be applied only on 3.5 or all versions?

> It is hard to write reliable test
Ah? Why not using something like mock.patch('mock.updatecache', side_effect=MemoryError)?

> The question is left: should the patch be applied only on 3.5 or all versions?
linecache_memoryerror_2.patch looks good to me, but it would be better with an unit test.
IMO Python 2.7 and 3.4 can be fixed too.

This test depends on implementation details. I prefer to avoid adding such tests for trivial changes.

On other side, linecache tests already use updatecache(). Here is a patch with a test.

linecache_memoryerror_3.patch reviewed (comments on Rietveld).

New changeset d444496e714a by Serhiy Storchaka in branch '2.7':
Issue #23838: linecache now clears the cache and returns an empty result on
https://hg.python.org/cpython/rev/d444496e714a
New changeset 88a0e6cd93c3 by Serhiy Storchaka in branch '3.4':
Issue #23838: linecache now clears the cache and returns an empty result on
https://hg.python.org/cpython/rev/88a0e6cd93c3
New changeset ab2326c801c2 by Serhiy Storchaka in branch 'default':
Issue #23838: linecache now clears the cache and returns an empty result on
https://hg.python.org/cpython/rev/ab2326c801c2 

Thank you for your review Victor. Will look on OpenIndiana buildbot.