homepage

Recently, after a small seemingly unrelated refactoring, the SymPy test suite in Python 3 started dying with "Fatal Python error: Cannot recover from stack overflow." 
Here's how to reproduce the error
git clone git://github.com/sympy/sympy.git # Clone the development version of SymPy
cd sympy
git checkout 0856119bd7399a416c21e1692855a1077164f21c # This is the first commit to exhibit the problem. Do this in case we make an unrelated change that removes the problem.
./bin/use2to3 # Convert the codebase to Python 3
python3 py3k-sympy/setup.py test # Run the tests
The issue is described in more detail at http://groups.google.com/group/sympy/browse_thread/thread/f664fe88e6b4f29d/3a44691c945695db#3a44691c945695db. Some key points:
- The test that triggers the error is an XFAIL test (test that is expected to fail) that raises RuntimeError: maximum recursion depth exceeded.
- The change that caused the error, 0856119bd7399a416c21e1692855a1077164f21c (see https://github.com/sympy/sympy/commit/0856119bd7399a416c21e1692855a1077164f21c), is seemingly unrelated. The only thing that I can think of is that it has added another call to the stack.
- This kills Python with Abort Trap: 6 in Mac OS X, which generates a problem report to be sent to Apple. I have included a copy of it here: https://gist.github.com/2317869.
- Others have reproduced this error as well, as can be seen by our test reporter tool. See the mailing list thread for more info.
- I tested this in 3.2.3r2, and the error still occurred. I tried testing in the 3.3 alpha, but I could not get it to compile.

It looks like an issue in SymPy, a stack overflow. Why did you report this issue on CPython bug tracker?
Is there an infinite loop in a recursive function call?
Can you try to get the full Python traceback using faulthandler? Use "-X faulthandler" command line option, or use "import faulthandler; faulthandler.enable()" at the beginning of your program. You have to install it manually for Python < 3.3.

We do have a stack overflow, but this should be raising a RuntimeError, not killing Python. The way it is now, Python dies completely with abort trap 6 (hence the Mac OS X problem report). Sorry if I didn't make this clear in the OP.

SymPy uses a C extension, numpy. The problem may be related to numpy. CPython tries to catch segmentation fault, but when a C extension is used, CPython may fail to protect your program against stack overflow.

No it does not. SymPy is a pure Python library.

The (first) Python stack overflow occurs at:
-----------
(gdb) py-bt
Traceback (most recent call first):
 File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/expr.py", line 2531, in expand
 from sympy.simplify.simplify import fraction
 File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/cache.py", line 91, in wrapper
 func_cache_it_cache[k] = r = func(*args, **kw_args)
 File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 514, in as_real_imag
 return (self.expand(deep, **hints), S.Zero)
 File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 525, in _eval_expand_complex
 re_part, im_part = self.as_real_imag(deep=deep, **hints)
 File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/expr.py", line 2551, in expand
 expr = func(deep=deep, **hints)
 File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/cache.py", line 91, in wrapper
 func_cache_it_cache[k] = r = func(*args, **kw_args)
 File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 514, in as_real_imag
 return (self.expand(deep, **hints), S.Zero)
 File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 525, in _eval_expand_complex
(gdb) where
#0 _Py_CheckRecursiveCall (where=0x624f4b " in comparison") at Python/ceval.c:736
#1 0x0000000000419e6a in PyObject_RichCompare (v='sympy', w='sympy', op=2) at Objects/object.c:604
#2 0x0000000000419f2d in PyObject_RichCompareBool (v='sympy', w='sympy', op=2) at Objects/object.c:628
#3 0x00000000005e854e in lookdict (mp=0x7ffff1b3b1a8, key='sympy', hash=-191038920480525830) at Objects/dictobject.c:341
#4 0x00000000005e99ca in PyDict_GetItem (op=
 {'sympy.logic.inference': <module at remote 0x7fffeef80ba0>, ..., key='sympy') at Objects/dictobject.c:752
#5 0x00000000004f1510 in import_submodule (mod=None, subname='sympy', fullname='sympy') at Python/import.c:3313
#6 0x00000000004f064b in load_next (mod=None, altmod=None, inputname='sympy.simplify.simplify', p_outputname=0x7fffffe67e28, p_prefix=0x7fffffe67e20)
 at Python/import.c:3150
#7 0x00000000004ef610 in import_module_level (name='sympy.simplify.simplify', globals=
 {...}, locals=None, fromlist=('fraction',), level=0)
 at Python/import.c:2843
-----------
Ok, now I see the problem: import_submodule() uses PyDict_GetItem() whereas PyDict_GetItem() *clears* the exception (!). It should use PyDict_GetItemWithError() instead. Try attached patch. The problem is a generic problem: PyDict_GetItem() should be replaced by PyDict_GetItemWithError() everywhere... But it would be nice to start with import.c :-)

I didn't tested this patch with SymPy, but in general it LGTM.

This code fragment doesn't seem to exist any more. Since there's no test case, I'm going to close this. I presume it is no longer a problem for SymPy, since there's been no followup.

The OP describes how to get the original code. Anyway, the issue was definitely fixed.

Or do you mean the code in CPython doesn't exist any more?

Yes, it appears that the code does not exist in import.c any more.
When I said "test case", I meant "unit test", sorry.