homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Bug in _PyDict_Pop() on a splitted table
Type: crash Stage:
Components: Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, methane, python-dev, skrah, vstinner, xiang.zhang
Priority: normal Keywords: patch

Created on 2016年09月13日 07:45 by vstinner, last changed 2022年04月11日 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
issue28120.patch xiang.zhang, 2016年09月13日 13:42 review
issue28120_v2.patch xiang.zhang, 2016年09月13日 14:11 review
issue28120_v3.patch xiang.zhang, 2016年09月13日 14:27 review
Pull Requests
URL Status Linked Edit
PR 552 closed dstufft, 2017年03月31日 16:36
Messages (21)
msg276204 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016年09月13日 07:45
In the issue #4a5b61b0d090, Stefan Krah wrote: "The Blaze test suite segfaults with 4a5b61b0d090". I may be related to compact dict and the issue #28040.
Stefan: Can you please explain what is Blaze? Explain how to reproduce the issue? Try to get the Python traceback?
Did you try a debug build? If you use a release build, please set the env var PYTHONMALLOC=debug.
"""
Blaze is pushing Python's dynamic capabilities to absolute limits,
so perhaps this is specific to a few applications only.
For Blaze *itself* there is no difference whether this is fixed now
or in the next beta. So releasing 3.6.0b1 now and setting this back
to blocker afterwards sounds good to me.
"""
and
"""
It could still be a stack overflow, but on the surface it does
not look like one. It's definitely related to the aforementioned
revision:
==3442== Invalid read of size 8
==3442== at 0x49DBD8: _PyDict_Pop (dictobject.c:1743)
==3442== by 0x4A0BE2: dict_pop (dictobject.c:2732)
==3442== by 0x4AA5F8: _PyCFunction_FastCallDict (methodobject.c:229)
==3442== by 0x4AA70B: _PyCFunction_FastCallKeywords (methodobject.c:267)
==3442== by 0x55FE63: call_function (ceval.c:4794)
==3442== by 0x55AA82: _PyEval_EvalFrameDefault (ceval.c:3267)
==3442== by 0x54D9CC: PyEval_EvalFrameEx (ceval.c:718)
==3442== by 0x560123: _PyFunction_FastCall (ceval.c:4876)
==3442== by 0x56023B: fast_function (ceval.c:4906)
==3442== by 0x55FF91: call_function (ceval.c:4815)
==3442== by 0x55AA82: _PyEval_EvalFrameDefault (ceval.c:3267)
==3442== by 0x54D9CC: PyEval_EvalFrameEx (ceval.c:718)
==3442== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==3442== 
==3442== 
==3442== Process terminating with default action of signal 11 (SIGSEGV)
==3442== Access not within mapped region at address 0x0
==3442== at 0x49DBD8: _PyDict_Pop (dictobject.c:1743)
==3442== by 0x4A0BE2: dict_pop (dictobject.c:2732)
==3442== by 0x4AA5F8: _PyCFunction_FastCallDict (methodobject.c:229)
==3442== by 0x4AA70B: _PyCFunction_FastCallKeywords (methodobject.c:267)
==3442== by 0x55FE63: call_function (ceval.c:4794)
==3442== by 0x55AA82: _PyEval_EvalFrameDefault (ceval.c:3267)
==3442== by 0x54D9CC: PyEval_EvalFrameEx (ceval.c:718)
==3442== by 0x560123: _PyFunction_FastCall (ceval.c:4876)
==3442== by 0x56023B: fast_function (ceval.c:4906)
==3442== by 0x55FF91: call_function (ceval.c:4815)
==3442== by 0x55AA82: _PyEval_EvalFrameDefault (ceval.c:3267)
==3442== by 0x54D9CC: PyEval_EvalFrameEx (ceval.c:718)
"""
msg276224 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2016年09月13日 09:36
Sorry Victor, I have been reporting memory related issues for a long
time here. ;)
I'm already using Valgrind, why would I set PYTHONMALLOC=debug?
msg276226 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016年09月13日 09:46
> I'm already using Valgrind, why would I set PYTHONMALLOC=debug?
Valgrind doesn't understand Python frames. PYTHONMALLOC=debug can show you more information from Python internals, especially when using the tracemalloc module:
https://docs.python.org/dev/whatsnew/3.6.html#pythonmalloc-environment-variable
At this point, I'm not convinced that the issue comes from Python itself. It would help if at least you can explain how to reproduce the issue. What is Blaze? Where can I find it?
msg276229 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2016年09月13日 09:49
Victor, I'm beginning to resent your tone.
msg276240 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016年09月13日 10:42
See also the issue #28123: "_PyDict_GetItem_KnownHash ignores DKIX_ERROR return".
msg276248 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016年09月13日 11:12
The blaze test suite fails in pytest.
$ sudo dnf builddep python-numpy python-h5py
$ virtualenv -p ../bin/python3.6 venvblaze
$ cd venvblaze
$ bin/pip install install blaze pyyaml requests dask pytest h5py
bin/pytest -v lib/python3.6/site-packages/blaze/tests/test_sql.py 
=================================================================== test session starts ====================================================================
platform linux -- Python 3.6.0b1+, pytest-3.0.2, py-1.4.31, pluggy-0.3.1 -- /home/heimes/dev/python/python3.6/blaze/bin/python3.6
cachedir: .cache
rootdir: /home/heimes/dev/python/python3.6/blaze, inifile: 
collected 16 items 
lib/python3.6/site-packages/blaze/tests/test_sql.py::test_column Aborted (core dumped)
#0 0x00007ffff711f6f5 in raise () from /lib64/libc.so.6
#1 0x00007ffff71212fa in abort () from /lib64/libc.so.6
#2 0x00007ffff7117f97 in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007ffff7118042 in __assert_fail () from /lib64/libc.so.6
#4 0x000000000048f12c in _PyDict_Pop (mp=mp@entry=0x7fffbc6bba30, key=0x7fffe07ab270, deflt=0x885ec0 <_Py_NoneStruct>) at Objects/dictobject.c:1743
#5 0x000000000048f274 in dict_pop (mp=0x7fffbc6bba30, args=<optimized out>) at Objects/dictobject.c:2736
#6 0x000000000049767d in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7fffbc616850, args=args@entry=0x1c73600, nargs=nargs@entry=2, 
 kwargs=kwargs@entry=0x0) at Objects/methodobject.c:234
#7 0x00000000004978d4 in _PyCFunction_FastCallKeywords (func=func@entry=0x7fffbc616850, stack=stack@entry=0x1c73600, nargs=nargs@entry=2, 
 kwnames=kwnames@entry=0x0) at Objects/methodobject.c:295
#8 0x0000000000528c5e in call_function (pp_stack=pp_stack@entry=0x7fffffff1678, oparg=oparg@entry=2, kwnames=kwnames@entry=0x0) at Python/ceval.c:4763
#9 0x00000000005321d4 in _PyEval_EvalFrameDefault (f=0x1c73448, throwflag=<optimized out>) at Python/ceval.c:3250
#10 0x0000000000527d6b in PyEval_EvalFrameEx (f=f@entry=0x1c73448, throwflag=throwflag@entry=0) at Python/ceval.c:718
#11 0x0000000000527e3b in _PyFunction_FastCall (co=<optimized out>, args=0x7fffbc6245e0, nargs=nargs@entry=2, globals=<optimized out>)
 at Python/ceval.c:4845
#12 0x0000000000528a8b in fast_function (func=func@entry=0x7fffe06e69b0, stack=<optimized out>, nargs=nargs@entry=2, kwnames=kwnames@entry=0x0)
 at Python/ceval.c:4880
#13 0x0000000000528d19 in call_function (pp_stack=pp_stack@entry=0x7fffffff1828, oparg=oparg@entry=1, kwnames=kwnames@entry=0x0) at Python/ceval.c:4784
#14 0x00000000005321d4 in _PyEval_EvalFrameDefault (f=0x7fffbc624438, throwflag=<optimized out>) at Python/ceval.c:3250
#15 0x0000000000527d6b in PyEval_EvalFrameEx (f=f@entry=0x7fffbc624438, throwflag=throwflag@entry=0) at Python/ceval.c:718
#16 0x0000000000527e3b in _PyFunction_FastCall (co=co@entry=0x7fffe061df40, args=0x7fffffff19d0, args@entry=0x7fffffff19c0, nargs=nargs@entry=2, 
 globals=globals@entry=0x7fffe06a4f58) at Python/ceval.c:4845
#17 0x0000000000533f46 in _PyFunction_FastCallDict (func=func@entry=0x7fffe05f7f70, args=args@entry=0x7fffffff19c0, nargs=nargs@entry=2, 
 kwargs=kwargs@entry=0x0) at Python/ceval.c:4947
#18 0x0000000000446837 in _PyObject_FastCallDict (func=func@entry=0x7fffe05f7f70, args=args@entry=0x7fffffff19c0, nargs=2, kwargs=kwargs@entry=0x0)
 at Objects/abstract.c:2297
#19 0x00000000004479da in PyObject_CallFunctionObjArgs (callable=0x7fffe05f7f70) at Objects/abstract.c:2811
#20 0x00000000005b2888 in property_descr_set (self=<optimized out>, obj=<optimized out>, value=<optimized out>) at Objects/descrobject.c:1419
#21 0x000000000049bf42 in _PyObject_GenericSetAttrWithDict (obj=0x7fffbc695058, name=0x7ffff02bcba0, value=0x885ec0 <_Py_NoneStruct>, dict=dict@entry=0x0)
 at Objects/object.c:1152
#22 0x000000000049c174 in PyObject_GenericSetAttr (obj=<optimized out>, name=<optimized out>, value=<optimized out>) at Objects/object.c:1194
#23 0x000000000049b823 in PyObject_SetAttr (v=v@entry=0x7fffbc695058, name=<optimized out>, name@entry=0x7ffff02bcba0, 
 value=value@entry=0x885ec0 <_Py_NoneStruct>) at Objects/object.c:932
#24 0x000000000052e4eb in _PyEval_EvalFrameDefault (f=0x1c73218, throwflag=<optimized out>) at Python/ceval.c:2248
#25 0x0000000000527d6b in PyEval_EvalFrameEx (f=f@entry=0x1c73218, throwflag=throwflag@entry=0) at Python/ceval.c:718
#26 0x000000000052884a in _PyEval_EvalCodeWithName (_co=0x7fffe061dd00, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, 
 argcount=3, kwnames=0x0, kwargs=0x1c8a460, kwcount=0, kwstep=1, defs=0x7fffe06213c8, defcount=3, kwdefs=0x0, closure=0x0, name=0x7ffff7f7e430, 
 qualname=0x7fffe061ab08) at Python/ceval.c:4094
#27 0x00000000005289ba in fast_function (func=func@entry=0x7fffe05f7d48, stack=<optimized out>, nargs=nargs@entry=3, kwnames=kwnames@entry=0x0)
 at Python/ceval.c:4904
#28 0x0000000000528d19 in call_function (pp_stack=pp_stack@entry=0x7fffffff1e38, oparg=oparg@entry=2, kwnames=kwnames@entry=0x0) at Python/ceval.c:4784
---Type <return> to continue, or q <return> to quit---
#29 0x00000000005321d4 in _PyEval_EvalFrameDefault (f=0x1c8a288, throwflag=<optimized out>) at Python/ceval.c:3250
#30 0x0000000000527d6b in PyEval_EvalFrameEx (f=f@entry=0x1c8a288, throwflag=throwflag@entry=0) at Python/ceval.c:718
#31 0x000000000052884a in _PyEval_EvalCodeWithName (_co=_co@entry=0x7fffe06a2ac0, globals=globals@entry=0x7fffe07012b0, locals=locals@entry=0x0, 
 args=args@entry=0x7fffffff2080, argcount=argcount@entry=3, kwnames=kwnames@entry=0x7fffbc670bc0, kwargs=0x7fffbc670bc8, kwcount=8, kwstep=2, defs=0x0, 
 defcount=0, kwdefs=0x0, closure=0x7fffe037a6d8, name=0x7ffff7f7e430, qualname=0x7fffe0351120) at Python/ceval.c:4094
#32 0x0000000000534020 in _PyFunction_FastCallDict (func=func@entry=0x7fffe02978f8, args=args@entry=0x7fffffff2080, nargs=nargs@entry=3, 
 kwargs=kwargs@entry=0x7fffbc6bb8c8) at Python/ceval.c:4996
#33 0x0000000000446837 in _PyObject_FastCallDict (func=func@entry=0x7fffe02978f8, args=args@entry=0x7fffffff2080, nargs=nargs@entry=3, 
 kwargs=kwargs@entry=0x7fffbc6bb8c8) at Objects/abstract.c:2297
#34 0x0000000000446acb in _PyObject_Call_Prepend (func=0x7fffe02978f8, obj=obj@entry=0x7fffbc695058, args=0x7fffbc0f22f8, kwargs=0x7fffbc6bb8c8)
 at Objects/abstract.c:2360
#35 0x0000000000460410 in method_call (method=<optimized out>, args=<optimized out>, kwargs=<optimized out>) at Objects/classobject.c:317
#36 0x000000000044652e in PyObject_Call (func=func@entry=0x7fffbdf9ef38, args=args@entry=0x7fffbc0f22f8, kwargs=kwargs@entry=0x7fffbc6bb8c8)
 at Objects/abstract.c:2248
#37 0x00000000004b4dcb in slot_tp_init (self=<optimized out>, args=0x7fffbc0f22f8, kwds=0x7fffbc6bb8c8) at Objects/typeobject.c:6359
#38 0x00000000004b128b in type_call (type=0x16f0168, args=0x7fffbc0f22f8, kwds=0x7fffbc6bb8c8) at Objects/typeobject.c:915
#39 0x000000000044652e in PyObject_Call (func=0x16f0168, args=0x7fffbc0f22f8, kwargs=0x7fffbc6bb8c8) at Objects/abstract.c:2248
#40 0x0000000000527237 in do_call_core (func=func@entry=0x16f0168, callargs=callargs@entry=0x7fffbc0f22f8, kwdict=kwdict@entry=0x7fffbc6bb8c8)
 at Python/ceval.c:5032
#41 0x0000000000532700 in _PyEval_EvalFrameDefault (f=0x1c2a6c8, throwflag=<optimized out>) at Python/ceval.c:3332
#42 0x0000000000527d6b in PyEval_EvalFrameEx (f=f@entry=0x1c2a6c8, throwflag=throwflag@entry=0) at Python/ceval.c:718
#43 0x000000000052884a in _PyEval_EvalCodeWithName (_co=0x7fffe0391700, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, 
 argcount=2, kwnames=0x7fffe0336ad0, kwargs=0x1cb3998, kwcount=3, kwstep=1, defs=0x7fffe06a4800, defcount=3, kwdefs=0x0, closure=0x0, 
 name=0x7fffe06707b0, qualname=0x7fffe03504f0) at Python/ceval.c:4094
#44 0x00000000005289ba in fast_function (func=func@entry=0x7fffe0297f70, stack=<optimized out>, nargs=nargs@entry=2, kwnames=kwnames@entry=0x7fffe0336aa8)
 at Python/ceval.c:4904
#45 0x0000000000528d19 in call_function (pp_stack=pp_stack@entry=0x7fffffff24f0, oparg=<optimized out>, kwnames=kwnames@entry=0x7fffe0336aa8)
 at Python/ceval.c:4784
#46 0x00000000005322ed in _PyEval_EvalFrameDefault (f=0x1cb37d8, throwflag=<optimized out>) at Python/ceval.c:3266
#47 0x0000000000527d6b in PyEval_EvalFrameEx (f=f@entry=0x1cb37d8, throwflag=throwflag@entry=0) at Python/ceval.c:718
#48 0x0000000000527e3b in _PyFunction_FastCall (co=<optimized out>, args=0x7fffbc644970, nargs=nargs@entry=1, globals=<optimized out>)
 at Python/ceval.c:4845
#49 0x0000000000528a8b in fast_function (func=func@entry=0x7fffe02ef3f0, stack=<optimized out>, nargs=nargs@entry=1, kwnames=kwnames@entry=0x0)
 at Python/ceval.c:4880
#50 0x0000000000528d19 in call_function (pp_stack=pp_stack@entry=0x7fffffff2698, oparg=oparg@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:4784
#51 0x00000000005321d4 in _PyEval_EvalFrameDefault (f=0x7fffbc6447d8, throwflag=<optimized out>) at Python/ceval.c:3250
#52 0x0000000000527d6b in PyEval_EvalFrameEx (f=f@entry=0x7fffbc6447d8, throwflag=throwflag@entry=0) at Python/ceval.c:718
#53 0x0000000000527e3b in _PyFunction_FastCall (co=<optimized out>, args=0x1cb7eb0, nargs=nargs@entry=1, globals=<optimized out>) at Python/ceval.c:4845
#54 0x0000000000528a8b in fast_function (func=func@entry=0x7fffe02d0788, stack=<optimized out>, nargs=nargs@entry=1, kwnames=kwnames@entry=0x0)
 at Python/ceval.c:4880
#55 0x0000000000528d19 in call_function (pp_stack=pp_stack@entry=0x7fffffff2848, oparg=oparg@entry=1, kwnames=kwnames@entry=0x0) at Python/ceval.c:4784
---Type <return> to continue, or q <return> to quit---
#56 0x00000000005321d4 in _PyEval_EvalFrameDefault (f=0x1cb7cf8, throwflag=<optimized out>) at Python/ceval.c:3250
#57 0x0000000000527d6b in PyEval_EvalFrameEx (f=f@entry=0x1cb7cf8, throwflag=throwflag@entry=0) at Python/ceval.c:718
#58 0x0000000000527e3b in _PyFunction_FastCall (co=co@entry=0x7fffe0738d00, args=0x7fffffff29f8, args@entry=0x7fffffff29e0, nargs=nargs@entry=3, 
 globals=globals@entry=0x7fffe0a7fc88) at Python/ceval.c:4845
#59 0x0000000000533f46 in _PyFunction_FastCallDict (func=func@entry=0x7fffe06e6618, args=args@entry=0x7fffffff29e0, nargs=nargs@entry=3, 
 kwargs=kwargs@entry=0x0) at Python/ceval.c:4947
#60 0x0000000000446837 in _PyObject_FastCallDict (func=func@entry=0x7fffe06e6618, args=args@entry=0x7fffffff29e0, nargs=3, kwargs=kwargs@entry=0x0)
 at Objects/abstract.c:2297
#61 0x00000000004479da in PyObject_CallFunctionObjArgs (callable=0x7fffe06e6618) at Objects/abstract.c:2811
#62 0x00000000004b6930 in slot_tp_descr_get (self=self@entry=0x7fffe0340810, obj=obj@entry=0x7fffbc692f60, type=0x16e2238) at Objects/typeobject.c:6330
#63 0x000000000049bde4 in _PyObject_GenericGetAttrWithDict (obj=0x7fffbc692f60, name=0x7ffff7e82450, dict=<optimized out>, dict@entry=0x0)
 at Objects/object.c:1099
...
msg276249 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016年09月13日 11:15
$ PYTHONFAULTHANDLER=1 bin/pytest -v lib/python3.6/site-packages/blaze/tests/test_sql.py 
=================================================================== test session starts ====================================================================
platform linux -- Python 3.6.0b1+, pytest-3.0.2, py-1.4.31, pluggy-0.3.1 -- /home/heimes/dev/python/python3.6/blaze/bin/python3.6
cachedir: .cache
rootdir: /home/heimes/dev/python/python3.6/blaze, inifile: 
collected 16 items 
lib/python3.6/site-packages/blaze/tests/test_sql.py::test_column Aborted (core dumped)
[heimes@hamiller blaze]$ PYTHONFAULTHANDLER=1 bin/pytest -s -v lib/python3.6/site-packages/blaze/tests/test_sql.py 
=================================================================== test session starts ====================================================================
platform linux -- Python 3.6.0b1+, pytest-3.0.2, py-1.4.31, pluggy-0.3.1 -- /home/heimes/dev/python/python3.6/blaze/bin/python3.6
cachedir: .cache
rootdir: /home/heimes/dev/python/python3.6/blaze, inifile: 
collecting 0 items/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/flask/exthook.py:71: ExtDeprecationWarning: Importing flask.ext.cors is deprecated, use flask_cors instead.
 .format(x=modname), ExtDeprecationWarning
collected 16 items 
lib/python3.6/site-packages/blaze/tests/test_sql.py::test_column python3.6: Objects/dictobject.c:1743: _PyDict_Pop: Assertion `ix >= 0' failed.
Fatal Python error: Aborted
Thread 0x00007f9671ffb700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 295 in wait
 File "/home/heimes/dev/python/python3.6/lib/python3.6/queue.py", line 164 in get
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 429 in _handle_results
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f96727fc700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 295 in wait
 File "/home/heimes/dev/python/python3.6/lib/python3.6/queue.py", line 164 in get
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 376 in _handle_tasks
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f9672ffd700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 367 in _handle_workers
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f96737fe700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 295 in wait
 File "/home/heimes/dev/python/python3.6/lib/python3.6/queue.py", line 164 in get
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 108 in worker
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f9673fff700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 295 in wait
 File "/home/heimes/dev/python/python3.6/lib/python3.6/queue.py", line 164 in get
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 108 in worker
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f9688fb9700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 295 in wait
 File "/home/heimes/dev/python/python3.6/lib/python3.6/queue.py", line 164 in get
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 108 in worker
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f96897ba700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 295 in wait
 File "/home/heimes/dev/python/python3.6/lib/python3.6/queue.py", line 164 in get
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 108 in worker
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f9689fbb700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 295 in wait
 File "/home/heimes/dev/python/python3.6/lib/python3.6/queue.py", line 164 in get
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 108 in worker
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f968a7bc700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 295 in wait
 File "/home/heimes/dev/python/python3.6/lib/python3.6/queue.py", line 164 in get
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 108 in worker
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f968afbd700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 295 in wait
 File "/home/heimes/dev/python/python3.6/lib/python3.6/queue.py", line 164 in get
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 108 in worker
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f968b7be700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 295 in wait
 File "/home/heimes/dev/python/python3.6/lib/python3.6/queue.py", line 164 in get
 File "/home/heimes/dev/python/python3.6/lib/python3.6/multiprocessing/pool.py", line 108 in worker
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 864 in run
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 916 in _bootstrap_inner
 File "/home/heimes/dev/python/python3.6/lib/python3.6/threading.py", line 884 in _bootstrap
Current thread 0x00007f96a3ebc700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 796 in expire_instance
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 3310 in _set_table
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 3288 in __init__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 1157 in __init__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 1414 in _make_proxy
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/sql/selectable.py", line 3212 in _populate_column_collection
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/sql/selectable.py", line 577 in columns
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 754 in __get__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/sql/selectable.py", line 1149 in _populate_column_collection
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/sql/selectable.py", line 586 in primary_key
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 754 in __get__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/blaze/compute/sql.py", line 731 in compute_up
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/multipledispatch/dispatcher.py", line 164 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/blaze/compute/core.py", line 325 in bottom_up_until_type_break
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/blaze/compute/core.py", line 158 in top_then_bottom_then_top_again_etc
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/blaze/compute/core.py", line 189 in top_then_bottom_then_top_again_etc
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/blaze/compute/core.py", line 409 in compute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/multipledispatch/dispatcher.py", line 164 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/blaze/interactive.py", line 195 in compute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/multipledispatch/dispatcher.py", line 164 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/blaze/interactive.py", line 474 in convert_base
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/blaze/interactive.py", line 500 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/blaze/interactive.py", line 428 in table_length
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/blaze/tests/test_sql.py", line 31 in test_column
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/python.py", line 149 in pytest_pyfunc_call
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 596 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 264 in __init__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 247 in _wrapped_call
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 595 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 333 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 338 in _hookexec
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 724 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/python.py", line 1552 in runtest
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 104 in pytest_runtest_call
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 596 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 264 in __init__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 247 in _wrapped_call
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 595 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 333 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 338 in _hookexec
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 724 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 151 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 163 in __init__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 151 in call_runtest_hook
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 133 in call_and_report
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 79 in runtestprotocol
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 66 in pytest_runtest_protocol
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 596 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 264 in __init__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 247 in _wrapped_call
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 595 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 333 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 338 in _hookexec
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 724 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/main.py", line 152 in pytest_runtestloop
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 596 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 333 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 338 in _hookexec
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 724 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/main.py", line 131 in _main
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/main.py", line 96 in wrap_session
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/main.py", line 125 in pytest_cmdline_main
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 596 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 333 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 338 in _hookexec
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 724 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/config.py", line 57 in main
 File "bin/pytest", line 11 in <module>
Aborted (core dumped)
msg276250 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016年09月13日 11:24
It's a bug in sqlalchemy. The test suite of sqlalchemy is crashing, too.
$ PYTHONFAULTHANDLER=1 ../bin/pytest -v -s
=================================================================== test session starts ====================================================================
platform linux -- Python 3.6.0b1+, pytest-3.0.2, py-1.4.31, pluggy-0.3.1 -- /home/heimes/dev/python/python3.6/blaze/bin/python3.6
cachedir: .cache
rootdir: /home/heimes/dev/python/python3.6/blaze/sqlalchemy, inifile: setup.cfg
collected 7883 items 
test/aaa_profiling/test_compiler.py::CompileTest_sqlite_pysqlite::test_insert SKIPPED
test/aaa_profiling/test_compiler.py::CompileTest_sqlite_pysqlite::test_select SKIPPED
test/aaa_profiling/test_compiler.py::CompileTest_sqlite_pysqlite::test_select_labels python3.6: Objects/dictobject.c:1743: _PyDict_Pop: Assertion `ix >= 0' failed.
Fatal Python error: Aborted
Current thread 0x00007f5606ac8700 (most recent call first):
 File "/home/heimes/dev/python/python3.6/blaze/sqlalchemy/test/../lib/sqlalchemy/util/langhelpers.py", line 796 in expire_instance
 File "/home/heimes/dev/python/python3.6/blaze/sqlalchemy/test/../lib/sqlalchemy/sql/selectable.py", line 666 in _reset_exported
 File "/home/heimes/dev/python/python3.6/blaze/sqlalchemy/test/../lib/sqlalchemy/sql/selectable.py", line 1918 in _generate
 File "/home/heimes/dev/python/python3.6/blaze/sqlalchemy/test/../lib/sqlalchemy/sql/base.py", line 44 in _generative
 File "<string>", line 2 in apply_labels
 File "/home/heimes/dev/python/python3.6/blaze/sqlalchemy/test/aaa_profiling/test_compiler.py", line 72 in test_select_labels
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/python.py", line 149 in pytest_pyfunc_call
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 596 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 264 in __init__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 247 in _wrapped_call
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 595 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 333 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 338 in _hookexec
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 724 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/python.py", line 1552 in runtest
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 104 in pytest_runtest_call
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 596 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 264 in __init__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 247 in _wrapped_call
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 595 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 333 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 338 in _hookexec
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 724 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 151 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 163 in __init__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 151 in call_runtest_hook
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 133 in call_and_report
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 79 in runtestprotocol
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/runner.py", line 66 in pytest_runtest_protocol
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 596 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 264 in __init__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 247 in _wrapped_call
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 595 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 333 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 338 in _hookexec
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 724 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/main.py", line 152 in pytest_runtestloop
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 596 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 333 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 338 in _hookexec
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 724 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/main.py", line 131 in _main
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/main.py", line 96 in wrap_session
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/main.py", line 125 in pytest_cmdline_main
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 596 in execute
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 333 in <lambda>
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 338 in _hookexec
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 724 in __call__
 File "/home/heimes/dev/python/python3.6/blaze/lib/python3.6/site-packages/_pytest/config.py", line 57 in main
 File "../bin/pytest", line 11 in <module>
Aborted (core dumped)
msg276251 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2016年09月13日 11:26
It fails in pytest and has a segfault. The pytest issue should probably
be separate issue (it could also be a blaze issue).
This is a minimal reproducer for the segfault:
=========================================================
from blaze.expr import symbol
from blaze.interactive import data
from blaze.compute import compute
t = symbol('t', 'var * {amount: int64, id: int64, name: string}')
sql = data('sqlite:///:memory:::accounts', dshape=t.dshape)
expr = t.distinct().nrows
x = expr._subs({t: sql})
result = compute(x)
=========================================================
Blaze itself is pure Python, a third part issue could be in sqlite
or sqlalchemy (but I think that is also mostly pure Python).
msg276253 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016年09月13日 11:36
The issue is not related to Blaze. It is caused by a SQLAlchemy method. expire_instance() calls pop(name, None) on instance __dict__.
msg276254 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2016年09月13日 11:46
Thanks, Christian!
msg276264 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016年09月13日 12:40
I added debug traces to _PyDict_Pop(). It looks like the assertion failed after dictresize() converts a splitted table into a combined table. After the resize, the key '_columns_plus_names' cannot be found anymore, whereas it was found before.
The string thing about the key '_columns_plus_names' is that I cannot see it in the output of repr(dict).
The length of the dictionary is 7, but it looks like it contains 8 items... Something is wrong, no?
Note: I'm not sure that my debug traces are correct, especially the number of entries may be larger. I don't understand yet exactly the internal structure of dictionaries.
@@ delete memoized
{'_auto_correlate': True, 
 '_from_obj': OrderedSet([]), '_raw_columns': [Table('t1', MetaData(bind=None), Column('c1', Integer(), table=<t1>, primary_key=True, nullable=False), Column('c2', String(length=30), table=<t1>), schema=None)],
 '_whereclause': <sqlalchemy.sql.elements.BinaryExpression object at 0x7fffe93d1190>,
 '_having': None,
 'use_labels': False,
 '_bind': None}
_PyDict_Pop(): ix=7, resize!
before resize:
<splitted dict 0x7fffe93cda30 used=7 keys->dk_size=16 keys->dk_nentries=8 keys->dk_usable=2 || usable=10>
index[0]=-1
index[1]=4
index[2]=-1
index[3]=-1
index[4]=0
index[5]=3
index[6]=-1
index[7]=7 <----- it looks like the entry #7 is referenced, no?
index[8]=1
index[9]=2
index[10]=-1
index[11]=-1
index[12]=-1
index[13]=-1
index[14]=5
index[15]=6
entry[0]: key='_auto_correlate', hash=-4930419206435490684
entry[1]: key='_from_obj', hash=-717737569259548376
entry[2]: key='_raw_columns', hash=6382956110151689156
entry[3]: key='_whereclause', hash=-7105285138948878507
entry[4]: key='_having', hash=-8328824464742262616
entry[5]: key='use_labels', hash=8687253172181039710
entry[6]: key='_bind', hash=-5556925267210855474
entry[7]: key='_columns_plus_names', hash=-7771762227888271625
entry[8]: key='NULL', hash=0
entry[9]: key='NULL', hash=0
object : {'_auto_correlate': True, '_from_obj': OrderedSet([]), '_raw_columns': [Table('t1', MetaData(bind=None), Column('c1', Integer(), table=<t1>, primary_key=True, nullable=False), Column('c2', String(length=30), table=<t1>), schema=None)], '_whereclause': <sqlalchemy.sql.elements.BinaryExpression object at 0x7fffe93d1190>, '_having': None, 'use_labels': False, '_bind': None}
type : dict
***
after resize:
<combined dict 0x7fffe93cda30 used=7 keys->dk_size=32 keys->dk_nentries=7 keys->dk_usable=14 || usable=21>
index[0]=-1
index[1]=-1
index[2]=-1
index[3]=-1
index[4]=0
index[5]=-1
index[6]=-1
index[7]=-1
index[8]=1
index[9]=-1
index[10]=-1
index[11]=-1
index[12]=-1
index[13]=-1
index[14]=6
index[15]=-1
index[16]=-1
index[17]=4
index[18]=-1
index[19]=-1
index[20]=-1
index[21]=3
index[22]=-1
index[23]=-1
index[24]=-1
index[25]=2
index[26]=-1
index[27]=-1
index[28]=-1
index[29]=-1
index[30]=5
index[31]=-1 <------ no more entry #7 ???
entry[0]: key='_auto_correlate', value=True, hash=-4930419206435490684
entry[1]: key='_from_obj', value=OrderedSet([]), hash=-717737569259548376
entry[2]: key='_raw_columns', value=[Table('t1', MetaData(bind=None), Column('c1', Integer(), table=<t1>, primary_key=True, nullable=False), Column('c2', String(length=30), table=<t1>), schema=None)], hash=6382956110151689156
entry[3]: key='_whereclause', value=<sqlalchemy.sql.elements.BinaryExpression object at 0x7fffe93d1190>, hash=-7105285138948878507
entry[4]: key='_having', value=None, hash=-8328824464742262616
entry[5]: key='use_labels', value=False, hash=8687253172181039710
entry[6]: key='_bind', value=None, hash=-5556925267210855474
entry[7]: key='NULL', value='NULL', hash=0
entry[8]: key='NULL', value='NULL', hash=0
entry[9]: key='NULL', value='NULL', hash=0
entry[10]: key='NULL', value='NULL', hash=0
entry[11]: key='NULL', value='NULL', hash=0
entry[12]: key='NULL', value='NULL', hash=0
entry[13]: key='NULL', value='NULL', hash=0
entry[14]: key='NULL', value='NULL', hash=0
entry[15]: key='NULL', value='NULL', hash=0
entry[16]: key='NULL', value='NULL', hash=0
entry[17]: key='NULL', value='NULL', hash=0
entry[18]: key='NULL', value='NULL', hash=0
entry[19]: key='NULL', value='NULL', hash=0
entry[20]: key='NULL', value='NULL', hash=0
object : {'_auto_correlate': True, '_from_obj': OrderedSet([]), '_raw_columns': [Table('t1', MetaData(bind=None), Column('c1', Integer(), table=<t1>, primary_key=True, nullable=False), Column('c2', String(length=30), table=<t1>), schema=None)], '_whereclause': <sqlalchemy.sql.elements.BinaryExpression object at 0x7fffe93d1190>, '_having': None, 'use_labels': False, '_bind': None}
type : dict
refcount: 3
python: Objects/dictobject.c:1810: _PyDict_Pop: Assertion `ix >= 0' failed.
Program received signal SIGABRT, Aborted.
0x00007ffff711f6f5 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-20.fc24.x86_64 keyutils-libs-1.5.9-8.fc24.x86_64 krb5-libs-1.14.3-8.fc24.x86_64 libcom_err-1.42.13-4.fc24.x86_64 libselinux-2.5-9.fc24.x86_64 openssl-libs-1.0.2h-3.fc24.x86_64 pcre-8.39-3.fc24.x86_64 sqlite-libs-3.13.0-1.fc24.x86_64 xz-libs-5.2.2-2.fc24.x86_64 zlib-1.2.8-10.fc24.x86_64
(gdb) up
#1 0x00007ffff71212fa in abort () from /lib64/libc.so.6
(gdb) 
#2 0x00007ffff7117f97 in __assert_fail_base () from /lib64/libc.so.6
(gdb) 
#3 0x00007ffff7118042 in __assert_fail () from /lib64/libc.so.6
(gdb) 
#4 0x00000000004bdfd7 in _PyDict_Pop (mp=0x7fffe93cda30, key='_columns_plus_names', deflt=None) at Objects/dictobject.c:1810
1810	 assert(ix >= 0);
(gdb) p key
1ドル = '_columns_plus_names'
(gdb) l
1805	 fprintf(stderr, "after resize:\n");
1806	 dict_dump_key_value_table(mp);
1807	 _PyObject_Dump((PyObject *)mp);
1808	
1809	 ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, &hashpos);
1810	 assert(ix >= 0);
1811	 }
1812	
1813	 old_value = *value_addr;
1814	 assert(old_value != NULL);
(gdb) p hash
2ドル = -7771762227888271625
(gdb) print value_addr
3ドル = (PyObject **) 0x0
(gdb) print value_addr
4ドル = (PyObject **) 0x0
(gdb) print hashpos
5ドル = 23
msg276266 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016年09月13日 13:34
To reproduce the issue:
* compile Python 3.6 in debug mode
* download https://pypi.python.org/pypi/SQLAlchemy/1.1.0b3
* Then type:
---
./python -m venv venvsqla
cd venvsqla
source bin/activate
python -m pip install pytest 
tar -xf ../SQLAlchemy-1.1.0b3.tar.gz 
cd SQLAlchemy-1.1.0b3/
python setup.py install 
python -X faulthandler -m pytest test/aaa_profiling/test_compiler.py -v -s
---
I get:
---
================================================ test session starts ================================================
platform linux -- Python 3.7.0a0, pytest-3.0.2, py-1.4.31, pluggy-0.3.1 -- /home/haypo/prog/python/default/venvsqla/bin/python
cachedir: .cache
rootdir: /home/haypo/prog/python/default/venvsqla/SQLAlchemy-1.1.0b3, inifile: setup.cfg
collecting 0 itemspython: Objects/dictobject.c:1739: _PyDict_Pop: Assertion `ix >= 0' failed.
Fatal Python error: Aborted
Current thread 0x00007f6d909da700 (most recent call first):
 File "/home/haypo/prog/python/default/venvsqla/SQLAlchemy-1.1.0b3/test/../lib/sqlalchemy/util/langhelpers.py", line 796 in expire_instance
 File "/home/haypo/prog/python/default/venvsqla/SQLAlchemy-1.1.0b3/test/../lib/sqlalchemy/sql/selectable.py", line 666 in _reset_exported
 File "/home/haypo/prog/python/default/venvsqla/SQLAlchemy-1.1.0b3/test/../lib/sqlalchemy/sql/selectable.py", line 1908 in _generate
 File "/home/haypo/prog/python/default/venvsqla/SQLAlchemy-1.1.0b3/test/../lib/sqlalchemy/sql/base.py", line 44 in _generative
 File "<string>", line 2 in apply_labels
 File "/home/haypo/prog/python/default/venvsqla/SQLAlchemy-1.1.0b3/test/aaa_profiling/test_compiler.py", line 12 in <module>
 File "/home/haypo/prog/python/default/venvsqla/lib/python3.7/site-packages/_pytest/assertion/rewrite.py", line 207 in load_module
 File "<frozen importlib._bootstrap>", line 616 in _load_backward_compatible
 File "<frozen importlib._bootstrap>", line 646 in _load_unlocked
 File "<frozen importlib._bootstrap>", line 950 in _find_and_load_unlocked
 File "<frozen importlib._bootstrap>", line 961 in _find_and_load
 File "/home/haypo/prog/python/default/venvsqla/lib/python3.7/site-packages/py/_path/local.py", line 650 in pyimport
 File "/home/haypo/prog/python/default/venvsqla/lib/python3.7/site-packages/_pytest/python.py", line 411 in _importtestmodule
...
---
msg276268 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016年09月13日 13:40
No need, you can reproduce it by this:
>>> class C:
... pass
... 
>>> a, b = C(), C()
>>> a.a, a.b = 1, 2
>>> b.a = 1
>>> b.__dict__.pop('b')
python: Objects/dictobject.c:1739: _PyDict_Pop: Assertion `ix >= 0' failed.
Aborted (core dumped)
msg276269 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016年09月13日 13:42
I submit a patch that can solve this. Actually I find it before but forget about it....
msg276270 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016年09月13日 13:44
Good work! This example is a bit easier to understand:
class Cls:
 pass
a = Cls()
a.x = 1
a.y = 2
b = Cls()
b.x = 1
print(hasattr(b, 'y'))
print(b.__dict__.pop('y', None))
$ ./python d.py 
False
python: Objects/dictobject.c:1743: _PyDict_Pop: Assertion `ix >= 0' failed.
Aborted (core dumped)
msg276276 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016年09月13日 14:11
v2 now enhances existing dict.pop() unittest to cover this case.
msg276277 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016年09月13日 14:13
+ # issue 28120
+ c['a'] = 4
Can you please elaborate the comment? Something like:
# issue 28120: Try to remove a pending key in a splitted table,
# it must not crash
msg276282 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016年09月13日 15:03
New changeset dc627ef9020f by Victor Stinner in branch '3.6':
Fix _PyDict_Pop() on pending key
https://hg.python.org/cpython/rev/dc627ef9020f 
msg276283 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016年09月13日 15:04
Thanks Stefan Krah for the bug report, thanks Christian Heimes for helping to identify the bug, and obvious thanks to Xiang Zhang to write a fix with an unit test ;-)
msg276289 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2016年09月13日 15:20
Oh, very thank you, Xiang.
History
Date User Action Args
2022年04月11日 14:58:36adminsetgithub: 72307
2017年03月31日 16:36:19dstufftsetpull_requests: + pull_request926
2016年09月13日 15:20:08methanesetmessages: + msg276289
2016年09月13日 15:04:21vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg276283

versions: + Python 3.7
2016年09月13日 15:03:07python-devsetnosy: + python-dev
messages: + msg276282
2016年09月13日 14:27:52xiang.zhangsetfiles: + issue28120_v3.patch
2016年09月13日 14:13:25vstinnersetmessages: + msg276277
2016年09月13日 14:11:08xiang.zhangsetfiles: + issue28120_v2.patch

messages: + msg276276
2016年09月13日 13:44:21christian.heimessetmessages: + msg276270
2016年09月13日 13:42:27xiang.zhangsetfiles: + issue28120.patch
keywords: + patch
messages: + msg276269
2016年09月13日 13:40:07xiang.zhangsetnosy: + xiang.zhang
messages: + msg276268
2016年09月13日 13:34:40vstinnersetmessages: + msg276266
2016年09月13日 13:15:21vstinnersettitle: SQLAlchemy's expire_instance() causes _PyDict_Pop: Assertion `ix >= 0' failed. -> Bug in _PyDict_Pop() on a splitted table
2016年09月13日 12:40:17vstinnersetmessages: + msg276264
2016年09月13日 11:46:20skrahsetmessages: + msg276254
2016年09月13日 11:36:10christian.heimessetmessages: + msg276253
2016年09月13日 11:34:40christian.heimessettitle: The Blaze test suite segfaults with 4a5b61b0d090 -> SQLAlchemy's expire_instance() causes _PyDict_Pop: Assertion `ix >= 0' failed.
2016年09月13日 11:26:02skrahsetnosy: + skrah
messages: + msg276251
2016年09月13日 11:24:25christian.heimessetmessages: + msg276250
2016年09月13日 11:15:25christian.heimessetmessages: + msg276249
2016年09月13日 11:12:51christian.heimessetnosy: + christian.heimes
messages: + msg276248
2016年09月13日 10:42:15vstinnersetmessages: + msg276240
2016年09月13日 10:06:11skrahsetnosy: - skrah
2016年09月13日 09:49:53skrahsetmessages: + msg276229
2016年09月13日 09:46:06vstinnersetmessages: + msg276226
2016年09月13日 09:36:05skrahsetmessages: + msg276224
2016年09月13日 07:45:56vstinnercreate

AltStyle によって変換されたページ (->オリジナル) /