homepage

The XincludeTest test-case in test_xml_etree is now skipped, because it fails in an intermittent manner. I can reproduce the failure when running full regrtest with -j1, but not -j8, and not when run individually.
The failure is most likely due to the test itself, in the way it imports the Python ET, leaving _elementtree out. 
This may be related to http://bugs.python.org/issue14035 

After some tinkering I found which test when run before test_xml_etree causes it to crash:
$ ./python -m test.regrtest test___all__ test_xml_etree
[1/2] test___all__
[2/2] test_xml_etree
Fatal Python error: Segmentation fault
Current thread 0x00007f771ecec700:
 File "/home/eliben/python-src/33/Lib/xml/etree/ElementTree.py", line 895 in _namespaces
 File "/home/eliben/python-src/33/Lib/xml/etree/ElementTree.py", line 843 in write
 File "/home/eliben/python-src/33/Lib/test/test_xml_etree.py", line 103 in serialize
 File "/home/eliben/python-src/33/Lib/test/test_xml_etree.py", line 2020 in test_xinclude_default
 File "/home/eliben/python-src/33/Lib/unittest/case.py", line 385 in _executeTestPart
 File "/home/eliben/python-src/33/Lib/unittest/case.py", line 440 in run
 File "/home/eliben/python-src/33/Lib/unittest/case.py", line 492 in __call__
 File "/home/eliben/python-src/33/Lib/unittest/suite.py", line 105 in run
 File "/home/eliben/python-src/33/Lib/unittest/suite.py", line 67 in __call__
 File "/home/eliben/python-src/33/Lib/unittest/suite.py", line 105 in run
 File "/home/eliben/python-src/33/Lib/unittest/suite.py", line 67 in __call__
 File "/home/eliben/python-src/33/Lib/test/support.py", line 1282 in run
 File "/home/eliben/python-src/33/Lib/test/support.py", line 1383 in _run_suite
 File "/home/eliben/python-src/33/Lib/test/support.py", line 1417 in run_unittest
 File "/home/eliben/python-src/33/Lib/test/test_xml_etree.py", line 2311 in test_main
 File "/home/eliben/python-src/33/Lib/test/regrtest.py", line 1238 in runtest_inner
 File "/home/eliben/python-src/33/Lib/test/regrtest.py", line 919 in runtest
 File "/home/eliben/python-src/33/Lib/test/regrtest.py", line 710 in main
 File "/home/eliben/python-src/33/Lib/test/regrtest.py", line 1829 in <module>
 File "/home/eliben/python-src/33/Lib/runpy.py", line 75 in _run_code
 File "/home/eliben/python-src/33/Lib/runpy.py", line 162 in _run_module_as_main
Segmentation fault (core dumped)
Note that this is the Python test crashing. test_xml_etree_c works fine

As suspected, the cause is that xinclude_loader manages to somehow import the C version of ET, although test_xml_etree tries to enforce the Python version. This is probably because test___all__ imports all modules and leaves stuff in the import cache.
What causes the segfault itself is that C Elements are mixed with Python Elements, and they're incompatible.

Here is a patch that solves the current problem.
A longer term solution would be to have a cleaner test plan for ET in general, without monkey-patching at all, and without state that causes test-order dependencies.

New changeset 5782efaa8d68 by Eli Bendersky in branch 'default':
Make the test more resilient to test-run order (closes #15075)
http://hg.python.org/cpython/rev/5782efaa8d68 

Opened #15083 to track the *actual* solution to this problem, which should restructure the tests to be safer.

> File "/home/eliben/python-src/33/Lib/test/regrtest.py", line 710 in main
> File "/home/eliben/python-src/33/Lib/test/regrtest.py", line 1829 in <module>
> File "/home/eliben/python-src/33/Lib/runpy.py", line 75 in _run_code
> File "/home/eliben/python-src/33/Lib/runpy.py", line 162 in _run_module_as_main
> Segmentation fault (core dumped)
Even if the ET tests are hacking sys.modules, I assume that the Python interpreter should not crash so badly.
It reveals probably a flaw somewhere else.
I suggest to re-open this to track the Segmentation fault.

No need, the cause for the segfault is known. What happens is that Element objects from the Python module get intermixed with Element objects from the C module, and these are incompatible. The C module traversal functions assume they can cast Elements to a known structure (ElementObject), and when they get the invalid Elements they crash. Adding runtime checks everywhere is too costly. This situation cannot arise in a valid way.

If runtime checks are needed to prevent mixing arbitrary objects into the tree, then I don't think they should be considered too costly.
I agree with Florent that this is worth reopening. It doesn't look like a "Tests" bug to me rather a "Lib"/"XML" bug.

I investigated a bit, and type checks were added to protect from such mixing. Please open a new patch with a speficic reproducer if you run into similar problems (interepreter crash).