homepage

The be able to debug bpo-30280 using my bisection script:
https://github.com/haypo/misc/blob/205fea00d509ddd1cffa30c5c6f2da012f3ec54e/python/bisect_test.py
I would like to be able to mark tests which alter the environment as failed, using an explicit option.
It would be nice to be able to enable this option by default, or even remove the option and just make the behaviour as the default, but it would require to fix some issues first (like bpo-30280).

I implemented the feature:
https://github.com/python/cpython/pull/2402
My PR changes also the exit code from 1 to 2 (tests failed) or 3 (interrupted), to be able to distinguish how regrtest failed. It should also help to bisect failing tests.

I updated my PR to use the exit code 130 when regrtest is interrupted. Serhiy Storchaka found a reference:
"On Linux scripts terminated with Ctrl-C return exit code 130 (http://tldp.org/LDP/abs/html/exitcodes.html)."

New changeset cdee3f14f7f4c995e7eedb0bf6a67e260c739f7d by Victor Stinner in branch 'master':
bpo-30764: test_subprocess uses SuppressCrashReport (#2405)
https://github.com/python/cpython/commit/cdee3f14f7f4c995e7eedb0bf6a67e260c739f7d

New changeset 849b062a82ca2f09e33259d34067faba196c9e23 by Victor Stinner in branch '3.5':
bpo-30764: test_subprocess uses SuppressCrashReport (#2405) (#2411)
https://github.com/python/cpython/commit/849b062a82ca2f09e33259d34067faba196c9e23

New changeset 63f54c68936d648c70ca411661e4208329edcf26 by Victor Stinner in branch 'master':
bpo-30764: regrtest: add --fail-env-changed option (#2402)
https://github.com/python/cpython/commit/63f54c68936d648c70ca411661e4208329edcf26

New changeset 9ad50d94599aed0c37beaf78948ec271c8aa3881 by Victor Stinner in branch '3.6':
bpo-30764: test_subprocess uses SuppressCrashReport (#2405) (#2410)
https://github.com/python/cpython/commit/9ad50d94599aed0c37beaf78948ec271c8aa3881

New changeset 2097b9e0ef32ab7a0d745edc0f707c615780c006 by Victor Stinner in branch '2.7':
[2.7] bpo-30764: test_subprocess uses SuppressCrashReport (#2405) (#2412)
https://github.com/python/cpython/commit/2097b9e0ef32ab7a0d745edc0f707c615780c006

I tried to implement test.support.SuppressCrashReport on Windows using only ctypes for Python 2.7, but I failed: see attached suppress_crash.py. So I created https://github.com/python/cpython/pull/2423 which adds new functions and constants to the _testcapi module. IMHO it's more reliabe to do that, but it adds more C code.

Test failure on x86-64 Sierra 2.7 buildbot:
http://buildbot.python.org/all/builders/x86-64%20Sierra%202.7/builds/111/steps/test/logs/stdio
test_shell_string_with_spaces (test.test_subprocess.CommandsWithSpaces) ... test test_subprocess failed -- Traceback (most recent call last):
 File "/Users/buildbot/buildarea/2.7.billenstein-sierra/build/Lib/test/test_subprocess.py", line 1302, in test_child_terminated_in_stopped_state
 self.assertLess(returncode, 0) # signal death, likely SIGSEGV.
AssertionError: 1 not less than 0
skipped 'mswindows only'
======================================================================
FAIL: test_child_terminated_in_stopped_state (test.test_subprocess.POSIXProcessTestCase)
Test wait() behavior when waitpid returns WIFSTOPPED; issue29335.
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/Users/buildbot/buildarea/2.7.billenstein-sierra/build/Lib/test/test_subprocess.py", line 1302, in test_child_terminated_in_stopped_state
 self.assertLess(returncode, 0) # signal death, likely SIGSEGV.
AssertionError: 1 not less than 0

Oops, I used the wrong bpo number for the test_subprocess fixes: the right bpo is bpo-30448!

New changeset 95a91df0363f88722ea2860d9d42a0b04b3f8be2 by Victor Stinner in branch '2.7':
bpo-30764: Windows support.SuppressCrashReport (#2423)
https://github.com/python/cpython/commit/95a91df0363f88722ea2860d9d42a0b04b3f8be2

New changeset 35d2ca2b94a6ff29e763ddb7727166f0592edfa2 by Victor Stinner in branch '3.6':
[3.6] bpo-30523, bpo-30764, bpo-30776: Sync regrtest from master (#2441)
https://github.com/python/cpython/commit/35d2ca2b94a6ff29e763ddb7727166f0592edfa2

New changeset de1850bb03f8225cbff85f437b6e972bf9b68c2a by Victor Stinner in branch '3.5':
[3.5] bpo-30523, bpo-30764, bpo-30776: Sync regrtest from master (#2442)
https://github.com/python/cpython/commit/de1850bb03f8225cbff85f437b6e972bf9b68c2a

New changeset fea98bfcff6ccf9142daa97677fe86c1fdf8e63e by Victor Stinner in branch '2.7':
[2.7] bpo-30523, bpo-30764, bpo-30776: Sync regrtest from master (#2444)
https://github.com/python/cpython/commit/fea98bfcff6ccf9142daa97677fe86c1fdf8e63e

The initial issue, the "regrtest: Add --fail-env-changed option" feature, has been implemented in 2.7, 3.5; 3.6 and master branches. So I close this issue.
Again, this issue got hijacked by the test_subprocess change, bpo-30448, because I wrote the wrong bpo number in my commits ;-)

New changeset 5e87592fd12e0b7c41edc11d4885ed7298d5063b by Victor Stinner in branch 'master':
bpo-30764: Fix regrtest --fail-env-changed --forever (#2536)
https://github.com/python/cpython/commit/5e87592fd12e0b7c41edc11d4885ed7298d5063b

New changeset 4132adb4b8f90893d8f19ff46edff56f189faca0 by Victor Stinner in branch '3.6':
bpo-30764: Fix regrtest --fail-env-changed --forever (#2536) (#2539)
https://github.com/python/cpython/commit/4132adb4b8f90893d8f19ff46edff56f189faca0

New changeset 0b1210739d12c26e4a161ffd494bd572d49b2483 by Victor Stinner in branch '3.5':
[3.5] bpo-29512, bpo-30764, bpo-30776: Backport regrtest enhancements from 3.6 to 3.5 (#2540)
https://github.com/python/cpython/commit/0b1210739d12c26e4a161ffd494bd572d49b2483

New changeset d7955b8196578306e9d86f6c61c9cb3ee72edab0 by Victor Stinner in branch '2.7':
[2.7] bpo-29512, bpo-30764: Backport regrtest enhancements from 3.5 to 2.7 (#2541)
https://github.com/python/cpython/commit/d7955b8196578306e9d86f6c61c9cb3ee72edab0

New changeset 8207c17486baece8ed0ac42d9f8d69ecec4ba7e4 by Victor Stinner in branch 'master':
Revert "bpo-30822: Fix testing of datetime module." (#2588)
https://github.com/python/cpython/commit/8207c17486baece8ed0ac42d9f8d69ecec4ba7e4