[Python-Dev] Checking in a broken test was: Re: [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py

Sun Jan 8 00:24:06 CET 2006

[Neal Norwitz]
> ...
> In the past, we haven't checked in tests which are known to be broken.

It's an absolute rule that you never check in a change (whether a test
or anything else) that causes ``regretst.py -uall`` to fail. Even if
it passes on your development box, but fails on someone else's box,
it's fair game to revert that change.
Since this particular change caused -uall to fail on all boxes, I had
no qualms about reverting rev 41940.
Reinhold, if you're working on a fix for the problem too, feel free to
check that test in on a branch instead (short-lived branches are easy
& fast under SVN). Branches aren't required to pass any tests. The
test on the branch can be merged into the trunk later, after a fix has
also been checked in on that branch.
...
> There are many open bug reports that fall into two broad categories:
> incorrect behaviour and crashers. I've been thinking about adding two
> tests which incorporate these bugs as a way of consolidating where the
> known problems are. Also, it's great when we have test cases that can
> be moved to the proper place once the fix has been checked in.
>> I'm proposing something like add two files to Lib/test:
> outstanding_bugs.py and outstanding_crashes.py. Both would be normal
> test files with info about the bug report and the code that causes
> problems.
>> This test in test_compiler should be moved to outstanding_bugs.py.

That would be fine provided that the failing tests aren't normally run
via -uall.