• 1

• 35.2k
• 13
• 134
• 238

• 997
• 1
• 5
• 23

Apparently everyone gets burned by their Python unittests not running in the order they want.

I am not in the business of telling people not to do perfectly reasonable things they want to do, so I consider "write your tests differently" Not An Answer to the question "how do I control the order of my TestCase subclasses".

With that in mind, I also consider "Why do you want to control the order of unittests? Just write them differently" as a lame, non-answer to this CR post.

I do, however, consider the following (more than) an answer to the above question:

import unittest
def suiteFactory(
 *testcases,
 testSorter = None,
 suiteMaker = unittest.makeSuite,
 newTestSuite = unittest.TestSuite
 ):
 """
 make a test suite from test cases, or generate test suites from test cases.
 
 *testcases = TestCase subclasses to work on
 testSorter = sort tests using this function over sorting by line number
 suiteMaker = should quack like unittest.makeSuite.
 newTestSuite = should quack like unittest.TestSuite.
 """
 if testSorter is None:
 ln = lambda f: getattr(tc, f).__code__.co_firstlineno
 testSorter = lambda a, b: ln(a) - ln(b)
 
 test_suite = newTestSuite()
 for tc in testcases:
 test_suite.addTest(suiteMaker(tc, sortUsing=testSorter))
 
 return test_suite
def caseFactory(
 scope = globals().copy(),
 caseSorter = lambda f: __import__("inspect").findsource(f)[1],
 caseSuperCls = unittest.TestCase,
 caseMatches = __import__("re").compile("^Test")
 ):
 """
 get TestCase-y subclasses from frame "scope", filtering name and attribs
 scope = iterable to use for a frame; preferably a hashable (dictionary).
 caseMatches = regex to match function names against; blank matches every TestCase subclass
 caseSuperCls = superclass of test cases; unittest.TestCase by default
 caseSorter = sort test cases using this function over sorting by line number
 """
 from re import match
 return sorted(
 [
 scope[obj] for obj in scope
 if match(caseMatches, obj)
 and issubclass(scope[obj], caseSuperCls)
 ],
 key=caseSorter
 )
if __name__ == '__main__':
 cases = suiteFactory(*caseFactory())
 runner = unittest.TextTestRunner(verbosity=2)
 runner.run(cases)
 
 

A gist.

For reference, here're some example tests:

import unittest
class Test_MyTests(unittest.TestCase):
 def test_run_me_first(self): pass
 def test_2nd_run_me(self): pass
 def test_and_me_last(self): pass
 
class Test_AnotherClass(unittest.TestCase):
 def test_first(self): pass
 def test_after_first(self): pass
 def test_de_last_ding(self): pass
if __name__ == "__main__": unittest.main(verbosity=2) 

(The names are all unittest cares about, and all I need for demonstration.)

Here's what running that looks like:

test_after_first (__main__.Test_AnotherClass) ... ok
test_de_last_ding (__main__.Test_AnotherClass) ... ok
test_first (__main__.Test_AnotherClass) ... ok
test_2nd_run_me (__main__.Test_MyTests) ... ok
test_and_me_last (__main__.Test_MyTests) ... ok
test_run_me_first (__main__.Test_MyTests) ... ok

Oh no! My tests weren't run in the order I (削除) thought they'd be (削除ここまで) wanted.

Running the content of the gist, aka same tests, but replacing the ifmain with the full code from above:

test_run_me_first (__main__.Test_MyTests) ... ok
test_2nd_run_me (__main__.Test_MyTests) ... ok
test_and_me_last (__main__.Test_MyTests) ... ok
test_first (__main__.Test_AnotherClass) ... ok
test_after_first (__main__.Test_AnotherClass) ... ok
test_de_last_ding (__main__.Test_AnotherClass) ... ok

Success! The tests were run based on where in the file they were defined.

I think this is pretty useful, and quite optimal. But last time I thought that, I was really wrong.

^{Incidentally, if you don't want the TestCases to run all as a single suite, but as individual suites with individual runners, just change suiteFactory to be a generator, and change the ifmain to iterate over said generator. I way prefer when my tests run all together, and functions are either generators or they aren't, hence the way it's written.}