-
Notifications
You must be signed in to change notification settings - Fork 33
Open
@simon-weber
Description
With this test suite:
# test_example.py from unittest import TestCase class TestExample(TestCase): def test_fail(self): assert False
Running works as expected:
$ pytest -p ciqueue.pytest --queue 'redis://localhost:6379/0?worker=0&build=repro&retry=0&timeout=10' test_example.py
========================================================= test session starts =========================================================
platform darwin -- Python 2.7.13, pytest-4.0.1, py-1.7.0, pluggy-0.6.0
...
collected 1 item
test_example.py F
============================================================== FAILURES ===============================================================
________________________________________________________ TestExample.test_fail ________________________________________________________
self = <test_example.TestExample testMethod=test_fail>
def test_fail(self):
> assert False
E AssertionError: assert False
test_example.py:5: AssertionError
====================================================== 1 failed in 0.06 seconds =======================================================
However, reporting shows that the test passed:
$ pytest -p ciqueue.pytest_report --queue 'redis://localhost:6379/0?worker=0&build=repro&retry=0&timeout=10' test_example.py
========================================================= test session starts =========================================================
platform darwin -- Python 2.7.13, pytest-4.0.1, py-1.7.0, pluggy-0.6.0
...
collected 1 item
test_example.py . [100%]
====================================================== 1 passed in 0.02 seconds =======================================================
This appears to be due to ciqueue's makereport triggering before the one in pytest which is responsible for setting call.excinfo for TestCase-style tests.
From these docs it looks like:
- the ordering of multiple
tryfirst=Truehooks is undefined hookwrapper=Truecan execute code before atryfirsthook
So, one fix could be to get pytest to switch to using hookwrapper. I figured I'd raise it with you folks first, though, in case it's something that's easier to fix on this end (by making ciqueue execute later, somehow).
Metadata
Metadata
Assignees
Labels
No labels