Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Generalize PYTEST_CURRENT_TEST for threading #13837

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Liam-DeVoe wants to merge 3 commits into pytest-dev:main
base: main
Choose a base branch
Loading
from Liam-DeVoe:threading-current-test

Conversation

@Liam-DeVoe
Copy link
Contributor

@Liam-DeVoe Liam-DeVoe commented Oct 22, 2025

Part of #13768.

I'm starting to work earnestly on #13768, and figured I would start with a relatively simple fix + some general work for threading tests.

If we really cared about making PYTEST_CURRENT_TEST as good as it could be, then we could track a stack of pushes/pops to PYTEST_CURRENT_TEST, and when a pop occurs, we set it to the most-recent value on the stack instead of setting it to None. This would ensure that PYTEST_CURRENT_TEST always has a value when pytest is running, and is always "a test that is currently running", though there may be multiple currently-running tests. I am happy to implement this if you guys would prefer!

@psf-chronographer psf-chronographer bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Oct 22, 2025
*.orig
*~
.hypothesis/

Copy link
Contributor Author

@Liam-DeVoe Liam-DeVoe Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this ignore rule is duplicated farther down the file

Copy link
Contributor Author

Liam-DeVoe commented Oct 22, 2025
edited
Loading

Ah hm, test_concurrent is going to fail until Session is thread-safe. I have this hacked around locally, but not something I would want to PR yet. I'll drop the tests for now I guess

(Current very-WIP work is here: https://github.com/pytest-dev/pytest/compare/main...Liam-DeVoe:pytest:threading?expand=1)

Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a unacceptable wrong prediction

The current test env var itself doesn't translate to threads to begin with

Copy link
Contributor Author

The current test env var itself doesn't translate to threads to begin with

Yeah, agreed. If this PR is no good, how would you prefer PYTEST_CURRENT_TEST is handled under threading?

Copy link
Member

We need one env var per thread

So some type of indication would be needed

Id says the existing reserved for the pytest session on the main thread

Other threads need a curret test of thread style variables

Existing of the per thread values should be the indicator of multithread pytest as far ad observing processes are concerned

Details should be expermented with

Liam-DeVoe reacted with thumbs up emoji

Copy link
Contributor Author

OK nice. Wasn't sure if pytest wanted to support threads as such a first class citizen with special envvars and such.

We'll need some way for pytest to know when a "runner" thread is executing a test, vs when a helper thread has been spawned by a test that happens to interact with pytest in some way. In other words I don't think it's sufficient to check threading.get_ident() at any point to get "the current thread which is responsible for running this test as part of a thread pool". Currently my ideas are to track the thread id that pytest_runtest_protocol is executed from and call that a runner thread, or to have some explicit way to tell pytest "run with a thread pool" and pytest spawns and tracks the threads itself

Copy link
Member

threading.current_thread() is threading.main_thread() only on the main thread, else its another one

names and idents will be tricky

but one thing is for clear - just hiding key errors on a env var that shouldn't be thread-shared is not gona cut it

Liam-DeVoe reacted with thumbs up emoji

Copy link
Member

as for thread pools - thats not a consideration as of now as setupstate is not separated

Copy link
Contributor Author

Something like this perhaps? PYTEST_CURRENT_TEST remains on the main thread, and PYTEST_CURRENT_TEST_THREAD_{n} for other threads. runtestprotocol tracks what thread is executing each test. I imagine we'll use the _item_to_thread mapping in the future as well, for example mapping captured thread output back to the corresponding item.

Two tests are failing because they invoke pytest_runtest_call directly and bypass runtestprotocol. I initially put the setting of _item_to_thread in runner.py::pytest_runtest_setup, but skipping.py::pytest_runtest_setup is a tryfirst and so if a test has a skip, it raises and never invokes runner.py::pytest_runtest_setup and then the teardown errors because _item_to_thread doesn't have that item.

if thread is threading.main_thread():
return 0

return threads.index(thread) + 1
Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not invent own thread numbering schemes

Copy link
Contributor Author

@Liam-DeVoe Liam-DeVoe Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the envvars be named PYTEST_CURRENT_TEST_THREAD_{thread.ident}instead, eg. PYTEST_CURRENT_TEST_THREAD_8422586432? My thinking was that a more human readable approach would be nice, and also that way someone could tell how many threads are active. Happy to just use thread.ident though.

Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt Oct 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should require a "pytest thread" name for a thread run to be picked no guessing, no random long integers - the env names should be something people can deal with

Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt Oct 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(i considered picking the threading thread name but their default is not ux friendly for env vars)

@Liam-DeVoe Liam-DeVoe changed the title (削除) Make sets to PYTEST_CURRENT_TEST thread-safe (削除ここまで) (追記) Generalize PYTEST_CURRENT_TEST for threading (追記ここまで) Oct 23, 2025
Copy link
Member

@Liam-DeVoe would you mind opening a specific issue about PYTEST_CURRENT_TEST and the problem with threads?

Liam-DeVoe reacted with thumbs up emoji

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@RonnyPfannschmidt RonnyPfannschmidt RonnyPfannschmidt requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /