homepage

Currently we use the _PyRuntime static global to access the runtime state in various places. At the same time, in thread contexts we get access to the thread state from Thread-Local Storage and the interpreter state by indirection from there. We should do the same for the runtime state instead of using the global directly.
My plan is to add a PyInterpreterState.runtime field. It can then be used in the same way we use PyThreadState.interp to access the interpreter state.

I don't see the point of PyInterpreterState.runtime:
* I understood that we should have and only one _PyRuntimeState instance: _PyRuntime. Why not accessing it directly?
* I understood that most _PyRuntimeState fields should be moved to PyInterpreterState. Once this refactoring will be done, we will almost never have to access _PyRuntime.

Eric is working on a different approach: https://bugs.python.org/issue36854 

New changeset 396e0a8d9dc65453cb9d53500d0a620602656cfe by Eric Snow in branch 'master':
bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)
https://github.com/python/cpython/commit/396e0a8d9dc65453cb9d53500d0a620602656cfe

New changeset 0fd2c300c2a85b3b227a907b2a39ef79fa86d850 by Victor Stinner in branch 'master':
Revert "bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)" (GH-13795)
https://github.com/python/cpython/commit/0fd2c300c2a85b3b227a907b2a39ef79fa86d850

This change introduced a regression:
https://bugs.python.org/issue37135#msg344511
I had to revert the change to fix Python for 3.8 beta1 release.

By the way, I have questions about the rationale of this change.
> Currently we use the _PyRuntime static global to access the runtime state in various places. At the same time, in thread contexts we get access to the thread state from Thread-Local Storage and the interpreter state by indirection from there. We should do the same for the runtime state instead of using the global directly.
Right now I'm confused. _PyRuntimeState contains scatted states: things which should be moved to PyInterpreterState or even PyThreadState, and things which should remain in _PyRuntimeState. My notes on this topic:
https://pythoncapi.readthedocs.io/runtime.html
Is PyInterpreterState.runtime a temporary fix until things are moved?
Or do you plan to never access _PyRuntime directly, and always through interp->runtime?
It seems like in the short term, we must continue to access "_PyRuntime" through a separated "runtime" pointer passed to functions, to handle daemon threads during Python shutdown.

See https://bugs.python.org/issue37135#msg344509 to reproduce https://bugs.python.org/issue37135 crash without multiprocessing: with this procedure, I reproduce the crash immediately on Linux. No need for a magic super slow buildbot.

I added PyInterpreterState.runtime in bpo-36710:
commit 01b1cc12e7c6a3d6a3d27ba7c731687d57aae92a
Author: Victor Stinner <vstinner@python.org>
Date: Wed Nov 20 02:27:56 2019 +0100
 bpo-36710: Add PyInterpreterState.runtime field (GH-17270)
 
 Add PyInterpreterState.runtime field: reference to the _PyRuntime
 global variable. This field exists to not have to pass runtime in
 addition to tstate to a function. Get runtime from tstate:
 tstate->interp->runtime.
 
 Remove "_PyRuntimeState *runtime" parameter from functions already
 taking a "PyThreadState *tstate" parameter.
 
 _PyGC_Init() first parameter becomes "PyThreadState *tstate".
--
> This change introduced a regression:
> https://bugs.python.org/issue37135#msg344511
Funny/not funny, I introduced the same bug and I fixed it in bpo-39877.