homepage

To get one GIL per interpreter (bpo-40512), remaining _PyRuntimeState.ceval members should be moved to PyInterpreterState.ceval. The work started in bpo-39984 with ceval "pending calls" and ceval "eval breaker".
There are 4 remaining fields:
struct _ceval_runtime_state {
 int recursion_limit;
 /* Request for dropping the GIL */
 _Py_atomic_int gil_drop_request;
 /* Request for checking signals. */
 _Py_atomic_int signals_pending;
 struct _gil_runtime_state gil;
};
The most complex part will be to move the "gil" member: bpo-40512 lists all sub-tasks which must be completed before being able to have one GIL per interpreter without crashing.

See also: https://github.com/ericsnowcurrently/multi-core-python/issues/34 

New changeset 4e01946cafca0cf49f796c3118e0d65237bcad69 by Victor Stinner in branch 'master':
bpo-40513: Per-interpreter signals pending (GH-19924)
https://github.com/python/cpython/commit/4e01946cafca0cf49f796c3118e0d65237bcad69

Python/ceval.c also has "int _Py_CheckRecursionLimit = Py_DEFAULT_RECURSION_LIMIT;". _Py_CheckRecursiveCall() has this comment:
 /* Needed for ABI backwards-compatibility (see bpo-31857) */
 _Py_CheckRecursionLimit = recursion_limit;
I converted Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() macros into opaque functions in bpo-38644: commit f4b1e3d7c64985f5d5b00f6cc9a1c146bbbfd613.
I don't think that the "ABI backwards-compatibility" rationale is correct. These macros never worked in the limited C API: they accessed PyThreadState.recursion_depth and PyThreadState.overflowed members, whereas the PyThreadState structure is opaque in the limited C API (on purpose, see also bpo-39947).
I think that it is now safe to simply remove _Py_CheckRecursionLimit.

New changeset 0b1e3307e24b0af45787ab6456535b8346e0239a by Victor Stinner in branch 'master':
bpo-40513: Per-interpreter gil_drop_request (GH-19927)
https://github.com/python/cpython/commit/0b1e3307e24b0af45787ab6456535b8346e0239a

For signals_pending, see also: bpo-40010 "Inefficient signal handling in multithreaded applications" which added _Py_ThreadCanHandleSignals() and _Py_ThreadCanHandlePendingCalls() functions.

New changeset 4e30ed3af06ae655f4cb8aad8cba21f341384250 by Victor Stinner in branch 'master':
bpo-40513: Per-interpreter recursion_limit (GH-19929)
https://github.com/python/cpython/commit/4e30ed3af06ae655f4cb8aad8cba21f341384250

FWIW, I think it would make sense to keep "signals_pending" under _PyRuntimeState rather than moving it to PyInterpreterState.
Signals are only handled by the main interpreter (in its main thread). Even though "signals_pending" is useful to only one interpreter in the runtime, making it per-interpreter sends the wrong message that it is significant at that level. This may be confusing to readers of the code.
At the very least there should be a clear comment with the field in Include/internal/pycore_interp.h explaining how it is only used by the main interpreter and why we made it per-interpreter anyway.

From a user perspective, does it make sense to have a different recursion_limit per interpreter? I don't see a problem with it. However, does it make sense to also keep a global value that we default to when a per-interpreter value is not set? I think it might.
I suppose a bigger question is what users will expect the recursion limit (AKA "sys.getrecursionlimit()") to be for a newly created subinterpreter. Will it be some global default? Will it be the value from the parent interpreter? I'd go with a global default, which would imply that the default value should be stored under _PyRuntimeState like we had it (but still keep the actual per-interpreter field for the actual value).
FWIW, the existing docs don't really block either approach.

If this issue covers the GIL (which it seems to) then I'd expect _PyRuntimeState.gilstate to be handled here too.

New changeset 299b8c61e9d1a42b929b8deb1b05067876e191e6 by Victor Stinner in branch 'master':
Revert "bpo-40513: Per-interpreter signals pending (GH-19924)" (GH-19932)
https://github.com/python/cpython/commit/299b8c61e9d1a42b929b8deb1b05067876e191e6

> FWIW, I think it would make sense to keep "signals_pending" under _PyRuntimeState rather than moving it to PyInterpreterState.
Oh right. I forgot about the details. I reverted my change, but this time I added a comment to prevent me to write the same change in 6 months :-)

> From a user perspective, does it make sense to have a different recursion_limit per interpreter?
Yes, I think so. Someone might use a lower limit to run a template rendering engine or "unsafe" code. Well, it's not hard to find random usage for interpreters with a configuration different than the main interpreter.

New changeset 0dd5e7a718997da2026ed64fe054dc36cae4fee7 by Victor Stinner in branch 'master':
bpo-40513: new_interpreter() init GIL earlier (GH-19942)
https://github.com/python/cpython/commit/0dd5e7a718997da2026ed64fe054dc36cae4fee7

New changeset 7be4e350aadf93c4be5c97b7291d0db2b6bc1dc4 by Victor Stinner in branch 'master':
bpo-40513: Per-interpreter GIL (GH-19943)
https://github.com/python/cpython/commit/7be4e350aadf93c4be5c97b7291d0db2b6bc1dc4

New changeset fb2c7c4afbab0514352ab0246b0c0cc85d1bba53 by Victor Stinner in branch 'master':
bpo-40513: _xxsubinterpreters.run_string() releases the GIL (GH-19944)
https://github.com/python/cpython/commit/fb2c7c4afbab0514352ab0246b0c0cc85d1bba53

Well, the remaining field which should be moved is the GIL lock itself. It will likely be the last thing to do in bpo-40512. I consider that the work is done in this issue and so I close it.