homepage

Currently, the type lookup cache is shared by all interpreter which causes multiple issues:
* The version tag is currently protected by the GIL, but it would require a new lock if the GIL is made per interpreter (bpo-40512)
* Clearing the cache in an interpreter clears the cache in all interpreters
* The cache has a fixed size of 4096 entries. The cache misses increase with the number of interpreters, since each interpreter has its own types.
I propose to make the type lookup cache per interpreter.

New changeset 41010184880151d6ae02a226dbacc796e5c90d11 by Victor Stinner in branch 'master':
bpo-42745: Make the type cache per-interpreter (GH-23947)
https://github.com/python/cpython/commit/41010184880151d6ae02a226dbacc796e5c90d11

41010184880151d6ae02a226dbacc796e5c90d11 introduced some reference leaks:
https://buildbot.python.org/all/#/builders/384/builds/136
41010184880151d6ae02a226dbacc796e5c90d11 is the first bad commit
commit 41010184880151d6ae02a226dbacc796e5c90d11
Author: Victor Stinner <vstinner@python.org>
Date: Sat Dec 26 01:45:43 2020 +0100
 bpo-42745: Make the type cache per-interpreter (GH-23947)
 Make the type attribute lookup cache per-interpreter.
 Add private _PyType_InitCache() function, called by PyInterpreterState_New().
 Continue to share next_version_tag between interpreters, since static
 types are still shared by interpreters.
 Remove MCACHE macro: the cache is no longer disabled if the
 EXPERIMENTAL_ISOLATED_SUBINTERPRETERS macro is defined.
 Include/internal/pycore_interp.h | 22 +++
 Include/internal/pycore_object.h | 3 +
 Include/internal/pycore_pylifecycle.h | 2 +-
 .../2020-12-25-23-30-58.bpo-42745.XsFoHS.rst | 1 +
 Objects/typeobject.c | 178 ++++++++++++---------
 Python/pylifecycle.c | 2 +-
 Python/pystate.c | 2 +
 7 files changed, 128 insertions(+), 82 deletions(-)
 create mode 100644 Misc/NEWS.d/next/Core and Builtins/2020-12-25-23-30-58.bpo-42745.XsFoHS.rst
bisect run success
I assume PR 23953 fixes those? If that is so, could we land it before more buildbots start to fail?

> I assume PR 23953 fixes those?
Yes, it fix the 7 tests which leak:
./python -m test -R 3:3 -j0 test__xxsubinterpreters test_ast test_atexit test_capi test_interpreters test_threading

New changeset f4507231e3f0cf8827cec5592571ce371c6813e8 by Victor Stinner in branch 'master':
bpo-42745: finalize_interp_types() calls _PyType_Fini() (GH-23953)
https://github.com/python/cpython/commit/f4507231e3f0cf8827cec5592571ce371c6813e8