On Thu, Dec 16, 2021 at 10:54 AM Guido van Rossum <[email protected]> wrote: > > Eric has been looking into this. It's probably the only solution if we can't > get immutable objects.
Yep. I've investigated the following approach (for the objects exposed in the public and limited C-API): * add a pointer field to PyInterpreterState (or a sub-struct) for each of the objects * for the main interpreter, set those pointers to the existing statically declared objects * for subinterpreters make a copy (memcpy()?) and fix it up * add a lookup API and encourage extensions to use it * for 3.11+ change the symbols to macros: + in the internal C-API (Py_BUILD_CORE), the macro would resolve to the corresponding PyInterpreterState field + in the public C-API (and limited API extensions built with 3.11+), the macro would resolve to a call to a (non-inline) lookup function + for limited API extensions built against earlier Python versions we'd still export the existing symbols * limited API extensions built against pre-3.11 Python would only be allowed to run in the main interpreter on 3.11+ + they probably weren't built with subinterpreters in mind anyway There are still a number of details to sort out, but nothing that seems like a huge obstacle. Here are the ones that come to mind, along with other details, caveats, and open questions: * the static types exposed in the C-API are PyObject values rather than pointers + I solved this by dereferencing the result of the lookup function (Guido's idea), e.g. #define PyTuple_Type (*(_Py_GetObject_Tuple())) * there is definitely a penalty to using a per-interpreter lookup function + this would only apply to extension modules since internally we would access the PyInterpreterState fields directly + this is mostly a potential problem only when the object is directly referenced frequently (e.g. a tight loop), + the impact would probably center on use of the high-frequency singletons (None, True, False) and possibly with Py*_CheckExact() calls + would it be enough of a problem to be worth mitigating? how would we do so? * static types in extensions can't have tp_base set to a builtin type (since the macro won't resolve) + extensions that support subinterpreters (i.e. PEP 489) won't be using static types (a weak assumption) + extensions that do not support subinterpreters and still have static types would probably break + how to fix that? * limited API extensions built against 3.11+ but running under older Python versions would break? + how to fix that? > But I would prefer the latter, if we can get the performance penalty low > enough. Absolutely. Using immortal objects to solve this is a much simpler solution. -eric _______________________________________________ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/7RPTHCLEUHR34PIJKRN453UEWCAI56NW/ Code of Conduct: http://python.org/psf/codeofconduct/