homepage

In September 2019, the _ast extension module was converted to PEP 384 (stable ABI): bpo-38113.
In bpo-41631, I moved the _ast module state back to a global state, rather than a regular module state, to fix multiple bugs.
The state should be made per intepreter (moved into PyInterpreterState).
Also, each interpreter should have its own _ast types (_ast.AST, _ast.Constant, etc.), rather than sharing all types.
Hopefully, in 3.9, _ast types have been converted to heap types.
To fix bpo-41631, I wanted to convert _ast.AST heap type back into a static type, to prevenet a subinterpreter to modify it, but Petr Viktorin asked me to leave it as it is:
https://github.com/python/cpython/pull/21961#issuecomment-685821519
I agree since I would like to convert all static types to heap types: bpo-40077.

I started this here: https://github.com/python/cpython/commit/60960cba606573450e76934c954787419524147d
Feel free to take that code.

> I started this here: https://github.com/python/cpython/commit/60960cba606573450e76934c954787419524147d Feel free to take that code.
Oh thanks! I was looking for your work, but I failed to find it.
It is really helpful!
I started to hack asdl_c.py locally, but Pablo asked me in private to wait a few days, since he is going to push large changes on this file soon.
This issue is an enhancement which can wait.

New changeset 5cf4782a2630629d0978bf4cf6b6340365f449b2 by Victor Stinner in branch 'master':
bpo-41796: Make _ast module state per interpreter (GH-23024)
https://github.com/python/cpython/commit/5cf4782a2630629d0978bf4cf6b6340365f449b2

There is a leak (I just marked bpo-42250 as duplicate of this issue):
https://buildbot.python.org/all/#/builders/205/builds/83
OK
......
test_ast leaked [23640, 23636, 23640] references, sum=70916
test_ast leaked [7932, 7930, 7932] memory blocks, sum=23794
1 test failed again:
 test_ast
test_subinterpreter() test leaks.
There are two problems:
* _PyAST_Fini() is only called in the main interpreter, I forgot to remove the "if _Py_IsMainInterpreter()"
* _PyAST_Fini() is called after the last GC collection, whereas AST_type contains a reference to itself (as any Python type) in its tp_mro member. A GC collection is required to destroy the type. _PyAST_Fini() must be called before the last GC collection.

New changeset fd957c124c44441d9c5eaf61f7af8cf266bafcb1 by Victor Stinner in branch 'master':
bpo-41796: Call _PyAST_Fini() earlier to fix a leak (GH-23131)
https://github.com/python/cpython/commit/fd957c124c44441d9c5eaf61f7af8cf266bafcb1

The leak is fixed, I close again the issue. Thanks for the report Pablo.