homepage

I'm opening a separate issue to prevent #40521 from getting too big.
Lists and tuples sometimes leak (starting 69ac6e58f and later):
==1445== 56 bytes in 1 blocks are definitely lost in loss record 1,542 of 4,898
==1445== at 0x4C2DE56: malloc (vg_replace_malloc.c:299)
==1445== by 0x550487: _PyObject_GC_Alloc (gcmodule.c:2233)
==1445== by 0x550487: _PyObject_GC_Malloc (gcmodule.c:2260)
==1445== by 0x550487: _PyObject_GC_New (gcmodule.c:2272)
==1445== by 0x44CB04: PyList_New (listobject.c:144)
==1445== by 0x4E3DE1: init_filters (_warnings.c:88)
==1445== by 0x4E3DE1: warnings_init_state (_warnings.c:120)
==1445== by 0x4E3DE1: _PyWarnings_InitState (_warnings.c:1372)
==1445== by 0x521720: pycore_init_import_warnings (pylifecycle.c:687)
==1445== by 0x521720: pycore_interp_init (pylifecycle.c:735)
==1445== by 0x5246A0: pyinit_config (pylifecycle.c:763)
==1445== by 0x5246A0: pyinit_core (pylifecycle.c:924)
==1445== by 0x5246A0: Py_InitializeFromConfig (pylifecycle.c:1134)
==1445== by 0x4285DC: pymain_init (main.c:66)
==1445== by 0x4296A1: pymain_main (main.c:653)
==1445== by 0x4296A1: Py_BytesMain (main.c:686)
==1445== by 0x578882F: (below main) (libc-start.c:291)
==1445== 64 bytes in 1 blocks are definitely lost in loss record 2,259 of 4,898
==1445== at 0x4C2DE56: malloc (vg_replace_malloc.c:299)
==1445== by 0x550611: _PyObject_GC_Alloc (gcmodule.c:2233)
==1445== by 0x550611: _PyObject_GC_Malloc (gcmodule.c:2260)
==1445== by 0x550611: _PyObject_GC_NewVar (gcmodule.c:2289)
==1445== by 0x48452C: tuple_alloc (tupleobject.c:76)
==1445== by 0x48452C: _PyTuple_FromArray (tupleobject.c:413)
==1445== by 0x435EE0: _PyObject_MakeTpCall (call.c:165)
==1445== by 0x436947: _PyObject_FastCallDictTstate (call.c:113)
==1445== by 0x436947: PyObject_VectorcallDict (call.c:142)
==1445== by 0x61DFC5: builtin___build_class__ (bltinmodule.c:232)
==1445== by 0x5E8A39: cfunction_vectorcall_FASTCALL_KEYWORDS (methodobject.c:440)
==1445== by 0x41F4D5: _PyObject_VectorcallTstate (abstract.h:114)
==1445== by 0x41F4D5: PyObject_Vectorcall (abstract.h:123)
==1445== by 0x41F4D5: call_function (ceval.c:5111)
==1445== by 0x42220E: _PyEval_EvalFrameDefault (ceval.c:3542)
==1445== by 0x4E6882: _PyEval_EvalFrame (pycore_ceval.h:40)
==1445== by 0x4E6882: _PyEval_EvalCode (ceval.c:4366)
==1445== by 0x4E6A65: _PyEval_EvalCodeWithName (ceval.c:4398)

> Lists and tuples sometimes leak (starting 69ac6e58f and later):
It's not a regression. It's just that bpo-40521 helped Valgrind to detect such bug :-)
I wrote bpo-20698 to fix this issue.
I'm also working on a more generic approach to ensure that we don't add new objects to free lists after free lists are cleared.

I'm curious about this bug, so I looked at code changes.
Before Python 3.6, warnings.c "garbage" is declared as a static variable. It is initialized once but never cleared.
In Python 3.7, the variable moved into _PyRuntimeState.gc.
In Python 3.8, _PyGC_Fini() started to clear the "garbage" list. This function is called by Py_FinalizeEx() *after* PyList_Fini().

New changeset 7907f8cbc6923240edb0b5b63adafb871c4c8875 by Victor Stinner in branch 'master':
bpo-40887: Fix finalize_interp_clear() for free lists (GH-20698)
https://github.com/python/cpython/commit/7907f8cbc6923240edb0b5b63adafb871c4c8875

New changeset bcb198385dee469d630a184182df9dc1463e2c47 by Victor Stinner in branch 'master':
bpo-40887: Don't use finalized free lists (GH-20700)
https://github.com/python/cpython/commit/bcb198385dee469d630a184182df9dc1463e2c47

Thanks Stefan for the bug report. It's now fixed and I made sure that such bug cannot strike back.

> It's not a regression. It's just that bpo-40521 helped Valgrind to detect such bug :-)
Yes, I suspected that previously reachable global objects were just unreachable after the new free lists. Thanks for the fix!

All free lists now have assertions to ensure that they are no longer used after their finalization.