[Python-Dev] Problems with the Python Memory Manager

Wed Nov 16 11:59:03 CET 2005

 Travis> More to the point, however, these scalar objects were allocated
 Travis> using the standard PyObject_New and PyObject_Del functions which
 Travis> of course use the Python memory manager. One user ported his
 Travis> (long-running) code to the new scipy core and found much to his
 Travis> dismay that what used to consume around 100MB now completely
 Travis> dominated his machine consuming up to 2GB of memory after only a
 Travis> few iterations. After searching many hours for memory leaks in
 Travis> scipy core (not a bad exercise anyway as some were found), the
 Travis> real problem was tracked to the fact that his code ended up
 Travis> creating and destroying many of these new array scalars.
What Python object were his array elements a subclass of?
 Travis> In the long term, what is the status of plans to re-work the
 Travis> Python Memory manager to free memory that it acquires (or
 Travis> improve the detection of already freed memory locations). 
None that I'm aware of. It's seen a great deal of work in the past and
generally doesn't cause problems. Maybe your user's usage patterns were
a bad corner case. It's hard to tell without more details.
Skip