[Python-Dev] Memory management in the AST parser & compiler

Tue Nov 29 01:11:11 CET 2005

Neal Norwitz wrote:
> This is an entire function from Python/ast.c.

> Sequences do not know what type they hold, so there needs to be
> different dealloc functions to free them properly (asdl_*_seq_free()).

Well, that's one complication that would go away if
the nodes were PyObjects.
> The memory leak occurs when FunctionDef fails. name, args, body, and
> decorator_seq are all local and would not be freed. The simple
> variables can be freed in each "constructor" like FunctionDef(), but
> the sequences cannot unless they keep the info about which type they
> hold.

If FunctionDef's reference semantics are defined so
that it steals references to its arguments, then here
is how the same function would look with PyObject
AST nodes, as far as I can see:
 static PyObject *
 ast_for_funcdef(struct compiling *c, const node *n)
 {
 /* funcdef: 'def' [decorators] NAME parameters ':' suite */
 PyObject *name = NULL;
 PyObject *args = NULL;
 PyObject *body = NULL;
 PyObject *decorator_seq = NULL;
 int name_i;
 REQ(n, funcdef);
 if (NCH(n) == 6) { /* decorators are present */
 	decorator_seq = ast_for_decorators(c, CHILD(n, 0));
 	if (!decorator_seq)
 	 goto error;
 	name_i = 2;
 }
 else {
 	name_i = 1;
 }
 name = NEW_IDENTIFIER(CHILD(n, name_i));
 if (!name)
 	goto error;
 else if (!strcmp(STR(CHILD(n, name_i)), "None")) {
 	ast_error(CHILD(n, name_i), "assignment to None");
 	goto error;
 }
 args = ast_for_arguments(c, CHILD(n, name_i + 1));
 if (!args)
 	goto error;
 body = ast_for_suite(c, CHILD(n, name_i + 3));
 if (!body)
 	goto error;
 return FunctionDef(name, args, body, decorator_seq, LINENO(n));
 error:
 Py_XDECREF(body);
 Py_XDECREF(decorator_seq);
 Py_XDECREF(args);
 Py_XDECREF(name);
 return NULL;
 }
The only things I've changed are turning some type
declarations into PyObject * and replacing the
deallocation functions at the end with Py_XDECREF!
Maybe there are other functions where it would not
be so straightforward, but if this really is a
typical AST function, switching to PyObjects looks
like it wouldn't be difficult at all, and would
actually make some things simpler.
-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	 | A citizen of NewZealandCorp, a	 |
Christchurch, New Zealand	 | wholly-owned subsidiary of USA Inc. |
greg.ewing at canterbury.ac.nz	 +--------------------------------------+