Library for programming Abstract Syntax Trees in Python

Question

I'm creating a tree to represent a simple language. I'm very familiar with Abstract Syntax Trees, and have worked on frameworks for building and using them in C++. Is there a standard python library for specifying or manipulating arbitrary ASTs? Failing that, is there a tree library which is useful for the same purpose?

Note, I am not manipulating Python ASTs, so I think the AST module isn't suitable.

Answer 1

ASTs are very simple to implement in Python. For example, for my pycparser project (a complete C parser in Python) I've implemented ASTs based on ideas borrowed from Python's modules. The various AST nodes are specified in a YAML configuration file, and I generate Python code for these nodes in Python itself.

• Paul Biggar: Quite nice. I had hoped for something already nicely abstracted into a library, with documentation, etc (no offence). My grammar has only 6 types, so its relatively straightforward to write it out myself. I just thought there might be a standard library people use.
• Eli Bendersky: @Paul, if you give it some thought, the best way to use an AST is to define a class for each node type. This is the most polymorphic and clean way to traverse the tree later (with a NodeVisitor). Therefore, you have to write those classes anyway. Considering how little code an AST in Python requires, I doubt that a library is needed here
• Eli Bendersky: @AndersonGreen: thanks, refreshed the main link and removed others

Answer 2

pyast is a package for building declarative abstract syntax trees.

Answer 3

If you represent your grammar elements as expressions in pyparsing, you can attach a parse action to each expression which returns a class instance containing the parsed tokens in a parser-specific type. There are a couple of examples on the pyparsing wiki that illustrate this technique (invRegex.py, simpleBool.py and evalArith.py). (These grammars all use the operatorPrecedence built-in, which can obscure some of the grammar structure, but

Answer 4

This blog entry, though short on implementation detail, describes a nice interface that Python ASTs could implement.

http://chris-lamb.co.uk/2006/12/08/visitor-pattern-in-python/

• Peteris: This article provides a good actual implementation of the pattern: tavianator.com/the-visitor-pattern-in-python.