Parse Python file and evaluate selected functions

Question

I have a file that contains several python functions, each with some statements.

def func1():
 codeX...
def func2():
 codeY...

codeX and codeY can be multiple statements. I want to be able to parse the file, find a function by name, then evaluate the code in that function.

With the ast module, I can parse the file, find the FunctionDef objects, and get the list of Stmt objects, but how do I turn this into bytecode that I can pass to eval? Should I use the compile module, or the parser module instead?

Basically, the function defs are just used to create separate blocks of code. I want to be able to grab any block of code given the name and then execute that code in eval (providing my own local/global scope objects). If there is a better way to do this than what I described that would be helpful too.

Thanks

• jon_darkstar: eval takes python code, not byte code. it looks like what you need here might be exec instead of eval, but either way they take python code. it sounds like youre pretty much there
• Lennart Regebro: Why? Why not just import the Python file and call the functions?
• Neal: @Lennart See my comment to S.Lott below

Answer 1

I want to be able to grab any block of code given the name and then execute that code ... (providing my own local/global scope objects).

A naive solution looks like this. This is based on the assumption that the functions don't all depend on global variables.

from file_that_contains_several_python_functions import *
Direction = some_value
func1()
func2()
func3()

That should do exactly what you want.

However, if all of your functions rely on global variables -- a design that calls to mind 1970's-era FORTRAN -- then you have to do something slightly more complex.

 from file_that_contains_several_python_functions import *
 Direction = some_value
 func1( globals() )
 func2( globals() )
 func3( globals() )

And you have to rewrite all of your global-using functions like this.

 def func1( context )
 globals().update( context )
 # Now you have access to all kinds of global variables

This seems ugly because it is. Functions which rely entirely on global variables are not really the best idea.

• Neal: What if func1() accesses something that is not in the default local scope though? E.g. say func1() accesses a variable called "Direction". "Direction" is not in the local or global scope. It is stored in a data structure in some class. I was going to populate the scope with the data required by the function instead of having the function have to know where that specific piece of data is.
• S.Lott: @Neal: You'd put "Direction=Someclass.somevalue" in this script to "populate the scope with the data" that "is stored in a data structure in some class".
• Neal: @S.Lott I guess another thing I was trying to do was eliminate access to anything but what is required. If I call func1() from within a class method, func1() could access the variable self, for example. I wanted to create a sort of sandboxed scope.
• S.Lott: "func1() could access the variable self"? How? "I wanted to create a sort of sandboxed scope." That's what a script is. If func1() depends on other things, compile and eval() won't change the dependencies. Those dependencies are in the body of func1().
• Falmarri: func1() could access the variable self I'm pretty sure that's not true

Answer 2

Using Python 2.6.4:

text = """
def fun1():
 print 'fun1'
def fun2():
 print 'fun2'
"""
import ast
tree = ast.parse(text)
# tree.body[0] contains FunctionDef for fun1, tree.body[1] for fun2
wrapped = ast.Interactive(body=[a.body[1]])
code = compile(wrapped, 'yourfile', 'single')
eval(code)
fun2() # prints 'fun2'

Take a look at grammar in ast doc: http://docs.python.org/library/ast.html#abstract-grammar. Top-level statement must be either Module, Interactive or Expression, so you need to wrap function def in one of those.

Answer 3

If you're using Python 2.6 or later, then the compile() function accepts AST objects in addition to source code.

>>> import ast
>>> a = ast.parse("print('hello world')")
>>> x = compile(a, "(none)", "exec")
>>> eval(x)
hello world

These modules have all been rearranged for Python 3.

• Neal: If I use ast.parse, then search for a FuncitonDef object, and pass that object to compile, I get a type error. It seems compile only works with the top-level Module ast object.
• Greg Hewgill: @Neal: you may have to use different mode flags to the compile() function, or maybe use exec instead of eval(). The Python runtime compiler is kind of finicky and requires some experimentation to get it to do what you want it to do.