How to compile a function at run-time?

Question

Is there a way in Python 3.x using the standard library to compile a function at run-time and use it through a function object?

I tried the following:

class A: 
 def meth1(self, par1):
 print("in A.meth1: par1 =", par1)
a = A()
s = '''def meth1(self, par1):
 print("in recompiled A.meth1: par1 = ", par1)'''
a.meth1.__func__.__code__ = compile( s, __file__, "exec")
a.meth1("via meth1")

But this gives an error:

TypeError: <module>() takes 0 positional arguments but 2 were given

In the docs for compile() it's writen that the code compiled with it can be run with eval() or exec(). Is there a way to compile a function and call it through a function object without eval() and exec()?

• Roland Smith: I wonder what the problem is that requires self-modifying code as a solution?
• Al Berger: @Roland I'm trying to use some info which is known only at run-time for optimization (speed enhancement).
• Roland Smith: Maybe you should post a question how to deal with this but with more details. There might be a better way. I'm assuming you've profiled the code to show where the program is spending most of its time?

Answer 1

You probably have to run it through exec() or eval() just to execute the function definition. You can pass it a separate environment from which you have to extract the function yourself (as user HelloWorld correctly said, Python cannot "guess" which function you meant).

Back to your example, here I first create an empty environment environment, then pass the compiled code and the dictionary to exec in order to evaluate the definition. With the MethodType class from the types library, one has to convert the function into a bound method, providing the class or instance as second parameter. Then you can attach the bound method to the original instance.

import types
class A: 
 def meth1(self, par1):
 print("in A.meth1: par1 =", par1)
a = A()
s = '''def meth1(self, par1):
 print("in recompiled A.meth1: par1 = ", par1)'''
code = compile(s, __file__, "exec")
environment = {}
exec(code, environment)
a.meth1 = types.MethodType(environment["meth1"], A)
a.meth1("via meth1")

Answer 2

jojonas gave a good answer and this post is more an additional comment than an answer.

You can't simply compile a string and replace the code object from a class. What you get after you called compile is a code-object. E.g. take a look at the following code.

obj = '''
def meth1(self, par1):
 print("in recompiled A.meth1: par1 = ", par1)
def meth2(self, par1):
 print("in recompiled A.meth2: par1 = ", par1)
'''
a.meth1.__func__.__code__ = compile(obj, __file__, "exec") # What should happen here?

As you can see from this example, we compiled an entire scope that can't be simply attached to a class. What you have to do is to extract the meth1 function from the code object and use that instead.

You should take a look at the following nice article about code objects what gives you some nice insights.

http://late.am/post/2012/03/26/exploring-python-code-objects.html

• Al Berger: Thanks for the link, I'll check it.

Answer 3

For what's worth, I recently created a @compile_fun goodie that considerably eases the process of applying compile on a function. Your example writes:

class A:
 @compile_fun
 def meth1(self, par1):
 print("in A.meth1: par1 =", par1)

You can see that you now can't debug into meth1 with your IDE. Note that this does not improve runtime performance, nor protects your code from reverse-engineering, but it might be convenient if you do not want your users to see the internals of your function when they debug. Note that the obvious drawback is that they will not be able to help you debug your lib, so use with care!

See makefundocumentation for details.