Is it possible to call exec so that it's compatible with both Python 3 and Python 2?

Question

I'm using the exec statement in some Python 2 code, and I'm trying to make that code compatible with both Python 2 and Python 3, but in Python 3, exec has changed from a statement into a function. Is it possible to write code that is compatible with both Python 2 and 3? I've read about Python 2 and Python 3 dual development, but I'm interested in specific solutions to the exec statement/function changes.

I realize that exec is generally discouraged, but I'm building an Eclipse plugin that implements live coding on top of PyDev. See the project page for more details.

• Lennart Regebro: Read the book: python3porting.com/differences.html#exec
• Antti Haapala: @LennartRegebro that source is wrong about exec
• Lennart Regebro: Huh, I'm pretty sure I tried that. Will try again.

Answer 1

Some Python porting guides get the exec wrong:

If you need to pass in the global or local dictionaries you will need to define a custom function with two different implementations, one for Python 2 and one for Python 3. As usual six includes an excellent implementation of this called exec_().

No such custom function is needed to port Python 2 code into Python 3 (*). You can do exec(code), exec(code, globs) and exec(code, globs, locs) in Python 2, and it works.

Python has always accepted Python 3 compatible "syntax" for exec for as long that exec existed. The reason for this is that Python 2 and Python 1 (?!) have a hack to stay backwards-compatible with Python 0.9.8 in which exec was a function. Now, if exec is passed a 2-tuple, it is interpreted as (code, globals) and in case of a 3-tuple, it is interpreted as (code, globals, locals). Yes, the exec_ in six is unnecessarily complicated.

Thus,

exec(source, global_vars, local_vars)

is guaranteed to work the same way in CPython 0.9.9, 1.x, 2.x, 3.x; and I have also verified that it works in Jython 2.5.2, PyPy 2.3.1 (Python 2.7.6) and IronPython 2.6.1:

Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_25
Type "help", "copyright", "credits" or "license" for more information.
>>> exec('print a', globals(), {'a':42})
42

---

*) There are subtle differences so that not all Python 3 code works in Python 2, namely

• foo = exec is valid in Python 3 but not in Python 2, and so is map(exec, ['print(a + a)', 'print(b + b)']), but I really don't know any reason why anyone would want to use these constructs in real code.

• As found out by Paul Hounshell, in Python 2, the following code will raise SyntaxError: unqualified exec is not allowed in function 'print_arg' because it contains a nested function with free variables:

  def print_arg(arg):
   def do_print():
   print(arg)
   exec('do_print()')
  

  The following construct works without exception.

  def print_arg(arg):
   def do_print():
   print(arg)
   exec 'do_print()' in {}
  

  Before Python 2.7.9, if one used exec('do_print()', {}) for the latter instead, the same SyntaxError would have been thrown; but since Python 2.7.9 the parser/compiler would allow this alternate tuple syntax too.

Again, the solution in edge cases might be to forgo the use of exec and use eval instead (eval can be used to execute bytecode that is compiled with compile in exec mode):

def print_arg(arg):
 def do_print():
 print(arg)
 eval(compile('do_print(); print("it really works")', '<string>', 'exec'))

---

I have written a more detailed answer on internals of exec, eval and compile on What's the difference between eval, exec, and compile in Python?

• Hounshell: This isn't true if exec is in a function. You get SyntaxError: unqualified exec is not allowed in function 'whatevs' it contains a nested function with free variables
• Antti Haapala: @Hounshell I added a disclaimer. However the original version of my answer states that "No such custom function is needed to port Python 2 code into Python 3". That code didn't work in Python 2 to begin with; and needs a wrapper in Python 2 anyway.
• Hounshell: But wouldn't exec 'print "a"' in {}, {} succeed in that situation in Python 2? exec('print "a"', {}, {}) throws the same exception.
• Hounshell: I'm getting different results on different machines. Not sure if it's due to OS or Python version. pastebin.com/xgCjf8MT TL;DR: Ubuntu 14.04 with Python 2.7.6 throws an error, OSX 10.11.6 with Python 2.7.10 does not.
• Hounshell: But that isn't dual-compliant with Python3. SyntaxError: invalid syntax I think the only way to do something that is compliant across all three (<2.7.9, >= 2.7.9, > 3.0) is to use eval. I tried my hand at that in a separate answer.

Answer 2

I found several options for doing this, before Antti posted his answer that Python 2 supports the Python 3 exec function syntax.

The first expression may also be a tuple of length 2 or 3. In this case, the optional parts must be omitted. The form exec(expr, globals) is equivalent to exec expr in globals, while the form exec(expr, globals, locals) is equivalent to exec expr in globals, locals. The tuple form of exec provides compatibility with Python 3, where exec is a function rather than a statement.

If you don't want to use that for some reason, here are all the other options I found.

Import Stubs

You can declare two different import stubs and import whichever one works with the current interpreter. This is based on what I saw in the PyDev source code.

Here's what you put in the main module:

try:
 from exec_python2 import exec_code #@UnusedImport
except:
 from exec_python3 import exec_code #@Reimport

Here's what you put in exec_python2.py:

def exec_code(source, global_vars, local_vars):
 exec source in global_vars, local_vars

Here's what you put in exec_python3.py:

def exec_code(source, global_vars, local_vars):
 exec(source, global_vars, local_vars)

Exec in Eval

Ned Batchelder posted a technique that wraps the exec statement in a call to eval so it won't cause a syntax error in Python 3. It's clever, but not clear.

# Exec is a statement in Py2, a function in Py3
if sys.hexversion > 0x03000000:
 def exec_function(source, filename, global_map):
 """A wrapper around exec()."""
 exec(compile(source, filename, "exec"), global_map)
else:
 # OK, this is pretty gross. In Py2, exec was a statement, but that will
 # be a syntax error if we try to put it in a Py3 file, even if it isn't
 # executed. So hide it inside an evaluated string literal instead.
 eval(compile("""\
def exec_function(source, filename, global_map):
 exec compile(source, filename, "exec") in global_map
""",
 "<exec_function>", "exec"
 ))

Six package

The six package is a compatibility library for writing code that will run under both Python 2 and Python 3. It has an exec_() function that translates to both versions. I haven't tried it.

• user395760: Six is awesome, you should give it a shot. Made the Python 3 porting I did much easier, and much less hacky (I'm looking at you, "Exec in Eval").
• Don Kirkby: I was a little hesitant about adding a project dependency, @delnan. Do you just include some extra files in your project, or does anyone who uses your project need to install six as well?
• user395760: In my case, I ported a build tool which bootstraps itself for installation, so dependencies were difficult (though there are some, they just aren't needed when bootstrapping), especially with six. So I ended up bundling it as a sub-module, which is permitted and was trivial except for six.moves (needs a one-line change, but still a change, and I didn't need it). You can see the changes in their entirety at github.com/paver/paver/pull/82 (don't be scared by the large number of changes, 90% of that is from a virtualenv bootstrap script and removing and adding bundled libraries).
• Lennart Regebro: @DonKirkby: You can either declare six a dependency in setup.py, meaning anyone who installs your module will automatically get six installed as well (assuming they use easy_install or pip or buildout, and they should). Or you can just put the six.py file in your module.

Answer 3

I needed to do this, I couldn't use six, and my version of Python doesn't support @Antti's method because I used it in a nested function with free variables. I didn't want unnecessary imports either. Here's what I came up with. This probably needs to be in the module, not in a method:

try:
 # Try Python2.
 _exec_impls = {
 0: compile('exec code', '<python2>', 'exec'),
 1: compile('exec code in _vars[0]', '<python2>', 'exec'),
 2: compile('exec code in _vars[0], _vars[1]', '<python2>', 'exec'),
 }
 def _exec(code, *vars):
 impl = _exec_impls.get(len(vars))
 if not impl:
 raise TypeError('_exec expected at most 3 arguments, got %s' % (len(vars) + 1))
 return eval(impl, { 'code': code, '_vars': vars })
except Exception as e:
 # Wrap Python 3.
 _exec = eval('exec')

Afterwards, _exec works like the Python3 version. You can either hand it a string, or run it through compile(). It won't get the globals or locals you probably want, so pass them in:

def print_arg(arg):
 def do_print():
 print(arg)
 _exec('do_print(); do_print(); do_print()', globals(), locals())
print_arg(7) # Prints '7'

Or not. I'm a StackOverflow post, not a cop.

Updates:

Why don't you just use eval()? eval() expects an expression, while exec() expects statements. If you've just got an expression it really doesn't matter what you use because all valid expressions are valid statements, but the converse is not true. Just executing a method is an expression, even if it doesn't return anything; there's an implied None returned.

This is demonstrated by trying to eval pass, which is a statement:

>>> exec('pass')
>>> eval('pass')
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "<string>", line 1
 pass
 ^
SyntaxError: unexpected EOF while parsing

• Antti Haapala: except, of course, you don't need exec to execute the code... you can use eval instead, and it wouldn't be subject to these syntax errors
• Hounshell: But eval doesn't allow multiple statements. This only uses eval to wrap up the exec. It's still exec that's running the code. Updated answer to demonstrate a case where eval would fail.
• Antti Haapala: see my answer on how to use eval.
• Don Kirkby: I just tried your do_print() example with Python 2.7 and 3.5. They both worked fine with the built-in exec() and printed 7. No special _exec() needed. What didn't work for you?
• Hounshell: Antti addressed it above. There was a change in 2.7.9, I was using 2.7.6