unexpected behaviour of `global` variables in Python

Question

prev, prev_re = '', (None) # these are globals
def find(h, p='', re=None):
 print h, p, re
 #global prev, prev_re
 if p == '' and prev == h: return prev_re
 prev, prev_re = h, re
 return re
print find ("abc")

results in this error:

 if p == '' and prev == h: return prev_re
UnboundLocalError: local variable 'prev' referenced before assignment
$ 

But if I comment the before last line of find,

prev, prev_re = '', (None)
def find(h, p='', re=None):
 print h, p, re
 #global prev, prev_re
 if p == '' and prev == h: return prev_re
 #prev, prev_re = h, re
 return re
print find ("abc")

the code runs correctly, as expected, without error.

My questions are why in the first case it does not find the global variable prev, and why in the second case it does find the global variable in the if-condition ?

edit: please help me understand the details of the structure of environments, to understand why the interpretor cannot find the variable.

• elssar: I've came across this behavior before, and from I made out I think by making an assignment to the global variables and not explicitly mark them out as globals in the function, you created new local variables, which you used before you assigned them.
• martineau: There is no if pre == '' and prev == h: return prev_re in your code.

Answer 1

Problem and solution

As MAK has mentioned, prev is treated as local because you assign value to it (after you check the value). The solution is to explicitly state these two variables as global:

prev, prev_re = '', (None) # these are globals
def find(h, p='', re=None):
 global prev, prev_re
 print h, p, re
 #global prev, prev_re # basically what you have done before commenting this
 if p == '' and prev == h: return prev_re
 prev, prev_re = h, re
 return re
print find ("abc")

Answers to your questions

why in the first case it does not find the global variable prev

It finds it, but then it finds the assignment that was not preceded by global statement. Thus the variable is treated as local. Uncomment the line with global and it will be fixed. Or use different variables for local assignments. You could also save the value as an attribute of the function (functions are objects too!).

why in the second case it does find the global variable in the if-condition

It treats it as global (because you only read it and you do not define it within the function, so it is not shadowed by local variable).

More reading

You asked for the place in the documentation when this behaviour is mentioned. I did not find any place where this is explicitly stated what the error means and why this specific case happens, but there is very good explanation of the execution model that may be sufficient:

If a name binding operation occurs anywhere within a code block, all uses of the name within the block are treated as references to the current block. This can lead to errors when a name is used within a block before it is bound. This rule is subtle. Python lacks declarations and allows name binding operations to occur anywhere within a code block. The local variables of a code block can be determined by scanning the entire text of the block for name binding operations.

If the global statement occurs within a block, all uses of the name specified in the statement refer to the binding of that name in the top-level namespace. Names are resolved in the top-level namespace by searching the global namespace, i.e. the namespace of the module containing the code block, and the builtins namespace, the namespace of the module __builtin__. The global namespace is searched first. If the name is not found there, the builtins namespace is searched. The global statement must precede all uses of the name.

So basically:

• if the assignments occurs within specific function, the variable is treated as local, unless...
• if the uses of the variable are preceded by global statement, the variable is treated as local,
• if there is no assignment, the variable is first looked up in the local namespace, then global one, then it is searched for in __builtin__ module,

Please also note that Python 3 has nonlocal statement that tells interpreter that the variable is from the outer scope (not the current one, but not necessarily the global one).

• alinsoar: I think I understand -- if an assignment takes place to a variable, the compiler will treat the variable as local... Can you show me the place in the Python's docs where this fact is explained, please ?
• Tadeck: @alinsoar: Done. Please see the updated version of the answer plus short summary
• martineau: @Tadeck: See 4.1 Naming and binding in the docs.
• martineau: @Tadeck: It's crucial to notice the anywhere in "occurs anywhere within a code block" because it means it has nothing to do with the order that the statements occur or are "executed" in.
• alinsoar: Thank you all . The problem is generated by the parser, as I finally found in the documentation . The same problem does not take place if you build the function as a string and call 'eval' . It was nice to receive such a quick response .

Answer 2

You did not declare prev as global. If you only ever do read actions on the variable, python will trybto treat the variable as global. However, because you do an assignment, python detects this and makes prev a local variable.

See http://docs.python.org/2/reference/simple_stmts.html#assignment-statements for more information. Specifically, the part that states the following:

Assignment of an object to a single target is recursively defined as follows.

If the target is an identifier (name):

• If the name does not occur in a global statement in the current code block: the name is bound to the object in the current local namespace.
• Otherwise: the name is bound to the object in the current global namespace.

If you intend for a variable name to reference a global, you should declare it as global with the global statement.

• alinsoar: can you show me in python's docs where this is written, please?

Answer 3

The global statement makes a global variable available in the scope of a function. If you don't declare something global, Python has no way to know that you are referring to a global vs. creating a new local.

The error you're getting just means that Python has recognized you're using a variable before setting it; you'd get the same error even if the global variables weren't defined at all.

• alinsoar: Thanks. I saw in docs: Programmer’s note: the global is a directive to the parser.

Answer 4

IINM, the Python interpreter detects that the variable is assigned to later in the function, and so treats it as a local.

A workaround would be to use a class and have prev and prev_re as instance variables of the class.

• alinsoar: That was expectedd, but I want details about the allocations of environments.
• alinsoar: What means to assign later ? I came from the direction of c/lisp, and never heard about this concept.
• Alex L: @alinsoar It means that you put prev_re = re after return prev_re, so Python thought that you were trying to use the local variable prev_re before you declared it. (see Tadeck's answer for more info)