How python interpret a function as a generator

Question

I know that if a "yield" statement is present in a function it will be treated as a generator. But how does python interpreter works in that case when the function(generator) called for the first time.I mean when the code is interpreted i think first "gen" will get bind to a function object. Why it doesn't execute the statements before yield unlike normal functions or classes when called for first time. How does it put a hold on the execution of the print function before yield

>>> def gen():
... print("In gen")
... yield 1
... yield 2
... 
>>> type(gen)
<type 'function'>
>>> a = gen
>>> type(a)
<type 'function'>
>>> 
>>> b = a()
>>> type(b)
<type 'generator'>
>>> b.next()
In gen
1
>>> b.next()
2
>>> 

• Lukas Graf: Because Python generators are, by definition, lazy. They only start yielding output once you start consuming them (i.e. iterate over them).
• bluefoggy: I mean before going to yield , why it doesn't execute the statements before yield. How does the interpreter know about it.
• Lukas Graf: In addition to @unutbu's answer: Because Python treats a function body with a yield statement differently. Once you put a yield in there, it knows you'll be defining a generator, not a regular function. Try putting yield and return (with an argument) in the same function, Python won't let you compile that code, even if you never use that function.
• bluefoggy: @Lukas : Cool, thanks. A little more explanation on def executable will help.
• Lukas Graf: See The yield statement: "Using a yield statement in a function definition is sufficient to cause that definition to create a generator function instead of a normal function." How exactly the Python bytecode compilation process works is highly involved and shouldn't have to concern you.

Answer 1

When Python parses the def statement, it decides if the code is defining a generator or a function. If the code contains a yield expression, then it is a generator. So when the generator function a is called, it returns a generator object, b.

The code inside the def statement is not executed until b.next() is called.

---

Consider these two functions:

def gen():
 print('In gen')
 yield 1
 yield 2
def func():
 print('In gen') 

Although gen and func are both functions, Python knows that gen is a generator:

In [96]: import inspect
In [101]: inspect.isgeneratorfunction(gen)
Out[101]: True
In [102]: inspect.isgeneratorfunction(func)
Out[102]: False

If you look inside the inspect module you see

def isgeneratorfunction(object):
 """Return true if the object is a user-defined generator function.
 Generator function objects provides same attributes as functions.
 See help(isfunction) for attributes listing."""
 return bool((isfunction(object) or ismethod(object)) and
 object.func_code.co_flags & CO_GENERATOR)

So apparently Python checks this first before executing the code inside the function.

• bluefoggy: How does the def statement does it without reaching the control to the block inside the function. How does it find it?
• bluefoggy: I don't how how def works, so a little more explanation will help. I thought it just creates a function object and bind a name to it.
• mtik00: All of the code gets compiled. The compiler inspects the contents of the function definition, finds a yield statement, and determines this object is, in fact, a generator.
• mtik00: @kingsdeb You can read some documentation here.
• unutbu: @kingdeb: Although dis.dis(gen) shows no indication that gen is a generator function, gen.func_code.co_flags has a flag indicating that gen is a generator. So the def statement is creating a function object that knows its a generator. Does this answer your question?