No 'print' output when using yield?

Question

I am implementing a web scraper using the scrapy framework. In order to implement pipeline, yield is necessary when parsing the response from the scraper. It seems to me that when using yield at the end of a function, all output from print statements is suppressed and replaced by the generator object.

def testFunc(arg1):
 print arg1
 yield arg1
testFunc('This does not print.')

Results in:

In [7]: testFunc('Will this print?')
Out[7]: <generator object testFunc at 0x10d636d20>

Simply commenting the yield restores the print call:

def testFunc(arg1):
 print arg1

Results in:

In [10]: testFunc('Will this print?')
Will this print?

How do I maintain print output when using yield?

• csunday95: Possible duplicate of What does the "yield" keyword do?
• ph0t0n: Despite all my reading on the "yield" keyword, and understanding that what was actually yield-ed is a generator that requires iteration, it wasn't clear that everything in the body was held for execution until iteration occurred. @Cubemaster describes as a "byproduct" and that makes sense. Perhaps this specific question will help others by singling out this particular byproduct since it is readily observable when working with yield.

Answer 1

The print statements will be executed whenever the generator is iterated over. If you just want to see what the iterator prints, you can call it in a list comprehension without saving the result.

def testFunc(arg1):
 print arg1
 yield arg1
>>> [_ for _ in testFunc(1)]
1
[1]

• Cubemaster: You beat me to it!
• Aran-Fey: Or just list(testFunc(1)).

Answer 2

Calling a generator function as in testFunc(1) simply creates a generator instance; it does not run the body of the code. Generators are iterators, so you can pass them to next(). In the case of Generators, the action of its __next__() is essentially to run up to the next yield statement and return the yielded value. So you can do things like:

>>> gen = testFunc(1)
>>> next(gen)
Will this print?
1

or, as others have noted, you can loop over it (though this is not necessary if you want to just yield one value).

When you define a generator, you can almost think of calling a generator as creating an instance of some class that implements a very specific state machine that works as an iterator. To be clear, that's not actually how it works, but it can be written equivalently that way. Generators are a much more elegant way to do this in most cases, however.

Answer 3

I don't think anyone gave the answer to "How do I maintain print output when using yield?" from the perspective of the programmer writing the function.

Here it is:

def testFunc(arg1):
 print(arg1)
 def foo():
 yield arg1
 return foo()

• Krishna: Why does it have to so complex?

Answer 4

@bphi beat me to the answer, but here is some elaboration on why his solution works:

Any time a yield is included in a function, the rest of the code in that body will not run until the generator created by the yield is used. This is just a byproduct of how yield works in python. If you ever want to execute code in the same function as you have a yield, just stick it in an empty for loop, like bphi demonstrated