Running a command line program and reading the result as it runs

Fri Aug 23 05:32:00 EDT 2013

Ian Simcock wrote:
> When I use this code I can see that the Popen works, any code between
> the Popen and the for will run straight away, but as soon as it gets to
> the for and tries to read p.stdout the code blocks until the command
> line program completes, then all of the lines are returned.
>> Does anyone know how to get the results of the program without it blocking?

I have tried your code with "ping google.com" as command and got the 
same results; apparently something buffers the output. The result is 
different when using Python 3.3: there, the lines are printed as they 
come in. This seems to indicate a bug in the Python 2.7 implementation.
There are some bug reports on bugs.python.org that may be related; see 
for example:
http://bugs.python.org/issue15532
I have been playing around a bit with the suggested approach of using 
the io library directly. I managed to get unbuffered output, but 
unfortunately the program hangs when the subprocess is done. It can't 
even be terminated with Control-C, I have to use task manager to kill 
python.exe.
Below is as far as I got; perhaps someone with more experience with 
pipes knows how to fix this.
Regards,
Gertjan.
#!/usr/bin/env python2.7
# coding: CP1252
from __future__ import print_function
import subprocess
import io, os
def main():
 i, o = os.pipe()
 piperead = io.open(i, 'rb', buffering=1)
 p = subprocess.Popen(["ping", "google.com"],
 stdout=o,
 stderr=subprocess.PIPE,
 bufsize=0,
 shell=False)
 for line in piperead:
 print(line)
if __name__ == '__main__':
 main()