Python - communicate with subprocess

Question

I'm running Python 2.7 on a Win32 OS, but i'm hoping to write platform-independent code. I'm trying to use Python to interact in real-time with a Java program i wrote, and figured pipes would be the best way to do this. My Python script is calling Java as a subprocess. Essentially, Java is the GUI and Python is the back end. (I don't want to use Jython or wxPython because i only want to be dependent upon the standard libraries each language provides.) Trying to set up communication between the two has been terrible. I can send a message from the (parent) Python script to the (child) Java class using

process.stdin.write(msg)
process.stdin.flush()

but reading Java's output has not worked. I use

process.stdout.read()

but apparently this blocks forever if there's nothing to read. And process.communicate() is off limits because it doesn't do anything until the subprocess terminates. According to my research, a common method people use to get around this problem is to "use threads" (although someone suggested appending a newline when writing -- didn't work), but being new to Python and threading in general i have no idea how that would look. I've tried looking over the standard library's subprocess.py source but that hasn't helped. Is there a way to see if stdout is empty, at least? If not, how do i accomplish this?

• Voo: In general you have to drain all streams in parallel because otherwise you run into problems (say you're waiting on stdout, but the process has written the buffer of stderr full and is waiting that someone reads from it). There's no timeout for read I'm aware of, so that won't help. What exactly is your problem? Blocking until you get the expected output from java seems like the usual wanted behavior anyhow.
• nuju: I'm fine with blocking; in fact, that's exactly what i want. I want Python to block until Java says something. My problem is that process.communicate() blocks until the program terminates, and process.stdout.read() blocks forever. As soon as Java says something, i want Python to know, but i can't figure out how to do that.

Answer 1

process.stdout.read()
but apparently this blocks forever if there's nothing to read.

well not exactly, it will basically block while its either reading/waiting until it hits EOF which is set when the file closes, one way to circumvent this is by stating how many bytes you want to read process.stdout.read(1) this will read 1 byte and return if theres no byte then again it will wait until theres at least one byte or EOF.

You may also use python select module which has an optional timeout period where select waits for this long or simply returns with empty values http://docs.python.org/library/select.html
though it may not be fully supported on windows.

(although someone suggested appending a newline when writing -- didn't work)

I've actually done this though from/to python, coupled with process.stdout.readline().rstrip() so data is a set of line(s) though you still have to strip them, due note you may have to flush in order for both processes to register the data.

I did find this java: how to both read and write to & from process thru pipe (stdin/stdout) which may help you.

good luck.

• nuju: Specifying the number of bytes to read did the trick. Thank you!