Controlling a python script from another script

Question

I am trying to learn how to write a script control.py, that runs another script test.py in a loop for a certain number of times, in each run, reads its output and halts it if some predefined output is printed (e.g. the text 'stop now'), and the loop continues its iteration (once test.py has finished, either on its own, or by force). So something along the lines:

for i in range(n):
 os.system('test.py someargument')
 if output == 'stop now': #stop the current test.py process and continue with next iteration
 #output here is supposed to contain what test.py prints

• The problem with the above is that, it does not check the output of test.py as it is running, instead it waits until test.py process is finished on its own, right?
• Basically trying to learn how I can use a python script to control another one, as it is running. (e.g. having access to what it prints and so on).
• Finally, is it possible to run test.py in a new terminal (i.e. not in control.py's terminal) and still achieve the above goals?

---

An attempt: test.py is this:

from itertools import permutations
import random as random
perms = [''.join(p) for p in permutations('stop')]
for i in range(1000000):
 rand_ind = random.randrange(0,len(perms))
 print perms[rand_ind]

And control.py is this: (following Marc's suggestion)

import subprocess
command = ["python", "test.py"]
n = 10
for i in range(n):
 p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
 while True:
 output = p.stdout.readline().strip()
 print output
 #if output == '' and p.poll() is not None:
 # break
 if output == 'stop':
 print 'sucess'
 p.kill()
 break
 #Do whatever you want
 #rc = p.poll() #Exit Code

• jmetz: os.system is deprecated; you should be using the subprocess module
• jmetz: to run test.py in a new terminal, you need to start that process, e.g. subprocess.call(["xterm", "-e", "python", "test.py", "someargument"])
• zondo: You might be interested in pexpect.
• user6039682: @zondo neat! would you be so kind to showcase an example here? using pexpect
• zondo: Here

Answer 1

You can use subprocess module or also the os.popen

os.popen(command[, mode[, bufsize]])

Open a pipe to or from command. The return value is an open file object connected to the pipe, which can be read or written depending on whether mode is 'r' (default) or 'w'.

With subprocess I would suggest

subprocess.call(['python.exe', command])

or the subprocess.Popen --> that is similar to os.popen (for instance)

With popen you can read the connected object/file and check whether "Stop now" is there.

The os.system is not deprecated and you can use as well (but you won't get a object from that), you can just check if return at the end of execution.

From subprocess.call you can run it in a new terminal or if you want to call multiple times ONLY the test.py --> than you can put your script in a def main() and run the main as much as you want till the "Stop now" is generated.

Hope this solve your query :-) otherwise comment again.

Looking at what you wrote above you can also redirect the output to a file directly from the OS call --> os.system(test.py *args>> /tmp/mickey.txt) then you can check at each round the file.

As said the popen is an object file that you can access.

• user6039682: Thanks Marco, I'm going to try a simple script and try this idea, I'll get back to you
• Marco smdm: Exactly what you are looking are threads. This is what you need in the end :-) What I suggested above is a bit more easier, but threads are the correct answer I guess :-) Have a nice day.
• user6039682: I added a first attempt to my post. it can do some of the things I wanted. Remains to call the subprocess in a new terminal (on ubuntu), any ideas?
• Marco smdm: Not sure if the creation flag will help you in the Popen for this particular case. creationflags=CREATE_NEW_CONSOLE . You can also check the subprocess doc here: docs.python.org/2/library/subprocess.html#subprocess.Popen hope this is helping you :-)
• user6039682: CREATE_NEW_CONSOLE is only for windows I think, I'm running ubuntu

Answer 2

What you are hinting at in your comment to Marc Cabos' answer is Threading

There are several ways Python can use the functionality of other files. If the content of test.py can be encapsulated in a function or class, then you can import the relevant parts into your program, giving you greater access to the runnings of that code.

As described in other answers you can use the stdout of a script, running it in a subprocess. This could give you separate terminal outputs as you require.

However if you want to run the test.py concurrently and access variables as they are changed then you need to consider threading.

• user6039682: oh wow I'm learning so much. Indeed in this specific example I'm solely attempting to access the print outputs of test.py here.
• user6039682: I added a failed attempt to my post, if you want to have a look

Answer 3

Yes you can use Python to control another program using stdin/stdout, but when using another process output often there is a problem of buffering, in other words the other process doesn't really output anything until it's done.

There are even cases in which the output is buffered or not depending on if the program is started from a terminal or not.

If you are the author of both programs then probably is better using another interprocess channel where the flushing is explicitly controlled by the code, like sockets.

• user6039682: oh :(( bummer... do you see a way around this? (because I only care about what test.py prints and not the other things that it does, so that's why I thought if I run it in another terminal, then maybe I have a chance of reading the terminal itself to learn what test.py is printing)

Answer 4

You can use the "subprocess" library for that.

import subprocess
command = ["python", "test.py", "someargument"]
for i in range(n):
 p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
 while True:
 output = p.stdout.readline()
 if output == '' and p.poll() is not None:
 break
 if output == 'stop now':
 #Do whatever you want
 rc = p.poll() #Exit Code

• user6039682: Hi, thanks for the suggestion. If I'm not mistaken, with this, answer, err are filled only once p has finished, right? I sort of want to read the subprocess' prints as it is running, and then stop that subprocess if among the prints, 'stop now' is printed. Is that achievable?
• Marc Cabos: Test if it works correctly... I can't check it now (just coded).
• user6039682: Thanks a lot, gonna try it now. I guess you meant to write the break after the second condition, right?
• user6039682: If I have multiple subprocesses, so e.g. p, p2, p3, how can I make the while loop you suggested, but specifically for process p, I ask because you simply wrote while True: but now with multiple at hand, I want to do the same loop except for process p, so something like while p==True: but that doesn't work