using mpirun -np in python script

Question

I want to run a pw.x in bash with this command: mpirun -np 4 pw.x < input.in through a python script. I used this:

from subprocess import Popen, PIPE
process = Popen( "mpirun -np 4 pw.x", shell=False, universal_newlines=True,
 stdin=PIPE, stdout=PIPE, stderr=PIPE )
output, error = process.communicate();
print (output);

but it gives me this error:

Original exception was:
Traceback (most recent call last):
 File "test.py", line 6, in <module>
 stdin=PIPE, stdout=PIPE, stderr=PIPE )
 File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
 restore_signals, start_new_session)
 File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
 raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'mpirun -np 4 pw.x': 'mpirun -np 4 pw.x'

How can I use "mpirun -np ..." in python scripts?

• Pavel: docs.python.org/2/library/subprocess.html#popen-constructor: split your command and pass it as a list to Popen

Answer 1

When you have shell=False in the Popen constructor, it expects the cmd to be a sequence; any type of str could be one but then the string is treated as the single element of the sequence -- this is happening in your case and the whole mpirun -np 4 pw.x string is being treated as the executable filename.

To solve this, you can:

• Use shell=True and keep everything else as-is, but beware of the security issues as this would be run directly in shell and you should not do this for any untrusted executable

• Use a proper sequence e.g. list for cmd in Popen:

  import shlex
  process = Popen(shlex.split("mpirun -np 4 pw.x"), shell=False, ...)
  

Both assuming mpirun exists in your PATH.

Answer 2

How about changing

shell=False

to

shell=True

Answer 3

With shell=False, you need to parse the command line into a list yourself.

Also, unless subprocess.run() is unsuitable for your needs, you should probably avoid calling subprocess.Popen() directly.

inp = open('input.in')
process = subprocess.run(['mpirun', '-np', '4', 'pw.x'],
 # Notice also the stdin= argument
 stdin=inp, stdout=PIPE, stderr=PIPE,
 shell=False, universal_newlines=True)
inp.close()
print(process.stdout)

If you are stuck on an older Python version, maybe try subprocess.check_output()

• tripleee: See also stackoverflow.com/questions/3172470/… for why you want to avoid shell=True
• amirre: Thanks for your answer, but It doesn't work for me and doesn't show anything in output.