Python subprocess, subshells, and redirection

Question

I want to use the magic of subshells and redirection with the python subprocess module, but it doesn't seem to work, complaining about unexpected tokens are the parenthesis. For example, the command

cat <(head tmp)

when passed to subprocess gives this

>>> subprocess.Popen("cat <(head tmp)", shell=True)
<subprocess.Popen object at 0x2b9bfef30350>
>>> /bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `cat <(head tmp)'

• S.Lott: Notice that your errors are coming from /bin/sh. I think that syntax may only with in bash. Do you want subprocess to run bash? Is that the question?
• pythonic metaphor: @S.Lott I thought that on my installation, sh was bash. Certainly, if I did man sh, the man page that came up was bash. Scott Lamb pointed out that when invoked as sh, this feature is unavailable, so in a sense, yes, I wanted it run as bash, even though I thought it already was.
• bash-o-logist: And why are you shelling out to the shell when Python can do what you want?

Answer 1

The <(head tmp) syntax is a bash feature called "process substitution". The basic/portable /bin/sh doesn't support it. (This is true even on systems where /bin/sh and /bin/bash are the same program; it doesn't allow this feature when invoked as plain /bin/sh so you won't inadvertently depend on a non-portable feature.)

>>> subprocess.Popen(["/bin/bash", "-c", "cat <(head tmp)"])
<subprocess.Popen object at 0x1004cca50>