/usr/bin/which has strange behavior when called as python subprocess

Question

I have a python (2.7) package which depends on a library written in C (The brain imaging suite Freesurfer). In order to check for the library I wrote the following:

def crash_if_freesurfer_not_found():
 import os, subprocess
 with open(os.devnull) as devnull:
 p = subprocess.call(['which', 'mri_info'], stdout=devnull, stderr=devnull)
 if p!=0:
 print 'Useful error message'
 sys.exit(1)

If which finds the program mri_info in the path, then it has return code 0, otherwise it has return code 1. On the systems I developed this on, this worked.

I am on a system now, where this code is failing. But I was quite confused why, because `os.environ['PATH'] includes ~/freesurfer/bin, where this program is located.

In [3]: os.environ['PATH'].split(':')[0]
Out[3]: '/home/aestrivex/freesurfer/freesurfer/bin'
In [11]: ls /home/aestrivex/freesurfer/freesurfer/bin | grep mri_info
 mri_info*

So I dug deeper and I found this odd behavior which I don't understand:

In [10]: with open(os.devnull) as nil:
 p = subprocess.call(['which', 'mri_info'], stdout=nil, stderr=nil)
 ....: 
In [11]: p
Out[11]: 1
In [12]: with open(os.devnull) as nil:
 p = subprocess.call(['which', 'mri_info'], stdout=nil)
 ....: 
 sh: printf: I/O error
In [13]: with open(os.devnull) as nil:
 p = subprocess.call(['which', 'mri_info'])
 ....: 
 /home/aestrivex/freesurfer/freesurfer/bin/mri_info
aestrivex@apocrypha ~/gselu $ which which
 /usr/bin/which

So, which fails with an I/O error whenever the python subprocess redirects stdout to /dev/null, and otherwise operates normally.

But I get perfectly normal behavior from which otherwise than in a python subprocess

aestrivex@apocrypha ~/gselu $ which mri_info
/home/aestrivex/freesurfer/freesurfer/bin/mri_info
aestrivex@apocrypha ~/gselu $ which mri_info > /dev/null
aestrivex@apocrypha ~/gselu $ echo $?
0

There are several ways for me to fix this check, but my question is, what possible context could cause this bug to see which behave like this in a python subprocess?

• mgilson: Can you execute other commands from python and write it's output to a file opened via open(os.devnull)? Also, it seems weird to expect anything written to that "file" since you've opened it in the default mode ('r') -- What happens if you open it for 'w'?
• aestrivex: Thanks, it works with 'w' passed to open. Which I should have caught, but I am confused by why it works without that additional flag on some other machines. I guess there is some magic somewhere which detects in some shell environments that you're trying to write to /dev/null, and doesn't, but for whatever reason in this case it actually wrote to /dev/null.
• mgilson: When dealing with pseudo-files, who knows what checks are actually implemented by the OS? I'll post as an answer.
• ivan_pozdeev: Heh, this means, my answer was technically correct, too, yet it missed the precise point.
• jfs: related: How to hide output of subprocess in Python 2.7

Answer 1

You are opening the "file" (/dev/null) in the default mode (which is 'r' [read]). However, subprocess is trying to write to the file -- So you probably want:

with open(os.devnull, 'w') as nil:
 p = subprocess.call(['which', 'program'], stdout=nil, stderr=nil)