select doesn't work for pipe in python?

Question

When using check_parent_select, after the reader side is closed, exception list is not filled.

But using check_parent_poll, after the reader side is closed, it can detects the pipe disconnection.

Does someone know the root cause?

#!/usr/bin/python2.7
import select
import sys
import os
log=open("./test.log","w")
(reader, writer) = os.pipe()
def check_parent_select(fh):
 (rlist, wlist, xlist) = select.select([], [], [fh], 1)
 if fh in xlist:
 print "parent exit"
 else:
 print "parent OK"
def check_parent_poll(fh):
 poller = select.poll()
 EVENTS = select.POLLERR
 poller.register(fh)
 events = poller.poll()
 for fd, flag in events:
 if flag & select.POLLERR:
 print "parent exit"
 else:
 print "parent OK"
open_file = os.fdopen(writer, "w")
check_parent_select(open_file)
os.close(reader)
check_parent_select(open_file)

Used strace to trace select function, select can't detect the pipe close.

pipe([4, 5]) = 0

select(6, [], [], [5], {1, 0}) = 0 (Timeout)

write(1, "parent OK\n", 10parent OK

close(4) = 0

select(6, [], [], [5], {1, 0}) = 0 (Timeout)

Answer 1

It is somewhat hidden, but if you follow the documentation, it becomes clearer: select() checks for pending error conditions, i.e. error conditions that make the file descriptor unusable, but only after the error has occurred.

After closing the read end, you haven't done any operation on the pipe yet, that causes an error condition. There are still valid operations for the writer: For example, you can close the fd. The pipe thus isn't yet in an error state.

The problem is more easily discerned when closing the writer side: Even after the close there could be readable data in the pipe's buffer, that hasn't been consumed yet. In such cases, you want read() to return 0 on EOF, not -1 for error. The other side behaves similarly, even though you really cannot write to a pipe whose read end is already closed.

The behavior is the same with socket.socketpair() (or actual sockets): As long as haven't done anything invalid yet, there's no error condition.

log=open("./test.log","w") 
(reader, writer) = socket.socketpair() 
def check_parent_select(fh): 
 (rlist, wlist, xlist) = select.select([], [], [fh], 1) 
 if fh in xlist: 
 print "parent exit" 
 else: 
 print "parent OK" 
def check_parent_poll(fh): 
 poller = select.poll() 
 EVENTS = select.POLLERR 
 poller.register(fh) 
 events = poller.poll() 
 for fd, flag in events: 
 if flag & select.POLLERR: 
 print "parent exit" 
 else: 
 print "parent OK" 
check_parent_select(writer) 
reader.close() 
check_parent_select(writer)

Answer 2

A quick fix for mitigating this unexpected result is to listen for read events in the writer. A better fix would be to use pselect() and listen for SIG_PIPE as well but afaik python doesn't have a pselect()

I call it "unexpected" because the first thing which comes to your mind is that the closing of the read end will be signaled as an exception for the writer. From the point of view of the writer it may be indeed an exceptional case as long as it still has something to write. But, from the point of view of the OS, this is just a simple close() of a file descriptor.

If you read the manpage for poll() system call you will find that the closing of a file descriptor will be signaled by marking the POLLHUP bit in the read events list. select() has the same behavior only that it doesn't have specific bits to set for identifying the close() call.

#!/usr/bin/python2.7
import select
import sys
import os
import time
log=open("./test.log","w")
(reader, writer) = os.pipe()
def check_parent_select(fh):
 (rlist, wlist, xlist) = select.select([fh], [fh], [fh], 1)
 print(rlist, wlist, xlist)
 if fh in rlist:
 print "oh i'm just writing. error" 
 if fh in xlist:
 print "parent exit"
 else:
 print "parent OK"
def check_parent_poll(fh):
 poller = select.poll()
 EVENTS = select.POLLERR
 poller.register(fh)
 events = poller.poll()
 for fd, flag in events:
 if flag & select.POLLERR:
 print "parent exit"
 else:
 print "parent OK"
#open_file = os.fdopen(writer, "w")
check_parent_select(writer)
os.close(reader)
#time.sleep(3)
check_parent_select(writer)

so when the pipe is closed you'll get a read event in the writer:

 python2 t1.py
([], [5], [])
parent OK
([5], [5], [])
oh i'm just writing. error
parent OK

• dhke: Is this by experience or can you point to the specification for this? Because it seems somehow weird that closing the other end triggers a read event.
• baldrunner2049: closing the read end of the pipe is NOT an error hence there will be no notifications for you waiting in the "x" list. for example, with poll system call, the closing of an end of a pipe is signaled by returning POLLHUP in read events list.