homepage

I had this sporadic traceback in a project:
 File "test.py", line 62, in <module>
 result = do_lqs(client, str(dnvn))
 File "test.py", line 25, in do_lqs
 qualif_service_id = client.create('ti.qualif.service', {})
 File "/srv/openerp/.buildout/eggs/ERPpeek-1.4.5-py2.6.egg/erppeek.py", line 894, in wrapper
 return self.execute(obj, method, *params, **kwargs)
 File "/srv/openerp/.buildout/eggs/ERPpeek-1.4.5-py2.6.egg/erppeek.py", line 636, in execute
 res = self._execute(obj, method, *params)
 File "/srv/openerp/.buildout/eggs/ERPpeek-1.4.5-py2.6.egg/erppeek.py", line 361, in <lambda>
 wrapper = lambda s, *args: s._dispatch(name, args)
 File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
 verbose=self.__verbose
 File "/usr/lib/python2.6/xmlrpclib.py", line 1235, in request
 self.send_content(h, request_body)
 File "/usr/lib/python2.6/xmlrpclib.py", line 1349, in send_content
 connection.endheaders()
 File "/usr/lib/python2.6/httplib.py", line 908, in endheaders
 self._send_output()
 File "/usr/lib/python2.6/httplib.py", line 780, in _send_output
 self.send(msg)
 File "/usr/lib/python2.6/httplib.py", line 739, in send
 self.connect()
 File "/usr/lib/python2.6/httplib.py", line 1112, in connect
 sock = socket.create_connection((self.host, self.port), self.timeout)
 File "/usr/lib/python2.6/socket.py", line 561, in create_connection
 raise error, msg
socket.error: [Errno 4] Interrupted system call
It seems that the EINTR should be caught by the standard library in all cases: http://bugs.python.org/issue1628205
But it's not the case for the "socket.create_connection" method (verified in 3.3 and 2.7 source code).

> It seems that the EINTR should be caught by the standard library in all
cases:
> http://bugs.python.org/issue1628205
Yes, it should.
> But it's not the case for the "socket.create_connection" method (verified
in 3.3 and 2.7 source code).
It's not the case for *almost all* syscalls.
See http://bugs.python.org/issue18885 

We encountered the same problem, this is in the context of using PyQt (specifically QProcess) or twisted. They both rely on SIGCHLD for their notification framework.
I've attached a httplib EINTR patch for 2.6.4 & 2.7.3.

Here is a reproducible test case:
import threading
import signal
import os
import httplib
def killer():
 while 1:
 os.kill(os.getpid(), signal.SIGINT)
def go():
 signal.signal(signal.SIGINT, lambda x,y: None)
 thread = threading.Thread(target=killer)
 thread.start()
 while 1:
 connection = httplib.HTTPConnection("localhost:80")
 connection.connect()
 connection.close()
 
if __name__ == '__main__':
 go()
Which gives:
Traceback (most recent call last):
 File "./repro1.py", line 22, in <module>
 go()
 File "./repro1.py", line 18, in go
 connection.connect()
 File ".../lib/python2.7/httplib.py", line 757, in connect
 self.timeout, self.source_address)
 File ".../lib/python2.7/socket.py", line 571, in create_connection
 raise err
socket.error:

As said in a previous comment, we don't want to have EINTR handling
code everywhere.
The right way to do this is to handle it at the syscall level.

@tholzer, to clarify Charles-François's comment: EINTR should be handled in socket.connect() and socket.getaddrinfo() (the two syscalls called by create_connection(), AFAIR).

No problem, I've attached a patch for socket.py for Python 2.7.3.
A few notes:
getaddrinfo (and gethostbyname, etc.) are already immune to this bug, so I've just fixed the connect() call.
The socket does need to be closed after EINTR, otherwise a EINPROGRESS might get returned on subsequent connect() calls.

I've also attached a potential patch for the C module Modules/socketmodule.c inside internal_connect().
A few notes:
This seems to work both without time-out and with time-out sockets (non-blocking).
One concern would be a signal storm prolonging the operation beyond the time-out. Should we keep track of the actual time taken in this loop and check it against the 'timeout' parameter ?
Also, I don't think we can call PyErr_CheckSignals() in this context. Does this need to happen at all ?

http://bugs.python.org/issue21602
The smtplib.py also has the same problem.
The EINTR cannot be handled properly.

@neologix,
May I attach the patch file of smtplib.py for review?

@meishao
Previous comments answer your question :
http://bugs.python.org/issue20611#msg218836
http://bugs.python.org/issue20611#msg218841 

@flox
Thank you for your comment.
So we just only modify the socket.py to handle the system level call, is it right?
Please let me attach the patch file of socket.py for 2.7.2.

Oops, I missed a break statement at the end of socket_2.7.3_eintr_patch.py. I've fixed this now in the attached patch.
@meishao
Could you please also update your socket_2_7_2_patch.py and add the missing break statement ?

@tholzer
I've updated socket_2_7_2_patch.py and added the missing break statement.

And a test case for smtplib:
import threading
import signal
import os
import smtplib
def go():
 running = True
 pid = os.getpid()
 def killer():
 while running: 
 os.kill(pid, signal.SIGINT)
 signal.signal(signal.SIGINT, lambda x,y: None)
 thread = threading.Thread(target=killer)
 thread.start()
 while 1:
 try:
 smtplib.SMTP('localhost')
 except Exception, ex:
 running = False
 raise
 
if __name__ == '__main__':
 go()
Fails with:
socket.error: [Errno 4] Interrupted system call

Something like the patch i'm attaching to socketmodule.c is what I would prefer. I haven't looked at or tried tests for it yet.

The issue is just an example of the main issue #18885 which proposes to retry interrupted syscalls. I hesitate to mark it as duplicate, but it contains an interesting patch.

See also PEP 475 and Issue 23285 for the general fix in Python 3

This issue was fixed in Python 3.5 as part of the PEP 475: see issue #23618 which modified socket.socket.connect() to handle EINTR. It's not as simple as retrying connect() in a loop. You must respect the socket timeout (it's better to have a monotonic clock here, it's now always the case in Python 3.5). When connect() returns EINTR, the connection runs asynchronously, you have to call select() to wait until the connection completes or fails. Then you have to call getsockopt() to get the socket error code.
In Python 3.5, socket.socket.connect() still raises InterruptedError if the socket is non-blocking:
https://docs.python.org/dev/library/socket.html#socket.socket.connect
issue20611-connect-eintr-gps01.diff calls again connect() if connect() failed with EINTR. According to the issue #23618, it might work on some platforms, but it's not portable.
For Python 2.7 and 3.4, instead of fixing socket.socket.connect(), which requires complex code, we may only workaround the issue in create_connection(). If connect() raises OSError(EINTR), drop the socket and retry with a fresh connection in a loop until the connection completes or raises a different exception. (And do that for each address.)

i'm moving this to the more recent issue as i like the patch in that one better.