I'm having problems implementing a basic HTTPServer in Python 3.6 on windows 10 Pro x64.
Essentially the selector being used in the base implementation of socketserver.serve_forever() uses a selector to determine if a socket is readable. The issue is that after a single GET request is received by the server it always evaluates true. So lets take the scenario where a single GET request is received. The GET is handled, the socket is flushed, but selector.select returns true still so the server tries to read the empty socket, causing it hang and block.
socketserver.serveforever:
def serve_forever(self, poll_interval=0.5):
"""Handle one request at a time until shutdown.
Polls for shutdown every poll_interval seconds. Ignores
self.timeout. If you need to do periodic tasks, do them in
another thread.
"""
self.__is_shut_down.clear()
try:
# XXX: Consider using another file descriptor or connecting to the
# socket to wake this up instead of polling. Polling reduces our
# responsiveness to a shutdown request and wastes cpu at all other
# times.
with _ServerSelector() as selector:
selector.register(self, selectors.EVENT_READ)
while not self.__shutdown_request:
ready = selector.select(poll_interval)
if ready:
self._handle_request_noblock()
self.service_actions()
finally:
self.__shutdown_request = False
self.__is_shut_down.set()
selector.select:
if sys.platform == 'win32':
def _select(self, r, w, _, timeout=None):
r, w, x = select.select(r, w, w, timeout)
return r, w + x, []
else:
_select = select.select
def select(self, timeout=None):
timeout = None if timeout is None else max(timeout, 0)
ready = []
try:
r, w, _ = self._select(self._readers, self._writers, [], timeout)
except InterruptedError:
return ready
r = set(r)
w = set(w)
for fd in r | w:
events = 0
if fd in r:
events |= EVENT_READ
if fd in w:
events |= EVENT_WRITE
key = self._key_from_fd(fd)
if key:
ready.append((key, events & key.events))
return ready
This is the line in selector that seems to be the issue. It perpetually returns a value for r once a GET request is received once:
r, w, x = select.select(r, w, w, timeout).
** EDIT1 A basic implimentatin that fails
MyServer.py
import http
from http import server
class Server(server.HTTPServer):
pass
MyRequestHandler.py
from http.server import BaseHTTPRequestHandler
import urllib
class OAuthGrantRequestHandler(BaseHTTPRequestHandler):
"""docstring"""
def do_GET(self):
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_headers()
parts = urllib.parse.urlparse(self.path)
self.wfile.write(
b'<html><head><title>Authentication Status</title></head>'
b'<body><p>The authentication flow has completed.</p>')
print("Request handler completed")
return
MyTest.py
import MyServer
import MyRequestHandler
import threading
def AwaitCallback(server_class=MyServer.Server,
handler_class=MyRequestHandler.OAuthGrantRequestHandler):
"""docstring"""
server_address = ("127.0.0.1", 8080)
Httpd = server_class(server_address, handler_class)
Httpd.timeout = 200
t1 = threading.Thread(target=Httpd.serve_forever, args=(1,))
try:
t1.start()
finally:
if t1:
t1.join()
print("thread 3 terminated")
if Httpd:
Httpd.server_close()
return
AwaitCallback()
After running my test I just drop localhost:8080 into my favorite browser and wallah, the server responds successfully then locks up.
2 Answers 2
Just in case someone else falls in this question and has the same problem with HTTPServer, let's mention the new update from The Python Standard Library since version 3.7:
class http.server.ThreadingHTTPServer(server_address, RequestHandlerClass)
This class is identical to HTTPServer but uses threads to handle requests by using the ThreadingMixIn. This is useful to handle web browsers pre-opening sockets, on which HTTPServer would wait indefinitely.
New in version 3.7.
Comments
on selector.select change
def _select(self, r, w, _, timeout=None):
into
def _select(self, r, w, x, timeout=None):
Comments
Explore related questions
See similar questions with these tags.
print (r)right after select returns?select.selectmethod documentation says it calls the select method of the OS.selectexample given here steelkiwi.com/blog/working-tcp-sockets . Where do you actually read the data, i.e.r.recv()?