Search code examples
pythonmultithreadingsocketssocketserver

Why does putting a socket in a queue close it?


I'm writing a server that operates with a fixed number of workers, each with different properties (in the snippet below, n is such a property.

Upon getting a request, I would like to put it into a queue, so the first available worker can deal with the task.

Unfortunately, the socket gets closed when it's enqueued.

import threading
from queue import Queue
import socketserver

thread = True

queue = Queue()


class BasicHandler(socketserver.BaseRequestHandler):
    def handle(self):
        while True:
            sock = self.request
            byte = sock.recv(10)
            print(byte)

class ThreadedHandler(socketserver.BaseRequestHandler):
    def handle(self):
        queue.put(self.request)

def worker(n):
    print('Started worker ' + str(n))
    while True:
        sock = queue.get()
        byte = sock.recv(10)
        print(byte)

if thread:
    [threading.Thread(target=worker, args=(n,)).start() for n in range(2)]
    handler = ThreadedHandler
else:
    handler = BasicHandler

socketserver.TCPServer.allow_reuse_address = True
server = socketserver.TCPServer(("localhost", 9999), handler)
server.serve_forever()

Running the above snippet with thread = False works as fine, but when I try to connect to the thread = True version, telnet immediately says:

Connection closed by foreign host.

and the server prints:

Started worker 0                                                                                                                               
Started worker 1                                                                                                                               
b''                                                                                                                                            

Solution

  • The request is automatically closed, when the method ThreadedHandler.handler finished. You have to override TCPServer.shutdown_request if you want to keep the socket open.