Search code examples
pythonservermultiprocessingshutdown

Stopping a server in a subprocess with its shutdown method


I am implementing a Server class in CPython 3.7 on Windows 10 with a Server.serve method that starts the serving forever and a Server.shutdown method that stops the serving. I need to run multiple server instances in subprocesses.

Running a server instance in a subthread stops the instance as expected:

import threading
import time


class Server:

    def __init__(self):
        self.shutdown_request = False

    def serve(self):
        print("serving")

        while not self.shutdown_request:
            print("hello")
            time.sleep(1)

        print("done")

    def shutdown(self):
        print("stopping")
        self.shutdown_request = True


if __name__ == "__main__":
    server = Server()
    threading.Thread(target=server.serve).start()
    time.sleep(5)
    server.shutdown()

However running a server instance in a subprocess does not stop the instance, unexpectedly:

import multiprocessing
import time


class Server:

    def __init__(self):
        self.shutdown_request = False

    def serve(self):
        print("serving")

        while not self.shutdown_request:
            print("hello")
            time.sleep(1)

        print("done")

    def shutdown(self):
        print("stopping")
        self.shutdown_request = True


if __name__ == "__main__":
    server = Server()
    multiprocessing.Process(target=server.serve).start()
    time.sleep(5)
    server.shutdown()

I suspect that in the multiprocessing case, the self.shutdown_request attribute is not shared between the parent process and the subprocess, and therefore the server.shutdown() call does not affect the running server instance in the subprocess.

I know I could solve this with multiprocessing.Event:

import multiprocessing
import time


class Server:

    def __init__(self, shutdown_event):
        self.shutdown_event = shutdown_event

    def serve(self):
        print("serving")

        while not self.shutdown_event.is_set():
            print("hello")
            time.sleep(1)

        print("done")


if __name__ == "__main__":
    shutdown_event = multiprocessing.Event()
    server = Server(shutdown_event)
    multiprocessing.Process(target=server.serve).start()
    time.sleep(5)
    shutdown_event.set()

But I want to keep the Server.shutdown method instead of changing the Server interface according to its usage (single processing v. multiprocessing) and I don't want clients to deal with multiprocessing.Event.


Solution

  • I have finally figured out a solution by myself:

    import multiprocessing
    import time
    
    
    class Server:
    
        def __init__(self):
            self.shutdown_event = multiprocessing.Event()
    
        def serve(self):
            print("serving")
    
            while not self.shutdown_event.is_set():
                print("hello")
                time.sleep(1)
    
            print("done")
    
        def shutdown(self):
            print("stopping")
            self.shutdown_event.set()
    
    
    if __name__ == "__main__":
        server = Server()
        multiprocessing.Process(target=server.serve).start()
        time.sleep(5)
        server.shutdown()
    

    It works in either case: single processing (multithreading) and multiprocessing.

    Remark. — With a multiprocessing.Event() in the __init__ method, Server instances are no longer pickable. That might be a problem if one wants to call a Server instance in a process pool (either with multiprocessing.pool.Pool or concurrent.futures.ProcessPoolExecutor). In this case, one should replace multiprocessing.Event() with multiprocessing.Manager().Event() in the __init__ method.