Create process in tornado web server

Question

I have a multiproccessing tornado web server and I want to create another process that will do some things in the background.

I have a server with to following code

start_background_process
app = Application([<someurls>])
server = HTTPServer(app)
server.bind(8888)
server.start(4)  # Forks multiple sub-processes
IOLoop.current().start()

def start_background_process():
    process = multiprocessing.Process(target=somefunc)
    process.start()

and everything is working great. However when I try to close the server (by crtl c or send signal) I get AssertionError: can only join a child process

I understood the cause of this problem: when I create a process with multiprocess a call for the process join method is registered in "atexit" and because tornado does a simple fork all its childs also call the join method of the process I created and the can't since the process is their brother and not their son? So how can I open a process normally in tornado?

Answer 1

"HTTPTserver start" uses os.fork to fork the 4 sub-processes as it can be seen in its source code.

If you want your method to be executed by all the 4 sub-processes, you have to call it after the processes have been forked.

Having that in mind your code can be changed to look as below:

import multiprocessing
import tornado.web
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop

# A simple external handler as an example for completion
from handlers.index import IndexHandler


def method_on_sub_process():
    print("Executing in sub-process")


def start_background_process():
    process = multiprocessing.Process(target=method_on_sub_process)
    process.start()


def main():
    app = tornado.web.Application([(r"/", IndexHandler)])
    server = HTTPServer(app)
    server.bind(8888)
    server.start(4)
    start_background_process()
    IOLoop.current().start()


if __name__ == "__main__":
    main()

Furthermore to keep the behavior of your program clean during any keyboard interruption , surround the instantiation of the sever by a try...except clause as below:

def main():
    try:
        app = tornado.web.Application([(r"/", IndexHandler)])
        server = HTTPServer(app)
        server.bind(8888)
        server.start(4)
        start_background_process()
        IOLoop.current().start()
    except KeyboardInterrupt:
        IOLoop.instance().stop()