Search code examples
pythonmultiprocessingtornado

Can I initialize tornado IOLoop.instance before forking new processes?


Can I initialize an IOLoop.instance and than fork new processes, which uses IOLoop.instance()? Something like:

#some code which initializes IOLoop.instance()           (1)
storage - Storage()
...
def f(storage):
    """some ioloop worker, which uses IOLoop.instance()"""
    storage.db_client.send(some value)
    ...

p1 = Process(target=f, args=(storage,))
p2 = Process(target=f, args=(storage,))

The IOLoop documentation doesn't says anything about usage IOLoop with multithreading, but tornado.process.fork_processes documentation forbids initializing IOLoop before forking.

The point is that the code in (1) creates storage object, which is used by function f. The storage contains asynchronous database client which is suppose to use the same ioloop as the worker process.


Solution

  • After asking on tornado user group, I received answer, that I can't use IOLoop created by the parent process in child process. The solution is very simple:

    def f(storage):
        """some ioloop worker, which uses IOLoop.instance()"""
        # worker code starts here
        del IOLoop._instance
    
        # here we can safe use IOLoop
        IOLoop.instance().add_callback(...)
        storage.db_client.send(some value)