How can I run a long running blocking function concurrently with tornado IOLoop

Question

I have a tornado application:

if __name__ == "__main__":
    app = make_app()
    app.listen(8090)

    tornado.ioloop.IOLoop.current().start()

How can I have a long running task operate concurrently?

Specifically, I have a redis pubsub which will notify my tornado app of updated authorization tokens.

I tried this:

tornado.ioloop.IOLoop.current().run_in_executor(None, redis_pubsub.subscribe_to_valid_tokens)

Since I never await the Future it doesn't seem to run the function, so i'm a bit stuck about the 'correct' way to do this.

• I'm wondering if I should do this with a separate python thread (or maybe leverage the multiprocessing library with a queue?
• Is there some better use/structure of async/await that I should use?

Answer 1

I had to store a reference to the redis object and pubsub object in a class that doesn't get garbage collected. Also, the pubsub's run_in_thread method was very helpful!