Search code examples
sqlalchemyfuturegunicorngeventconcurrent.futures

gevent + concurrent.futures and SQLAlchemy

I'm running and pyramid app inside gunicorn container with gevent async workers, one of endpoints is a long-pool endpoint pooling AMQP via kombu.

If the long pool withing 30s timeout returns some data from AMQP I need to save it to postgres before returning, now the question is:

Is it OK to start concurrent.futures.ThreadPoolExecutor in context on app running in gevent loop, and deal with SQLAlchemy sessions and data persistence inside a future submitted to executor?

Or am I completely wrong in my way of thinking?

PS DB driver is psycopg2

Solution

  • After using ThreadPoolExecutor with gevent in my high-concurrent app I can confirm that this works fine.

    But unnecessary if the DB Driver is already gevent "friendly" like: psycogreen