Search code examples
pythonoracle-databaseflaskgeventcx-oracle

Why o Flask-SocketIO (With Gevent) is paused when a query to db is made?


I'm working in a web based app to my company. Recently I've gotten stuck with a 'basic' problem to make SQL scripts to a database (OracleDB).

I'm using Flask-SocketIO webserver with async-mode='gevent', and apparentely when you execute the cx_Oracle.connection.cursor.execute(), is blocking my entire app, until the response returns (webserver stops receiving others requests).

I have searching a answer to the question, and I realized that the cx_Oracle isn't running parallel the others clients and requests.

Example of the problem:

from gevent import monkey; monkey.patch_all()
from flask_socketio import SocketIO
from flask import Flask
import cx_Oracle

app = Flask(__name__, template_folder='templates')

app.secret_key = 'testing'
app.config['DEBUG'] = False

socketio = SocketIO(app, async_mode='gevent')

@app.route('/')
def index():
    sql_query = 'select * from blabla'
    connection = cx_Oracle.connect(user, password, host, threaded=True)
    cursor = connection.cursor()
    cursor.execute(sql_query)
    transacoes = cursor.fetchall()

socketio.run(app, host='localhost', port=5005)

When I make more than 1 request to http://localhost/, my app doesn't response the 2+ requisitions until the first has done.

I tried to implement a gevent.ThreadPool to make more than 1 query in parallel, but i have faced the problem:

Example of the code with gevent.ThreadPool:

from gevent import monkey; monkey.patch_all()
from gevent.threadpool import ThreadPool
from flask_socketio import SocketIO
from flask import Flask
import cx_Oracle

app = Flask(__name__, template_folder='templates')

app.secret_key = 'testing'
app.config['DEBUG'] = False

socketio = SocketIO(app, async_mode='gevent')

def receive_data(user, password, host, sql_query):
    connection = cx_Oracle.connect(user, password, host, threaded=True)
    cursor = connection.cursor()
    cursor.execute(sql_query)
    response = cursor.fecthall()
    cursor.close()
    connection.close()
    return response

@app.route('/')
def index():
    sql_query = 'select * from blabla'

    pool = ThreadPool(1) # I tried with more than 100
    async_result = pool.apply_async(receive_data,
                                    args=(user, password, host, sql_query))
    transacoes = async_result.get()


socketio.run(app, host='localhost', port=5005)

I get the error when multiple requests is made in receive_data():

RuntimeError: Working outside of application context.

This typically means that you attempted to use functionality that needed to interface with the current application object in a way. To solve this set up an application context with app.app_context(). See the documentation for more information.

And:

'LoopExit: This operation would block forever


Solution

  • I found another solution for this problem.

    When a module doesn't support monkey_path, the eventlet sugests you to use eventlet.tpool http://eventlet.net/doc/threading.html.

    Example:

    from eventlet import tpool
    cur = tpool.execute(cx_Oracle.connect, connection_string, *args, **kwargs)
    

    This solves the main problem, and now I can use socketio whith "async_mode=eventlet".