Search code examples
pythonflaskredispython-rq

How to access redis connection in background task


enter image description here

I'm trying to extend the flask-base project https://github.com/hack4impact/flask-base/tree/master/app which comes with a user model only. I'm trying to add the ability to run a background task on redis using rq. I've found https://devcenter.heroku.com/articles/python-rq which is helpful.

this app has support for redis queues with a background redis queue being implemented by running :

@manager.command
def run_worker():
    """Initializes a slim rq task queue."""
    listen = ['default']
    conn = Redis(
        host=app.config['RQ_DEFAULT_HOST'],
        port=app.config['RQ_DEFAULT_PORT'],
        db=0,
        password=app.config['RQ_DEFAULT_PASSWORD'])

    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

using:

$ python manage.py run_worker

In my views I have:

@main.route('/selected')
def background_selected():
    from rq import Queue
    from manage import run_worker.conn
    q = Queue(connection=conn)
    return q.enqueue(selected)

The problem is I don't know how to import the connection created in run_worker() into my view. I've tried variations of :

from manage import run_worker.conn

but I'm getting:

SyntaxError: invalid syntax.

How can I get access to the conn variable in the background task?


Solution

  • from the documentation, python-rq Configuration

    Can you try by making the below changes:

    manager.py

    import redis
    
    """Initializes a slim rq task queue."""
    listen = ['default']
    conn = redis.Redis(host=app.config['RQ_DEFAULT_HOST'], 
                       port=app.config['RQ_DEFAULT_PORT'],
                       db=0,     
                       password=app.config['RQ_DEFAULT_PASSWORD'])
    
    @manager.command
    def run_worker():
        with Connection(conn):
            worker = Worker(map(Queue, listen))
            worker.work()
    

    and from view:

    from rq import Queue
    from manage import conn
    
    q = Queue(connection=conn)