Redis Python and virtual env

Question

I want to send to redis queue with python some task to execute.

There is a way to run them with different virtual env? i provide you an example:

from datetime import datetime, timedelta
import time
from redis import Redis
from rq import Queue,Retry

import tasks1 #this need some libs
import tasks2 #this need libs incompatible with task1

queue = Queue(connection=Redis())

def queue_tasks():
    queue.enqueue(tasks1.print_task, 5, retry=Retry(max=2))
    queue.enqueue_in(timedelta(seconds=10), tasks2.print_numbers, 5)

def main():
    queue_tasks()

if __name__ == "__main__":
    main()

or i can only run them with the env where i call the app?

Answer 1

Redis acts as a broker here for rq. Almost every implementation of message queue involves serialising the message along with its parameters into json (or other formats like pickle) and sending them to the redis server. A rq worker is configured to listen on the same queue and processes the messages as they are pushed. Message queues are typically used to execute tasks on a different machine.

In the current scenario, it's pretty much possible to execute the task in a different venv altogether. You just have to stringify the function name in a way rq does instead of importing functions and passing them as a reference. rq internally does the same. It takes the function reference, extrats just the name as text, serialises with job parameters and sends to the queue on redis server.