Search code examples
pythonredispython-rq

Reduce the number of workers on a machine in Python-RQ?

What is a good way to Reduce the number of workers on a machine in Python-RQ?

According to the documentation, I need to send a SIGINT or SIGTERM command to one of the worker processes on the machine:

Taking down workers

If, at any time, the worker receives SIGINT (via Ctrl+C) or SIGTERM (via kill), the worker wait until the currently running task is finished, stop the work loop and gracefully register its own death.

If, during this takedown phase, SIGINT or SIGTERM is received again,the worker will forcefully terminate the child process (sending it SIGKILL), but will still try to register its own death.

This seems to imply a lot of coding overhead:

  • Would need to keep track of the PID for the worker process
  • Would need to have a way to send a SIGINT command from a remote machine

Do I really need to custom build this, or is there a way to do this easily using the Python-RQ library or some other existing library?

Solution
    1. Get all running workers using rq.Worker.all()
    2. Select the worker you want to kill
    3. Use os.kill(worker.pid, signal.SIGINT)