Celery-Supervisor: How to restart a supervisor job to make newly updated celery-tasks working?

Question

I have a running supervisor job for my celery server. Now I need to add a new task to it, but unfortunately my celery server command is not configured to track those dynamic changes automatically.

Here is my celery command:

python manage.py celery worker --broker=amqp://username:password@localhost/our_app_vhost

To restart my celery process, I have tried,

sudo supervisorctl -c /etc/supervisor/supervisord.conf restart <process_name>
supervisorctl stop all
supervisorctl start all
service supervisor restart

But nothing found working. How to restart it?

Answer 1

If you want to manage process with supervisorctl, you should configure supervisorctl, rpcinterface in your configuration file.

Here is a sample configuration file.

sample.conf

[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10           ; (num of main logfile rotation backups;default 10)
loglevel=info                ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false               ; (start in foreground if true;default false)
minfds=1024                  ; (min. avail startup file descriptors;default 1024)
minprocs=200                 ; (min. avail process descriptors;default 200)

[program:my_worker]
command = python manage.py celery worker --broker=amqp://username:password@localhost/our_app_vhost

[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

Now start supervisor with

supervisord -c sample.conf

Now if you want to restart your worker you can do it with

supervisorctl -c sample.conf restart my_worker

This restarts your worker. Alternatively you can also drop to supervisor shell and you can restart it

sudo supervisorctl -c sample.conf
supervisor> restart my_worker
my_worker: stopped
my_worker: started

Note:

There is an option to autoreload workers in Celery

python manage.py celery worker --autoreload --broker=amqp://username:password@localhost/our_app_vhost

This should be used in development mode only. Using this in production is not recommended.

More about this on celery docs.