Search code examples
celerycelerydflask-celery

What is the difference between using celery with and without multi

I noticed that there are two separate ways of running celery workers. You can start the workers with or without a multi command suffix:

celery multi start worker -A main.celery --concurrency=10

and

celery -A main.celery worker --concurrency=10

Is there a difference between the two, and is one of them recommended to use in production over another?

Solution

  • celery multi is more like a CLI to manage one or multiple workers and it is mostly for interactive use. In your example, if you start your worker with

    celery multi start worker -A main.celery --concurrency=10

    it actually starts your worker as a daemon. Once it is up and running, celery multi provides a range of commands to interact with your worker, for example to shut it down.

    The 'standard' celery worker command starts the worker in the main thread:

    celery -A main.celery worker --concurrency=10

    In a production environment, I would use the standard (and not the multi) command. This separates the concern of actually running the worker from how/where to run your worker, think for example Docker or a Procfile.