I have a docker container setup with Django, where I'd like to use Celery. The broker is RabbitMQ, and for the results backend I'm using django-celery-backend. Once everything is up and running, I get an error if I call celery status Here is my setup:
docker-compose.yml
version: '3.8'
services:
django:
build: .
container_name: django
command: "python manage.py runserver 0.0.0.0:8000"
volumes:
- .:/usr/src/app/
ports:
- "8000:8000"
environment:
- DEBUG=1
- DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1
- CELERY_BROKER= amqp://guest:guest@rabbitmq:5672
depends_on:
- rabbitmq
- pgdb
celery:
build: .
container_name: celery
command: celery -A core worker -l info
volumes:
- .:/usr/src/app/
depends_on:
- django
- rabbitmq
pgdb:
image: postgres
container_name: pgdb
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=postgres
volumes:
- pgdata:/var/lib/postgresql/data/
ports:
- "5432:5432"
rabbitmq:
image: rabbitmq:management
container_name: 'rabbitmq'
environment:
- RABBITMQ_DEFAULT_USER=guest
- RABBITMQ_DEFAULT_PASS=guest
ports:
- 5672:5672
- 15672:15672
volumes:
pgdata:
settings.py
CELERY_BROKER_URL = os.environ.get('CELERY_BROKER_URL', 'amqp://guest:guest@rabbitmq:5672')
CELERY_RESULT_BACKEND = 'django-db'
I am able to send and execute tasks, and everything works fine in that regards. But if I try to call celery status inside of docker, I get the following error message:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/kombu/transport/__init__.py", line 57, in resolve_transport
transport = TRANSPORT_ALIASES[transport]
~~~~~~~~~~~~~~~~~^^^^^^^^^^^
KeyError: ''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/celery", line 8, in <module>
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/__main__.py", line 15, in main
sys.exit(_main())
^^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/bin/celery.py", line 217, in main
return celery(auto_envvar_prefix="CELERY")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/bin/base.py", line 134, in caller
return f(ctx, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/bin/control.py", line 82, in status
callback=callback).ping()
^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/app/control.py", line 294, in ping
return self._request('ping')
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/app/control.py", line 106, in _request
return self._prepare(self.app.control.broadcast(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/app/control.py", line 730, in broadcast
with self.app.connection_or_acquire(connection) as conn:
File "/usr/local/lib/python3.11/site-packages/celery/utils/objects.py", line 84, in __enter__
context = self._context = self.fallback(
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/app/base.py", line 892, in _acquire_connection
return self.pool.acquire(block=True)
^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/app/base.py", line 1206, in pool
self._pool = pools.connections[self.connection_for_write()]
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/app/base.py", line 816, in connection_for_write
return self._connection(url or self.conf.broker_write_url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/celery/app/base.py", line 867, in _connection
return self.amqp.Connection(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/kombu/connection.py", line 181, in __init__
if not get_transport_cls(transport).can_parse_url:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/kombu/transport/__init__.py", line 85, in get_transport_cls
_transport_cache[transport] = resolve_transport(transport)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/kombu/transport/__init__.py", line 66, in resolve_transport
raise KeyError(f'No such transport: {transport}')
KeyError: 'No such transport: '
I am able to call celery status if I use redis instead of RabbitMQ as the broker, but I would rather use RabbitMQ.
transport is a CELERY_BROKER_URL, so check your .env files and Docker ENVIRONMENTS, I think there is a problem.
Yes, I checked your environment and there is:
environment:
- DEBUG=1
- DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1
- CELERY_BROKER= amqp://guest:guest@rabbitmq:5672
But in settings:
CELERY_BROKER_URL = os.environ.get('CELERY_BROKER_URL', 'amqp://guest:guest@rabbitmq:5672')
CELERY_RESULT_BACKEND = 'django-db'
A solving of your issue:
Change here:
CELERY_BROKER_URL = os.environ.get('CELERY_BROKER', 'amqp://guest:guest@rabbitmq:5672')
CELERY_RESULT_BACKEND = 'django-db'
or here
django:
build: .
container_name: django
command: "python manage.py runserver 0.0.0.0:8000"
volumes:
- .:/usr/src/app/
ports:
- "8000:8000"
environment:
- DEBUG=1
- DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1
- CELERY_BROKER_URL= amqp://guest:guest@rabbitmq:5672
depends_on:
- rabbitmq
- pgdb