I had a working django project, but I started getting errors with celery after I added django-filter to my requirements file.
I'm using python 3.7 and here's the list of installed packages w/ versions:
Django-3.0.7
amqp-2.6.0
asgiref-3.2.7
billiard-3.6.3.0
celery-4.4.4
certifi-2020.4.5.1
chardet-3.0.4
django-bootstrap3-12.1.0
django-celery-results-1.2.1
django-filter-2.2.0
djangorestframework-3.11.0
idna-2.9 importlib-metadata-1.6.0
kombu-4.6.10
ovirt-engine-sdk-python-4.4.3
psycopg2-2.8.5
pycurl-7.43.0.5
python-gitlab-2.2.0
pytz-2020.1
redis-3.5.3
requests-2.23.0
six-1.15.0
sqlparse-0.3.1
urllib3-1.25.9
vine-1.3.0
zipp-3.1.0
I decided to remove my code that uses filtering and just left the pip install for the package, and I still saw the same errors. Below is a pertinent snippet, but I can add the whole trace if necessary.
celery-beat_1 | [2020-06-04 18:11:33,145: CRITICAL/MainProcess] beat raised exception <class 'ModuleNotFoundError'>: ModuleNotFoundError("No module named 'future'")
celery-beat_1 | Traceback (most recent call last):
celery-beat_1 | File "/usr/local/lib/python3.7/site-packages/kombu/utils/objects.py", line 42, in __get__
celery-beat_1 | return obj.__dict__[self.__name__]
celery-beat_1 | KeyError: 'scheduler'
celery-beat_1 |
celery-beat_1 | During handling of the above exception, another exception occurred:
celery-beat_1 |
celery-beat_1 | Traceback (most recent call last):
celery-beat_1 | File "/usr/local/lib/python3.7/site-packages/kombu/utils/objects.py", line 42, in __get__
celery-beat_1 | return obj.__dict__[self.__name__]
celery-beat_1 | KeyError: 'backend'
.
.
.
celery-beat_1 | [2020-06-04 18:11:33,161: WARNING/MainProcess] File "/usr/local/lib/python3.7/site-packages/celery/backends/base.py", line 10, in <module>
celery-beat_1 | [2020-06-04 18:11:33,161: WARNING/MainProcess] from future.utils import raise_with_traceback
celery-beat_1 | [2020-06-04 18:11:33,161: WARNING/MainProcess] ModuleNotFoundError
celery-beat_1 | [2020-06-04 18:11:33,161: WARNING/MainProcess] :
celery-beat_1 | [2020-06-04 18:11:33,161: WARNING/MainProcess] No module named 'future'
The project is constructed using docker compose to setup celery/redis/db/webapp.
Any ideas as to why django-filter is breaking everything?
If you install packages using a tool like pipenv or similar package managers, then they will upgrade all out of date packages unless you tell them not to.
In this case celery was upgraded to 4.4.4 and you've hit a rather embarrassing bug in celery (honestly... how that did that get through CI?), but at least with an easy fix of installing the future module.