Search code examples
pythonflaskceleryflask-restx

flask celery throw error "urls must start with a leading slash"


I'm trying to implement the celery using flask and when i run this command celery -A src.celery_worker.celery_app worker --loglevel=debug to start the celery worker, it throwing error:

Couldn't import 'src.celery_worker.celery_app': urls must start with a leading slash

Here's the full traceback:

Traceback (most recent call last):
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 58, in convert
    return find_app(value)
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/celery/app/utils.py", line 383, in find_app
    sym = symbol_by_name(app, imp=imp)
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/kombu/utils/imports.py", line 61, in symbol_by_name
    reraise(ValueError,
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/kombu/exceptions.py", line 34, in reraise
    raise value.with_traceback(tb)
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/kombu/utils/imports.py", line 59, in symbol_by_name
    module = imp(module_name, package=package, **kwargs)
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/celery/utils/imports.py", line 109, in import_from_cwd
    return imp(module, package=package)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/src/__init__.py", line 42, in <module>
    api = Api(
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask_restx/api.py", line 197, in __init__
    self.init_app(app)
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask_restx/api.py", line 236, in init_app
    self._init_app(app)
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask_restx/api.py", line 247, in _init_app
    self._register_doc(self.blueprint or app)
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask_restx/api.py", line 320, in _register_doc
    app_or_blueprint.add_url_rule(self._doc, "doc", self.render_doc)
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask/scaffold.py", line 56, in wrapper_func
    return f(self, *args, **kwargs)
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/flask/app.py", line 1083, in add_url_rule
    rule = self.url_rule_class(rule, methods=methods, **options)
  File "/home/sonnc/project/mac/lifescience-operator-management-bff/venv/lib/python3.8/site-packages/werkzeug/routing.py", line 698, in __init__
    raise ValueError("urls must start with a leading slash")
ValueError: Couldn't import 'src.celery_worker.celery_app': urls must start with a leading slash

Below is the file src/celery_worker/__init__.py:

import os
from flask import Flask

from celery import Celery, Task


def celery_init_app(app: Flask) -> Celery:
    class FlaskTask(Task):
        def __call__(self, *args: object, **kwargs: object) -> object:
            with app.app_context():
                return self.run(*args, **kwargs)

    celery_app = Celery(app.name, task_cls=FlaskTask)
    celery_app.config_from_object(app.config["CELERY"])
    celery_app.set_default()
    app.extensions["celery"] = celery_app
    return celery_app


def create_app() -> Flask:
    app = Flask(__name__)
    # url for local redis
    # redis_url = "redis://localhost:6379"
    # url for docker redis
    # redis_url = "redis://redis:6379/0"
    app.config.from_mapping(
        CELERY=dict(
            broker_url="redis://localhost:6379",
            result_backend="redis://localhost:6379",
            task_ignore_result=True,
            # import the tasks
            # imports=("src.common.send_email",),
        ),
    )
    app.config.from_prefixed_env()
    celery_init_app(app)
    return app


flask_app = create_app()
celery_app = flask_app.extensions["celery"]

Look like the error is related to the flask_restx module, but I can't trace how it causing this issue.
Please help, thanks.


EDIT 1: Here's the proof that every route and path in my source code are start with a slash:

enter image description here

enter image description here


Solution

  • Thanks you all for your answers and suggestions, I've tried to debug the next day but somehow it suddenly worked on the local environment, the issue is gone, I don't really understand why, but the issue still remains on the docker environment.
    So I've tried to debug further and found that the reason it's because the setting in docker-compose.yml didn't set the env_file file.
    So I added the env_file to the setting of celery_worker in docker-compose.yml and it worked, here's how I fixed it:

    celery_worker:
        build:
          context: .
          dockerfile: src/celery_worker/Dockerfile
        image: flask_celery_worker
        command: 
          ["/start-celeryworker", "sh", "-c"]
          # ['celery', '-A', 'src.celery_worker.celery_app', 'worker', '-l', 'debug']
          # celery -A src.celery_worker.celery_app worker --loglevel=debug
        # volumes:
        #   - .:/lifescience-operator-management-bff-worker
        env_file:
          - .env
        ports:
          - 6900:6900
        networks:
          - my-network
        # environment:
        #   - FLASK_APP=app
        depends_on:
          - redis
    

    Well, I know it sounds like doesn't related to the issue that I've posted, but believe me, that's all I did to fix this issue.
    Hope this answer help some one.