In the documentation, the @celery.task decorator is not passed arguments, yet in the GitHub example, it is named "tasks.add". Why is it? When I remove the name, the example no longer works, complaining of
KeyError: '__main__.add'
[1] http://flask.pocoo.org/docs/0.10/patterns/celery/ [2] https://github.com/thrisp/flask-celery-example/blob/master/app.py#L25
In the Flask documentation the task name was not set because the code is assumed to be inside a tasks module, so the task's name will be automatically generated as tasks.add, in the Celery docs:
Every task must have a unique name, and a new name will be generated out of the function name if a custom name is not provided
Check the Names section of the Celery docs for more info.
In the other example on Github, the author sets the name explicitly instead of relying on the automatic naming, which will be __main__.tasks if running as the main module, which is the case when running the Flask server.
Update on why you're having this issue:
The task is being sent from the function hello_world when you access the /test page by passing x and y:
res = add.apply_async((x, y))
Because the task add is inside the __main__ module it will be named __main__.add and sent to the worker with this name, but on the other hand the worker that you started using:
celery worker -A app.celery
Has this task registered as app.add that's why you're getting this error:
[2014-10-10 10:32:29,540: ERROR/MainProcess] Received unregistered task of type '__main__.add'.
The message has been ignored and discarded.
Did you remember to import the module containing this task?
Or maybe you are using relative imports?
Please see http://docs.celeryq.org/en/latest/userguide/tasks.html#task-names for more information.
The full contents of the message body was:
{'timelimit': (None, None), 'utc': True, 'chord': None, 'args': (2787476, 36096995), 'retries': 0, 'expires': None, 'task': '__main__.add', 'callbacks': None, 'errbacks': None, 'taskset': None, 'kwargs': {}, 'eta': None, 'id': '804e10a0-2569-4338-a5e3-f9e07689d1d1'} (218b)
Traceback (most recent call last):
File "/home/peter/env/celery/lib/python2.7/site-packages/celery/worker/consumer.py", line 455, in on_task_received
strategies[name](message, body,
KeyError: '__main__.add'
Check the output of the worker:
[tasks]
. app.add
. celery.backend_cleanup
. celery.chain
. celery.chord
. celery.chord_unlock
. celery.chunks
. celery.group
. celery.map
. celery.starmap
Celery only sends the task name to the worker to execute it, so when you explicitly set the task name the hello_world function will send the task with this name which is registered in the worker.
Update:
The task name can be whatever you want, it can just be add, and your celery tasks don't have to be in a tasks module at all, to understand more about task names try this:
Remove the explicit task name and start a worker:
celery worker -A app.celery
in another terminal window, cd to the code directory and start an interactive python shell and try this:
>>> import app
>>> app
<module 'app' from 'app.pyc'>
>>> app.add
<@task: app.add of app:0xb6a29a6c>
>>> # check the name of the task
... app.add.name
'app.add'
>>> t = app.add.delay(2, 3)
>>> t.result
5
As you can see we didn't use an explicit name and it worked as expected because the name of task from where we sent it is the same as registered in the worker (see above).
Now back to why you got this error when you removed the task name, the task is sent from app.py right, in the same directory run this:
$ python -i app.py
Then interrupt the Flask server with Ctrl + C, and try this:
>>> add.name
'__main__.add'
As you can see this is why you got this error and not because you removed the task name.