I am getting started with Celery module in python and have created a new script following the instructions on Celery website, but I received errors when creating a Celery worker server using
celery -A addTask worker --loglevel=info
addTask is a simple python script returning the sum:
from celery import Celery
app = Celery('addTask', broker = 'amqp://guest@localhost//')
@app.task
def add(x,y):
return x+y
Another script that calls the addTask.py is runAddTask.py:
import addTask
if __name__=='__main__':
result = addTask.add.delay(5,5)
The version of Celery I am using is 4.0.2, and the version of Python is 3.5.2. I am running on Windows 10. I installed rabbitmq as broker so the scripts are fine. This is exactly the same as listed on the celeryproject, however, I received error message as below:
C:\Users\user\desktop\parallel_python\celery> celery -A addTask worker --loglevel=info
celery : [2017-01-04 14:23:46,058: CRITICAL/MainProcess] Unrecoverable error: AttributeError("Can't pickle local object
'Pool.__init__.<locals>.Process'",)
At line:1 char:1
+ celery -A addTask worker --loglevel=info
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: ([2017-01-04 14:...ls>.Process'",):String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\worker\worker.py", line 203, in start
self.blueprint.start(self)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\bootsteps.py", line 119, in start
step.start(parent)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\bootsteps.py", line 370, in start
return self.obj.start()
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\concurrency\base.py", line 131, in
start
self.on_start()
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\concurrency\prefork.py", line 112, in
on_start
**self.options)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\pool.py", line 1008, in __init__
self._create_worker_process(i)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\pool.py", line 1117, in
_create_worker_process
w.start()
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\process.py", line 122, in start
self._popen = self._Popen(self)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\context.py", line 383, in _Popen
return Popen(process_obj)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\popen_spawn_win32.py", line 79, in
__init__
reduction.dump(process_obj, to_child)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\reduction.py", line 99, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'Pool.__init__.<locals>.Process'
-------------- celery@userv4.0.2 (latentcall)
---- **** -----
--- * *** * -- Windows-10-10.0.14393-SP0 2017-01-04 14:23:46
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app: addTask:0x1d17467e0b8
- ** ---------- .> transport: amqp://guest:**@localhost:5672//
- ** ---------- .> results: disabled://
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery
[tasks]
. addTask.add
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\spawn.py", line 159, in spawn_main
new_handle = steal_handle(parent_pid, pipe_handle)
File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\reduction.py", line 126, in
steal_handle
_winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied
I am really stuck here. Any help is greatly appreciated! Thank you!
https://github.com/celery/celery/issues/3551
windows is not officially supported by celery 4.
You can downgraded to Celery 3.x.x.