I want to use concurrency using celery in python. I have a tasks.py file, which is a web crawler using BeautifulSoup. The imports I have done are:
from celery import Celery
import eventlet
app = Celery('tasks', backend='amqp', broker='amqp://')
For celery worker, I am using this command:
celery -A tasks worker --loglevel=info --pool=eventlet -c 1000
Error:
mayank@mayank-Studio-1558:~/cognite/test$ celery -A tasks worker --loglevel=info -- pool=eventlet -c 1000
Traceback (most recent call last):
File "/usr/local/bin/celery", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/celery/__main__.py", line 28, in main
maybe_patch_concurrency()
File "/usr/local/lib/python2.7/dist-packages/celery/__init__.py", line 124, in maybe_patch_concurrency
patcher()
File "/usr/local/lib/python2.7/dist-packages/celery/__init__.py", line 89, in _patch_eventlet
eventlet.monkey_patch()
AttributeError: 'module' object has no attribute 'monkey_patch'
I want to perform tasks using 1000 threads. I have tried importing eventlet, and then did:
eventlet.monkey_patch(socket=True, select=True)
eventlet.import_patched('monkey_patch')
still didn't work and same error.
Please if anybody could help, would be great. Thanks.
This error was coming because of eventlet installation. Previously I used pip install eventlet, but sudo apt-get install eventlet worked for me, which resolves the monkey_patch dependency.