Run celery worker with a compiled python module compiled using pyinstaller

Question

I have a simple celery module (worker.py) that looks like:

from celery import Celery

app = Celery('celery')

@app.task(bind=True)
def some_task(self):
    print('HELLO WORLD')

I'm compiling the module using pyinstaller:

python3 -m PyInstaller --onefile worker.py

Then, I can run a celery worker with the python module quite easily using the command:

celery -A worker.app worker -l info

My question is: How can I run the celery worker with the compiled version of the module (Created by PyInstaller)?

Answer 1

The solution for this problem is to run the worker from inside a python module, so for example:

from celery import Celery

app = Celery('celery', fixups=[])

@app.task(bind=True)
def some_task(self):
    print('HELLO WORLD')

if __name__ == "__main__":
    app.worker_main(argv=['worker', '--loglevel=info'])

and then we can use pyinstaller to compile this module:

pyinstaller --one-file module.py --additional-hooks-dir=pyinstaller_hooks_folder

the additional-hooks-dir flag is for pyinstaller to know how to collect celery's submodules, etc.. we need to have pyinstaller_hooks_folder in our project folder and a celery hook file in it:

Project_folder
└── pyinstaller_hooks_folder
    └──hook-celery.py 

hook-celery.py file:

from PyInstaller.utils.hooks import collect_all

datas, binaries, hiddenimports = collect_all('celery')