Search code examples
celerydjango-celerycelery-task

upgrading celery 4.x.x to 5.x.x in Django app - execute_from_commandline() replacement


the usage in 4.x.x was as following:

from tenant_schemas_celery.app import CeleryApp

class TenantCeleryApp(CeleryApp):
    def create_task_cls(self):
        return self.subclass_with_self('...', abstract=True, name='...', attribute='_app')

tenant_celery = TenantCeleryApp()
base = celery.CeleryCommand(app=tenant_celery)
base.execute_from_commandline('...')
...

Now when updating celery lib to 5.x.x the following error show:

base = celery.CeleryCommand(app=tenant_celery) 
TypeError: __init__() got an unexpected keyword argument 'app'

from the documentation, the new CeleryCommand use click.Command class, how do I change my code to fit - what is the replacement usage for execute_from_commandline()?

EDIT: after some tries hard the following code works:

tenant_celery.worker_main(argv=['--broker=amqp://***:***@rabbitmq:5672//***',
                                    '-A', f'{__name__}:tenant_celery',
                                    'worker', '-c', '1', '-Q', 'c1,c2,c3,c4'])

Solution

  • You can do a few things here.


    The typical way to invoke / start a worker from within python is discussed at this answer:

    worker = tenant_celery.Worker(
        include=['project.tasks']
    )
    worker.start()
    

    In this case, you would be responsible for making the worker exit when you are done.


    To execute the CeleryCommand / click.Command, you pass in the arguments to the main function

    base = CeleryCommand()
    base.main(args=['worker', '-A', f'{__name__}:tenant_celery'])
    

    You would still be responsible for controlling how celery exits in this case, too. You may choose a verb other than worker such as multi for whatever celery subcommand you were expecting to call.

    You may also want to explicitly specify the name of the celery module for the -A parameter as discussed here.