Search code examples
pythoncelerybackground-process

Celery object editing best-practices


I have a celery task that performs an operation on an objects attributes and saves it. Let's say we're computing a persons bmi. We take a persons height and weight and calculate the bmi, then save it on person.bmi. In Rails's DelayedJob you would just make an instance method:

Class Person

    def compute_bmi()
        self.bmi = (self.weight / self.height) * 703
        self.save

person.delay.compute_bmi()

But in python the celery tasks normally seem to be separated out in tasks.py without access to the instance, Class, or database session. Is it bad practice to write celery tasks that do it the above way ?

My issue is I might have to do this on a few million records, so sending the task a json payload of the user data and returning the bmi is mostly pointless, as the load from the db read/write is one of the biggest parts of the job.

It also seems like doing it this way wouldn't be possible with services like ironmq's ironworker, the only way I can think of to put this in the cloud is to spin up an ec2/vps instance.


Solution

  • Yes, typically you would want to make a task that accepts a Person's id, retrieves it, and then performs the action requested. I'm assuming you're using Django as that's the closest analogy to Rails in Python.

    For example:

    @app.task
    def determine_bmi(person_id):
        person = Person.objects.get(pk=person_id)
        person.compute_bmi()
    

    Here's the section in the Celery docs on First Steps with Django.