How to add a task to a group in Celery?

Question

I want to incrementally build a group of Celery tasks in my code because I will be creating the tasks based on logic in a loop.

For example:

my_group = group()
for item in items:
    if item.is_special():
        # This doesn't work...
        my_group.add(special_processing.s(item.id))
    else:
        my_group.add(regular_processing.s(item.id))

 res = my_group()

I've read that groups are partials, which is good, but how do you combine partials so that they form a group?

Answer 1

A simple way I've found so far is to create a list of tasks, and to then convert that into a group.

So:

tasks = []
for item in items:
    if item.is_special():
        tasks.append(special_processing.s(item.id))
    else:
        tasks.append(regular_processing.s(item.id))
res = group(*tasks)

I haven't tested this yet, but I'll update this answer if this doesn't work.