Search code examples
pythonpython-asyncio

Grouping asynchronous functions to run


I have a code that outputs numbers from 1 to 10:

from asyncio import get_event_loop, gather, sleep
    
async def main(k):
    print(k)
    await sleep(1)
    
    
if __name__ == '__main__':
    list_objects = list()
    count_group = 3

    for i in range(1, 11):
        list_objects.append(i)

    list_func = [main(x) for x in list_objects]

    loop = get_event_loop()
    loop.run_until_complete(gather(
        *list_func
    ))

Output:

1 2 3 4 5 6 7 8 9 10

It is noticeable that in the example above, 10 functions are simultaneously launched at once. How can I fix the code so that the number of concurrent launched functions main() is equal to count_group? That is, immediately the output should be 123, then 456, then 789 and at the end 10.


Solution

  • Split your tasks into run groups and gather() each group one by one.

    Example:

    if __name__ == '__main__':
        count_group = 3
    
        list_func = [ main(x) for x in range(1,11) ]
        run_groups = [ list_func[i:i+count_group] for i in range(0, len(list_func), count_group)]
        loop = get_event_loop()
    
        for rg in run_groups:
            loop.run_until_complete(gather(*rg))