Search code examples
pythonpython-asyncio

Python asyncio gather does not exit after task complete


I have a coroutine foo1 from which I use asyncio.create_task() to call another coroutine foo2.

As expected, foo1 finishes running and does not wait for the task to complete because there is no await.

I have added asyncio.gather() at the end to finish up any pending tasks.

The problem is gather does not release control back to the main program after the task is complete. How do I get the program to end and run print("done") when complete?

import asyncio

async def foo3(x):
    for y in range(2):
        await asyncio.sleep(2)
        print(f"foo3 {x} {y}")

async def foo2():
    for x in range(2):
        await asyncio.sleep(0.5)
        asyncio.create_task(foo3(x))
        print(f"foo2 {x}")

async def foo1():
    t = asyncio.create_task(foo2())  # If I await this the code exits prior to completion of nested tasks.
    pending = asyncio.all_tasks()
    await asyncio.gather(*pending)   # This line never finishes even after foo2 & foo3 are complete
    

if __name__ == "__main__":
    asyncio.run(foo1())
    print("done")
# Output
foo2 0
foo2 1
foo3 0 0
foo3 1 0
foo3 0 1
foo3 1 1


Solution

    • Full answer is here

    but the issue with that can sometimes be an infinite loop where it waits for the asyncio.current_task() to complete, which is itself. Some answers suggested some complicated workarounds involving checking coro names or len(asyncio.all_tasks()), but it turns out it's very simple to do by taking advantage of set operations

    • Short answer: you must change one line:

    await asyncio.gather(*pending)await asyncio.gather(*pending - {asyncio.current_task()})

    import asyncio
    
    async def foo2():
        for x in range(3):
            await asyncio.sleep(0.5)
            print(x)
    
    async def foo1():
        asyncio.create_task(foo2())
        pending = asyncio.all_tasks()
        await asyncio.gather(*pending - {asyncio.current_task()})
    
    if __name__ == "__main__":
        asyncio.run(foo1())
        print("done")
    
    0
    1
    2
    done