Search code examples
pythontaskpython-asynciocancellationgraceful-shutdown

Is suppressing `asyncio.CancelledError` acceptable here?


Example:

    with suppress(asyncio.CancelledError):
        [await t for t in asyncio.all_tasks(loop=self.loop)
            if t is not asyncio.current_task()]

To avoid Task was destroyed but it is pending! warning, I have to await the tasks after cancelling, but awaiting them leads to the terminal being spammed with CancelledError. I know it's cancelled but I don't need to see that.

Does using contextlib.suppress here intervene negatively with the cancellation? The only other way that I can avoid seeing the cancelled error (or task destroyed warning without awaiting) is to start my initial tasks with asyncio.wait rather than asyncio.gather. For some reason, wait appears to suppress exceptions. I use return_when=asyncio.FIRST_EXCEPTION on wait and return_exceptions=True on gather. But it seems that regardless of how I set their keyword args, gather prints exceptions while wait does not.


Solution

  • CancelledError is used for two purposes in asyncio: one is to signal a cancellation request - that's the one you get inside the coroutine being cancelled - and the other is to signal a cancellation response - that's the one you get in the coroutine that awaits the task. Suppressing the cancellation request is a bad idea because it makes the coroutine not respond to cancellation, causing issues later. But suppressing the response is perfectly fine because you might want to await the cancelled coroutine (e.g. to avoid this warning) without propagating an exception.

    Note that the way you're doing doesn't quite look correct because the list comprehension will terminate on first CancelledError, so you won't get to awaiting the other coroutines. The correct way is to put suppress inside the loop, something like:

    for t in tasks:
        with contextlib.suppress(asyncio.CancelledError):
            await t
    
    # or, simpler:
    await asyncio.gather(*tasks, return_exceptions=True)
    

    wait() doesn't propagate exceptions because it returns sets of futures rather than their results. You'd get the exceptions if you tried to access the results of the returned futures. gather(return_exceptions=True) returns a mixture of results and exceptions and it should not raise anything. If it does, please edit the question to provide a minimal example.