Awaitable objects in Python after 3.10

Question

I was surprised by many errors like the following in code that was working without issues on Python 3.10, but failed when run using Python 3.11:

AttributeError: '<Some awaitable class>' object has no attribute 'add_done_callback'

After some searching, I found that this is due to some asyncio aspects having been deprecated. I likely missed this due to the way the asyncio library is implemented, so that it doesn't actually trigger deprecation warnings.

That's water under the bridge, but now I'm having trouble refactoring my code correctly. For example, this code (simplified example) used to work without issues (and still works in Python 3.10):

import asyncio


class ExampleAwaitable:
    def __await__(self):
        async def _():
            return await asyncio.sleep(0)

        return _().__await__()


print(f"{asyncio.iscoroutine(ExampleAwaitable())=}")


async def amain():
    await asyncio.wait([ExampleAwaitable()])
    print("waited!")


asyncio.run(amain())

But in Python 3.11, it results in:

Traceback (most recent call last):
  File "will_fail.py", line 20, in <module>
    asyncio.run(amain())
  File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "will_fail.py", line 16, in amain
    await asyncio.wait([ExampleAwaitable()])
  File "C:\Program Files\Python311\Lib\asyncio\tasks.py", line 418, in wait
    return await _wait(fs, timeout, return_when, loop)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\tasks.py", line 522, in _wait
    f.add_done_callback(_on_completion)
    ^^^^^^^^^^^^^^^^^^^
AttributeError: 'ExampleAwaitable' object has no attribute 'add_done_callback'

What is the most straightforward and recommended way to refactor this code, to avoid this error in 3.11 and beyond?

Answer 1

what's changed in this case is the behavior of asyncio.wait: it now expects task objects, and not raw co-routines: "Changed in version 3.11: Passing coroutine objects to wait() directly is forbidden." (https://docs.python.org/3/library/asyncio-task.html#asyncio.wait )

The bad news, on the other hand, is that create_task, on its side, wants a co-routine, and passing the instance of a class with am __await__ does not suffice.

I guess the easier work-around is to create a wrapper co-routine for it, and wrap it in a task:

import asyncio


class ExampleAwaitable:
    def __await__(self):
        async def _():
            return await asyncio.sleep(0)

        return _().__await__()

async def wrapper(awaitable):
    return await awaitable

print(f"\n{asyncio.iscoroutine(ExampleAwaitable())=}\n")


async def amain():
    await asyncio.wait([asyncio.create_task(wrapper(ExampleAwaitable()))])
    print("waited!")


asyncio.run(amain())

I've made some exercises in the sense of extending asyncio.Future or Task with the class to see if that would work, for no avail.

The way I found most confortable is just refactor the approach above so it takes less boilerplate code where the class is actually used - I felt confortable with this:

import asyncio

class Base:
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        origi_await = self.__await__

    async def _wrapper(self):
            return await self

    @property
    def as_task(self):
        return asyncio.create_task(self._wrapper())


class ExampleAwaitable(Base):
    def __await__(self):
        async def _():
            await asyncio.sleep(0)
            return 23

        return _().__await__()

print(f"\n{asyncio.iscoroutine(ExampleAwaitable())=}\n")


async def amain():
    d, p = await asyncio.wait([ExampleAwaitable().as_task])

    print(f"waited result: {d.pop().result()}")


asyncio.run(amain())

But - there seems to be no straightforward way to just use one's own awaitable class with most of the asyncio API - unless one go to the depths of implementing a good deal of asycio.Future (including callbacks) in his own code. Wrapping it in a co-routine is the way I found out to work with these changes.