How to integrate asyncronous python code into synchronous function?

Question

I have an external library that uses requests module to perform http requests. I need to use the library asynchronously without using many threads (it would be the last choice if nothing else works). And I can't change its source code either. It would be easy to monkey-patch the library since all the interacting with requests module are done from a single function, but I don't know if I can monkey-patch synchronous function with asynchronous one (I mean async keyword).

Roughly, the problem simplifies to the following code:

import asyncio
import aiohttp
import types
import requests


# Can't modify Library class.
class Library:
    def do(self):
        self._request('example.com')

        # Some other code here..

    def _request(self, url):
        return requests.get(url).text


# Monkey-patched to this method.
async def new_request(self, url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()


async def main():
    library = Library()

    # Do monkey-patch.
    library._request = types.MethodType(new_request, library)

    # Call library asynchronously in hope that it will perform requests using aiohttp.
    asyncio.gather(
        library.do(),
        library.do(),
        library.do()
    )

    print('Done.')

asyncio.run(main())

But as expected, it doesn't work. I get TypeError: An asyncio.Future, a coroutine or an awaitable is required on asyncio.gather call. And also RuntimeWarning: coroutine 'new_request' was never awaited on self._request('example.com').

So the question is: is it possible to make that code work without modifying the Library class' source code? Otherwise, what options do I have to make asynchronous requests using the library?

Answer 1

Is it possible to make that code work without modifying the Library class' source code? Otherwise, what options do I have to make asynchronous requests using the library?

Yes, it is possible, and you even do not need monkey-patching to perform that. You should use asyncio.to_thread to make the synchronous do method of Library an asynchronous function (coroutine). So the main coroutine should look like this:

async def main():
    library = Library()

    await asyncio.gather(
        asyncio.to_thread(library.do),
        asyncio.to_thread(library.do),
        asyncio.to_thread(library.do)
    )

    print('Done.')

Here the asyncio.to_thread wraps the library.do method and returns a coroutine object avoiding the first error, but you also need await before asyncio.gather.

NOTE: If you are going to check my answer with the above example, please do not forget to set a valid URL instead of 'example.com'.

Edit

If you do not want to use threads at all, I would recommend an async wrapper like the to_async function below and replace asyncio.to_thread with that.

async def to_async(func):
    return func()


async def main():
    library = Library()

    await asyncio.gather(
        to_async(library.do),
        to_async(library.do),
        to_async(library.do),
    )