Does python numpy support asyncio

Question

I need to move sync code includes numpy methods to async code. I tried to find documentation for it but found nothing.

The methods I'm using are:

array
datetime64
where

Will the process continue and not be stuck when I'll add 'await' to those methods?

I looked at some questions related to it but i'm not sure there are related/updated.

python multiprocessing with async shared numpy array: pool vs queue

np.unique blocks CPU with asyncio.to_thread

Answer 1

I agree with @freakish You should read:

• multiprocessing vs multithreading vs asyncio
• Understanding python GIL - I/O bound vs CPU bound
• useful article 1
• useful article 2

The idea behind async code is to suspend execution and wait for results from an external source (e.g., a file, a database, a network). This approach benefits your code and reduces the time needed to produce multiple results because the CPU typically stops and does nothing while waiting. If you need resources from a network, your code will request them one at a time, even though the CPU is not affected by these requests. With asynchronous requests (functions), you can request all the resources simultaneously, allowing the CPU to perform other tasks in the meantime.

Numpy does not support asynchronous tasks. But even if it did, would it make any sense? I think not. Numpy performs CPU-bound tasks, so there would be no advantage. If you use Numpy methods as async functions, the CPU will still be used to calculate the result. The program will be blocked at that method until it is completed before moving to the next one.

If for any other reasons you need it, as @freakish and @Leah said you need to make async every numpy function you use. A wrapper could help you. Something like this:

import functools
import asyncio
from concurrent.futures import ThreadPoolExecutor

executor = ThreadPoolExecutor()

def async_decorator(func):
    @functools.wraps(func)
    async def async_wrapper(*args, **kwargs):
        loop = asyncio.get_event_loop()
        return await loop.run_in_executor(executor, functools.partial(func, *args, **kwargs))
    
    return async_wrapper

@async_decorator
def async_function1(x, y)
    return function1(x, y)

@async_decorator
def async_function2()
    return function2()

But this code will not speed up your operations.

Consider using the multiprocessing module or a ProcessPoolExecutor