How to retrieve a task from a future in python?

Question

Let say I have the following code to run multiple task in parallel.

with concurrent.futures.ThreadPoolExecutor(max_workers=connections) as executor:
    loop = asyncio.get_event_loop()
    futures = [
        loop.run_in_executor(
             executor,
             fun,
             arg
        )
        for i in range(connections)
    ]
    for result in await asyncio.gather(*futures):
        # I want to access the futures task here
        pass

Is it possible to read the the futures' task once it has been executed?

Answer 1

Is it possible to read the the futures' task once it has been executed?

In asyncio, the word task has a specialized meaning, referring to a class that subclasses Future specialized for driving coroutines.

In your code, asyncio.gather() returns results, and you also have the futures variable that contains the Future objects which can also be used to access the same results. If you need to access additional information (like the original fun or arg), you can attach it to the appropriate Future or use a dict to map it. For example:

futures = []
for conn in connections:
    fut = loop.run_in_executor(executor, fun, arg)
    fut.conn = conn  # or other info you need
await asyncio.wait(futures)
# at this point all the futures are done, and you can use future.result()
# to access the result of an individual future, and future.conn to obtain
# the connection the future was created for