Search code examples
python-3.xpython-requestspython-asyncioaiohttp

Associating aiohttp requests with their responses


Very simply, I would simply like to associate responses from aiohttp asynchronous HTTP requests with an identifier, such as a dictonary key, so that I know which response corresponds to which request.

For example, the function below calls URIs suffixed with the dict values 1, 2 and 3. How could it be modified to return the the keys associated with each result? I just need to be able to keep track of which request is which… Doubtless trivial for someone familiar with asyncio

import asyncio
import aiohttp

items = {'a': '1', 'b': '2', 'c': '3'}

def async_requests(items):
    async def fetch(item):
        url = 'http://jsonplaceholder.typicode.com/posts/'
        async with aiohttp.ClientSession() as session:
            async with session.get(url + item) as response:
                return await response.json()

    async def run(loop):
        tasks = []
        for k, v in items.items():
            task = asyncio.ensure_future(fetch(v))
            tasks.append(task)
        responses = await asyncio.gather(*tasks)
        print(responses)

    loop = asyncio.get_event_loop()
    future = asyncio.ensure_future(run(loop))
    loop.run_until_complete(future)

async_requests(items)

Output (abbreviated):

[{'id': 2, ...}, {'id': 3, ...}, {'id': 1...}]

Desired output (for example):

{'b': {'id': 2, ...}, 'c': {'id': 3, ...}, 'a': {'id': 1, ...}}

Solution

  • Pass keys to fetch(), to return them with the corresponding response:

    #!/usr/bin/env python
    import asyncio
    import aiohttp  # $ pip install aiohttp
    
    async def fetch(session, key, item, base_url='http://example.com/posts/'):
        async with session.get(base_url + item) as response:
            return key, await response.json()
    
    async def main():
        d = {'a': '1', 'b': '2', 'c': '3'}
        with aiohttp.ClientSession() as session:
            ####tasks = map(functools.partial(fetch, session), *zip(*d.items()))
            tasks = [fetch(session, *item) for item in d.items()]
            responses = await asyncio.gather(*tasks)
        print(dict(responses))
    
    asyncio.get_event_loop().run_until_complete(main())