Search code examples
asynchronouspython-requestsparallel-processingpython-asyncioaiohttp

Unable to get multiple response requests on aiohttp


I am trying to pull multiple responses requests using aiohttp but I am getting an attribute error.

My code looks like this, I had to obscure it because I am using a private api.

import aiohttp
import asyncio

async def get_data(session, x):
    try:
        async with session.get(url=f'https://api.abc.com/{x}') as response:
            data = await response.json()
            return data.json()
    except Exception as e:
        print("Unable to get url {} due to {}.".format(ticker, e.__class__))

async def main(datas):
    async with aiohttp.ClientSession() as session:
        ret = await asyncio.gather(*[get_data(data, session) for data in datas])
        return ret

datas = ['x1', 'x2', 'x3', 'x4'] 
resukts = asyncio.run(main(datas))

My error looks like this,

Unable to get url <aiohttp.client.ClientSession object at 0x1013f6ee0> due to <class 'AttributeError'>.
Unable to get url <aiohttp.client.ClientSession object at 0x1013f6ee0> due to <class 'AttributeError'>.
Unable to get url <aiohttp.client.ClientSession object at 0x1013f6ee0> due to <class 'AttributeError'>.
Unable to get url <aiohttp.client.ClientSession object at 0x1013f6ee0> due to <class 'AttributeError'>.

What am I doing wrong?

I was expecting to get the response from the api in parallel or asynchronously.


Solution

  • response.json() already returns the parsed JSON data, so calling .json() on data again is unnecessary and will raise an AttributeError.

    Try like this

    import aiohttp
    import asyncio
    
    async def get_data(session, x):
        try:
            async with session.get(url=f'https://api.abc.com/{x}') as response:
                data = await response.json()
                return data  # Return the parsed JSON data directly, no need for .json()
        except Exception as e:
            print("Unable to get url {} due to {}.".format(x, e.__class__))
    
    async def main(datas):
        async with aiohttp.ClientSession() as session:
            ret = await asyncio.gather(*[get_data(session, data) for data in datas])  # Pass session and data in the correct order
            return ret
    
    datas = ['x1', 'x2', 'x3', 'x4'] 
    results = asyncio.run(main(datas))  # Correct the variable name to "results"
    

    The OP is asking for a way to return the result in a dict format: this is how you can do it :

    import aiohttp
    import asyncio
    
    async def get_data(session, x):
        try:
            async with session.get(url=f'https://api.abc.com/{x}') as response:
                data = await response.json()
                return data
        except Exception as e:
            print("Unable to get url {} due to {}.".format(x, e.__class__))
            return None
    
    async def main(datas):
        async with aiohttp.ClientSession() as session:
            ret = await asyncio.gather(*[get_data(session, data) for data in datas])
            return {datas[i]: ret[i] for i in range(len(datas))}  # Return the results as a dictionary
    
    datas = ['x1', 'x2', 'x3', 'x4']
    results = asyncio.run(main(datas))