aiohttp: Getting a server's response when the request status code is not 2XX

Question

I'm using aiohttp for async http requests, and I can't figure out how to get the response from a server when the request returns a 4XX error.

    async def login(self, username: str, password: str) -> None:
        ...
        async with aiohttp.ClientSession(headers=self._headers) as session:
            async with session.post(route, data=data, headers=self._headers) as resp:
                if resp.ok:
                    response = await resp.json()
                    self._headers['Authorization'] = 'Bearer ' + response['access_token']
                else:
                    response = await resp.json()
                    raise InvalidGrant(response)

Using resp.json() works just fine if the response returns a 2XX code, however when it returns a 4XX error (in this case 400), it raises a aiohttp.client_exceptions.ClientConnectionError and doesn't let me get the response that the server sent (which I need, since the server returns some sort of error message which is more descriptive than Bad Request). Is there no way to get the response with aiohttp if the request isn't a success?

Answer 1

The reason this issue is occuring is because of a side effect with response.ok. In older versions of aiohttp (3.7 and below), response.ok called response.raise_for_status(), which closed the TCP session and resulted in not being able to read the server's response anymore.

To fix this, you simply need to move response = await resp.json() above the response.ok line, that way you save the response beforehand. For example:

    async def login(self, username: str, password: str) -> None:
        ...
        async with aiohttp.ClientSession(headers=self._headers) as session:
            async with session.post(route, data=data, headers=self._headers) as resp:
                response = await resp.json()
                if resp.ok:
                    self._headers['Authorization'] = 'Bearer ' + response['access_token']
                else:
                    raise InvalidGrant(response)

This issue has been fixed in aiohttp 3.8 however: https://github.com/aio-libs/aiohttp/pull/5404