Search code examples
elasticsearchpython-3.5aiohttp

aiohttp download only first n-bytes of body


We are using aiohttp to post data into elastic search server. Elastic on such insertions generates response for each inserted line, which results in massive unwanted traffic coming back to client application. We wanted to get around this problem using following code

response = await http_session.request("POST", url, data = data, params = params)
first_n_bytes = (await response.content.read(n_bytes)).decode("utf-8")
response.release()
# response.close()

First we tried release method, but from documentation and from bandwidth measurements it seems to also download the whole content. Then we tried response.close() but we are quite unsure whether this is safe thing to do while maintaining the same http_session for other requests.

The question is whether response.close() is safe and whether it would even solve our problem, or alternatively whether there is some other way of doing it asynchronously.


Solution

  • Yes, calling resp.close() is safe. It closes opened connection to server without reading the response tail.

    Obviously keep-alives are not supported with explicit connection closing, what's why resp.release() is recommended for default usage.

    But in you case resp.close() should work pretty well.