Search code examples
httpchunked-encoding

HTTP Chunked transfer encoding


Thats from wikipedia:

For version 1.1 of the HTTP protocol, the chunked transfer mechanism is considered to be always and anyways acceptable, even if not listed in the TE (transfer encoding) request header field

Thats what I get from clients (Mozilla, Opera):

GET http://www.google.com/ HTTP/1.1 Host: www.google.com User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: keep-alive

Apparently there is neither Transfer-Encoding field there, nor I see any chunks (I've checked with HEX editor, no additional symbols).

I open connection as follows (Python)

socket.socket(socket.AF_INET, socket.SOCK_STREAM)

Is it lower level handling joins chunks into message? Is so, how can I know where the HTTP message ends so that I can stop reading the request and start handling it?


Solution

  • You should read the specification.

    But simply, in this case, since it's a GET, and there's not content, there's not going to be a Content-Length header. So, you stop reading when you get the empty line with just a CR/LF.

    Otherwise, you read past that blank line, and read Content-Length bytes.