Search code examples
bashhttpcurltelnetnetcat

Shell script - Force HTTP 1.1 pipelining in curl, nc or telnet

I need to force HTTP pipelining (1.1) on serveral GET requests with curl, telnet or netcat on a bash script. I've already tried to do so with curl, but as far as I know the tool has dropped HTTP pipelining support since version 7.65.0, and I wasn't able to find much information about how to do so. Still, if with telnet or netcat couldn't be possible, I have access to curl version 7.29.0 in other computer.

Solution

  • From Wikipedia:

    HTTP pipelining is a technique in which multiple HTTP requests are sent on a single TCP (transmission control protocol) connection without waiting for the corresponding responses.

    To send multiple GET requests with netcat, something like this should do the trick:

    echo -en "GET /index.html HTTP/1.1\r\nHost: example.com\r\n\r\nGET /other.html HTTP/1.1\r\nHost: example.com\r\n\r\n" | nc example.com 80

    This will send two HTTP GET requests to example.com, one for http://example.com/index.html and another for http://example.com/other.html.

    The -e flag means interpret escape sequences (the carriage returns and line feeds, or \r and \n). The -n means don't print a newline at the end (it would probably work without the -n).

    I just ran the above command and got two responses from this, one was a 200 OK, the other was a 404 Not Found.

    It might be easier to see the multiple requests and responses if you do a HEAD request instead of a GET request. That way, example.com's server will only respond with the headers.

    echo -en "HEAD /index.html HTTP/1.1\r\nHost: example.com\r\n\r\nHEAD /other.html HTTP/1.1\r\nHost: example.com\r\n\r\n" | nc example.com 80

    This is the output I get from the above command:

    $ echo -en "HEAD /index.html HTTP/1.1\r\nHost: example.com\r\n\r\nHEAD /other.html HTTP/1.1\r\nHost: example.com\r\n\r\n" | nc example.com 80
HTTP/1.1 200 OK
Content-Encoding: gzip
Accept-Ranges: bytes
Age: 355429
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Mon, 24 Feb 2020 14:48:39 GMT
Etag: "3147526947"
Expires: Mon, 02 Mar 2020 14:48:39 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (dna/63B3)
X-Cache: HIT
Content-Length: 648

HTTP/1.1 404 Not Found
Accept-Ranges: bytes
Age: 162256
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Mon, 24 Feb 2020 14:48:39 GMT
Expires: Mon, 02 Mar 2020 14:48:39 GMT
Last-Modified: Sat, 22 Feb 2020 17:44:23 GMT
Server: ECS (dna/63AD)
X-Cache: 404-HIT
Content-Length: 1256

    If you want to see more details, run one of these commands while wireshark is running. The request is sent in No. 7 (highlighted) and the two responses are received on No. 11 and No. 13. screenshot of wireshark