Parsing the verbose output of curl (e.g.):
>
* STATE: DO => DO_DONE handle 0x60002e090; line 1281 (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0x60002e090; line 1407 (connection #0)
* STATE: WAITPERFORM => PERFORM handle 0x60002e090; line 1420 (connection #0)
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 200 OK
< Date: Mon, 08 Sep 2014 16:34:30 GMT
* Server Apache/2.2.3 (Win32) mod_ssl/2.2.3 OpenSSL/0.9.8c mod_jk/1.2.19 PHP/5.2.7-dev is not blacklisted
< Server: Apache/2.2.3 (Win32) mod_ssl/2.2.3 OpenSSL/0.9.8c mod_jk/1.2.19 PHP/5.2.7-dev
< Set-Cookie: JSESSIONID=B256C7DA85AF756B86252810830C9284; Path=/hcs; Secure
< Transfer-Encoding: chunked
< Content-Type: text/html;charset=ISO-8859-1
<
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0{ [data not shown]
* STATE: PERFORM => DONE handle 0x60002e090; line 1590 (connection #0)
I need to generate a list w/ the HTTP status code and headers w/ the following format:
:STATUS: 200
Date: Mon, 08 Sep 2014 16:34:30 GMT
Server: Apache/2.2.3 (Win32) mod_ssl/2.2.3 OpenSSL/0.9.8c mod_jk/1.2.19 PHP/5.2.7-dev
Set-Cookie: JSESSIONID=B256C7DA85AF756B86252810830C9284; Path=/hcs; Secure
Transfer-Encoding: chunked
Content-Type: text/html;charset=ISO-8859-1
the sed line I created was:
cat $result | sed -rn '/^< /!d; s/^<\s+//; /^$/d; /:/!{s/\S+\s+//; s/\s.*//; s/^/:STATUS: /; h}; /:/H; ${x;p}'
but all I get is an empty result... any idea?
jose@DESKTOP-72
$ cat io.txt | sed -rn '/^< /!d; /^$/d; /:/!{s/\S+\s+//; s/\s.*//; s/^/:STATUS: /; h}; /:/H; ${x;p}'
jose@DESKTOP-72
I'm running this script under Cygwin 1.7.30(0.272/5/3) over MS Windows 8.
Thanks in advance, José
Not a sed expert and don't have the time to figure this out more but sed -r '/^< /!d; s/^<\s+//; /^$/d; /:/!{s/\S+\s+//; s/\s.*//; s/^/:STATUS: /}' seems to do what you want (for that example input at least).
Actually, while typing that (and because of some other tests I was running) I think I understand the problem. Your $ address never matches because you delete the last line before that address ever gets a chance to fire. Move it to the beginning of the sed script and it prints out data. (It doubles the first line but it prints it all out.)
Does the output from curl -s -D- -o/dev/null suit your needs as-is?
If not curl -s -D- -o/dev/null "$SITE" | awk 'NR==1{print ":STATUS:",$2}7 seems like what you are asking for.