Remove HTML header from server reply using Regular Expressions

Question

I have an ESP32 T-CALL with an integrated GSM-unit and used this as a base in Arduino IDE.

My code makes a server-call and processes the result. This code reads from the buffer and outputs it to a String. It uses a manually generated POST header, sent serially. However, I need to remove the HTTP header, leaving only the JSON.

while (client.connected() && millis() - timeout < 10000L) {
  // Print available data (HTTP response from server)
  while (client.available()) {
    char c = client.read();
    returnString += c;
    timeout = millis();
  }
}

The output comes with a complete header, like this:

HTTP/1.1 200 OK
Content-Type: application/json
Server: Microsoft-IIS/10.0
X-Powered-By: PHP/8.0.0
X-Powered-By: ASP.NET
Date: Tue, 25 Jan 2022 00:12:31 GMT
Connection: close
Content-Length: 23

{"status:":"code6"}

I used the <regexp.h> library by Nick Gammon and the Lua-reference here in order to filter out everything to tle left of the curlybrace - however, I can't seem to get it right. I figure, something like this:

char result = ms.Match ("{(%x+)"); // Find the first curlybrace and only include this and everything to the right.

Alas, using this RegEx, no match is found. I also tried splitting at \r\n\r\n, using the getValue-function here but couldn't get it to accept a double linebreak.

Any ideas on, how to remove the header, using RegEx?

Answer 1

This is not a direct answer on how to use the regex, however, if you want to skip the headers and get the payload, other than using regex, or a httpclient library that I suggested in the comment, it is not difficult to do that without using any library.

To skip the header and get the payload, you need to modify your code to find the end of the header.

// skip the http headers
while (client.connected()) {
  String line = client.readStringUntil('\n');
  if (line == '\r') break;    //if line only contain '\r', it's the end of headers
  }
}

// get the payload
String payload;
while (client.available()) {
  payload = client.readStringUntil('\n');
}

You can then using a JSON library to extract the data out from the JSON object. Or for the simple JSON object as you shown, you can do it without a library.

payload.trim();                  // remove the '\r\n' at the end
payload.replace("status:", "");  // replace "status:" with ""
payload.replace("\"", "");       // remove all the "\""
payload.trim();
Serial.println(payload);

This will print out the value of code6 in your JSON object.