I'm currently building an embedded device that uses the open source light-weight IP package (lwIP) to host a static webserver to clients using html files and http requests. lwIP is somewhat different than a normal webserver because it is bare bones and configured to serve these files through a NOSYS declaration.
In an extremely simple scenario, I'm trying to host this file (index.html):
<!DOCTYPE HTML>
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
</head>
<body>
Hello
</body>
</html>
When I'm connected directly to the embedded device, I get headers from the webserver that are valid, like below:
Host: 192.168.168.10
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
and I can view the page no problem.
However, if I try to go through a modem that port forwards to the embedded device, I get the following error on Firefox when trying to view index.html:
The character encoding of the plain text document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range.
The character encoding of the file needs to be declared in the transfer protocol or file needs to use a byte order mark as an encoding signature.
The odd thing is the headers are still the same (just a different ip and port):
Host: 192.168.2.31:8080
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
and the response data is still the same as index.html above.
I don't understand what's causing the error, as the content type is specified in the tag.
This problem was fixed through using Tom Vs suggestion of curl -v against the server. This gave a response of Received HTTP/0.9 when not allowed.
When looking through the configuration of LWIP in lwipopts.h, LWIP_HTTPD_DYNAMIC_HEADERS was set to 0. This should be set to 1 to enable the headers to be generated from lwIP and set to the user.