how to enable gzip compression in libwebsocket on ESP32

Question

I am running a webserver on an ESP32 chip using the libwebsocket library. The server files are in a ROMFS partition on the ESP32. Currently I am in the process of trying to improve the loading time by concatenating, minifying and compressing the javascript, html and css files. The concatenation and minification worked properly, I now only have a concatenated.js and concatenated.css file in my website. But the issue came when I tried to get the compression working.

Initially, I thought my server would compress the files by itself before sending them, however when I looked at the server file transfer using Chrome developper extension, I found out that the javascript file GET request was returned with "content-type: text/javascript".

I tried several solutions I could think of, but none seem to work:

• gzip the file before creating the romfs (ie there is now only a concatenated.js.gz in my ROMFS file system)
  • The server returns 404 when trying to access "concatenated.js"
• gzip the file before creating the romfs and make it live alongside the original file (I was thinking maybe libwebsocket would be able to see they were both there and pick the most efficient one)
  • The server only returns the js file, and never the gz file

Does anybody knows how to enable the gzip compression in libwebsocket ? I am guessing there must be some options I don't have enabled, but it has been hard finding resources on the web. Most of them only discuss about the ability of the libwebsocket to get gzip from a zipped file.

Regards,

Answer 1

The issue ended up coming directly from the libwebsocket code.

When opening a file from the ESP32, there was no logic in place to look for a file with the same name and ".gz" at the end. The logic to look for such a file if the browser accepted gzip file needed to be added to the function.

This change was done on an older version of the libwebsocket, and as such may not apply to the latest version (for anybody looking at this modification). Also, I needed to include <string.h> to have access to the string manipulation functions:

libwebsocket/lib/plate/freertos/esp32/esp32-helpers.c -> function esp32_lws_fops_open

Replace

f->i = romfs_get_info(lws_esp32_romfs, filename, &len, &csum);

By

// check for the gzip file if gzip is allowed by the browser
f->i = NULL;
if((*flags & LWS_FOP_FLAG_COMPR_ACCEPTABLE_GZIP) == LWS_FOP_FLAG_COMPR_ACCEPTABLE_GZIP)
{
  char *filename_gz = malloc(strlen(filename) + 3 + 1); // add space for ".gz" and null termination
  sprintf(filename_gz, "%s.gz", filename);
  f->i = romfs_get_info(lws_esp32_romfs, filename_gz, &len, &csum);
}

// if we haven't found a gz file (not allowed or no gzip), search for the regular file
if(!f->i)
{
  f->i = romfs_get_info(lws_esp32_romfs, filename, &len, &csum);
}
// otherwise, add the flags to let the library knows the file transfered is a gzip file
else
{
  *flags |= LWS_FOP_FLAG_COMPR_IS_GZIP;
}