Adding custom headers to a response from an upstream server in nginx

I am trying to add a custom response header to the response from an upstream server in Nginx based on the response body.

For simplicity, let this custom header be the SHA1 hash of the response body. To accomplish this, I'm attempting to use the njs scripting module for Nginx.

I've referred to the examples in the njs-examples repository. However, those examples treat the headers and body sections separately, and I'm struggling to combine them to achieve my goal.

Here is my current configuration:

# nginx.conf

load_module modules/ngx_http_js_module.so;

events {}

http {
    js_path "/etc/nginx/njs/";
    js_import main from hello.js;  

    # Configuration containing list of application servers
    upstream app_servers {
        server flask:5000;
    }

    server {
        listen 80;
        server_name localhost;

        location / {
            js_body_filter main.hello;
            proxy_pass http://app_servers/;
        }
    }
}

# hello.js

function hello(r, data, flags) {
    var val = data.length;
    ngx.log(1, val);
    r.headersOut["X-Hello"] = val;
    r.sendBuffer(data, flags);
}

export default { hello };

However, when I send a request to my Nginx server, I don't see the X-Hello header in the response.

Is there a way to achieve my use case using njs scripting in Nginx? If not, what alternative approaches should I consider, such as implementing a custom Nginx module? Any suggestions or guidance on how to proceed would be greatly appreciated.

PS: I am running this setup on a Docker container with the official nginx image with some script for hot-reloading. If needed, I can share the Dockerfile and docker-compose.yml as well.

Solution

Answered by Liam Crilly on Nginx community slack.

This probably won’t work, because NGINX will have sent the response headers to the client before reading the entire body. So by the time you’ve read the response body it will be too late to modify the headers.

You can try using a js_header_filter function for that (to use res.length) but as I said, the headers are already sent before the body is received. I don’t think it will work

Thinking again about this. You could do it with Trailers so long as the client accepts it.

Use js_header_filter to switch to chunked transfer encoding, delete the content-length header, and define a trailer.

Use js_body_filter to collect and hash the body, send the body, and then send the hash in the trailer we defined above.

OK, so I couldn’t let this drop :lolsob: Here’s a solution that adds a SHA-1 hash of the response as a trailer. Not sure if that solves your problem, @Gaurav Jain, but I had fun finding out!

nginx.conf snippet

js_import conf.d/body.js;
js_set $body_hash body.get_hash;

server {
    listen 80;
    location / {
        proxy_pass http://localhost:9001;
        js_body_filter body.set_hash;
        add_trailer Body-Hash $body_hash;
    }
}

server {
    listen 9001;
    root /usr/share/nginx/html;
}

body.js

var hash = "";
var res = "";
var buf = 0;
function set_hash(r, data, flags) {
    if (data.length) buf++;
    res += data;      // Collect the entire response,
    if (flags.last) { //  until we get the last byte.
        try {
            hash = require('crypto').createHash('sha1').update(res).digest('base64');
            r.sendBuffer(res, flags);
            ngx.log(ngx.INFO, `FILTERED ${res.length} bytes in ${buf} buffers`);
        } catch (e) {
            ngx.log(ngx.ERR, `ERROR ${e}`);
            r.sendBuffer("", flags);
        }
    }
}

function get_hash() {
    return hash;
}

export default { set_hash, get_hash }

Test

curl -i localhost
HTTP/1.1 200 OK
Server: nginx/1.25.2
Date: Wed, 13 Sep 2023 21:38:25 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
Last-Modified: Tue, 15 Aug 2023 17:03:04 GMT
ETag: "64dbafc8-267"
Accept-Ranges: bytes

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
Body-Hash: xRo/Dm3k64AtVjCUHD/Z4dDvrks=