NGINX - Error 502 unless User-Agent header is set

Question

I have a very strange issue with NGINX.

We're routing to different servers based on the URL. This all works completely fine. Using a browser, everything works great.

The weird thing is when I try to make a GET request to the API from a tool (like Postman, Insomnia, Curl, etc.) If the User-Agent header is not spoofed to a browser, I get a 502 Bad Gateway error.

Why is this happening? I need to call this API programmatically from another back-end, but I must set the User-Agent header otherwise it will fail. (Setting the user-agent to "curl" does not work browser)

Here's my NGINX config

    server {
        listen 443 ssl;
        server_name 10.10.10.10;
        ssl_certificate /https/10.10.10.10.crt;
        ssl_certificate_key /https/10.10.10.10.key;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers HIGH:!aNULL:!MD5;

    access_log              /var/log/nginx/10.10.10.10-access.log;
    error_log               /var/log/nginx/10.10.10.10-error.log error;

        location /abc {
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header Host $http_host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Frame-Options SAMEORIGIN;
                proxy_pass http://10.3.0.2:80;
        }

        location /def {
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Frame-Options SAMEORIGIN;
                proxy_pass http://10.3.0.3:80;
        }
}

Requests from the browser work completely fine.

Request from Insomnia WITH User-Agent spoofed (works great!)

Request from Insomnia WITHOUT User-Agent spoofed (error 502!?)

• Note: I'm setting the cookies in this request as they're used by the server on the back-end. Not sure if this is related to the user-agent issue...

Thank you!

Answer 1

Sorry. Found it. Rookie mistake on my end! It was not a problem with nginx at all.

The Flask server actually had a line to serve a "change your browser" message if the user wasn't on Chrome, to avoid browser compatibility issues. Unfortunately, the browser_splash file was deleted, so it just wasn't serving anything in that case. Didn't notice that this got committed:

if request.user_agent.browser != 'chrome':
  return render_template('browser_splash.html')