How to make a conditional proxy_pass within NGINX

Question

I am running our front and backend on K8s, this question is regarding our front-end setup using NGINX.

I am trying to find a way to gather metrics of my main front-end container (nginx + front-end) using a sidecar container that adjusts the stub_status running on /stub_status on port 8080 values to key value pairs for my application. Just so that I can scrape them using Prometheus on the sidecars /metrics on port 9113.

I want to block traffic for stub_staus outside of the pod, as only the sidecar needs to be able to reach it and I am trying to block /metrics from outside the platform (hence an example random 172 range address as an example). If I instead of a proxy pass (see below) use return 444 or 404 I get a nice big fat error. However, our front-end can handle 404 in a nice (graceful) way making it so you do not even exit the front-end but simply get a user friendly 404 message whilst staying in the front-end application, and end up on location/404. This also makes it appear like there is nothing on the /stub_status or /metrics. Which is nice to have.

I tried to do a rewrite ^/metrics$ to a /404 (for example) but that simply got me an NGINX 404 instead. Maybe it has something to do with the fact that /metrics runs on 9113 and there is nothing listening tot /404 on 9113. I am not sure about this.

I know the proxy pass example below is not possible within an IF statement, as I get the following error "proxy_pass" cannot have URI part in location given by regular expression, or inside named location, or inside "if" statement, or inside "limit_except" block in". However, below is to illustrate what I am trying to do. I am just not sure how to get it to behave like this.

Thanks in advance!

HTTP Block

http {

  #Block traffic from outside platform for /stub_status
  map $remote_addr $block_outside_pod {
      default               1;
      127.0.0.1             0;
  } 

  #Block traffic from outside platform for /metrics
  map $http_x_forwarded_for $block_outside_platform {
      default               1;
      ~^172\.30\.\d*.\d*$   0;
  }

}

Server Block (What I am trying to accomplish)

server {

    location /stub_status {
      #sidecar only
      if ($block_outside_pod = 1) {
        proxy_pass http://localhost:8080/404; 
      }
      stub_status;
    }  

    location /metrics {
      #platform only
      if ($block_outside_platform = 1) {
        proxy_pass http://localhost:8080/404; 
      }
    }   
} 

Answer 1

After more testing I found out the following works really well:

HTTP BLOCK

http {

  #Block traffic from outside platform for /stub_status
  map $remote_addr $block_outside_pod {
      default               1;
      127.0.0.1             0;
      ::1                   0;
  } 

  #Block traffic from outside platform for /metrics
  map $http_x_forwarded_for $block_outside_platform {
      default               1;
      ~^172\.30\.\d*.\d*$   0;
  }

}

SERVER BLOCK

 server { 
    listen       8080 default_server; 
    server_name  _; 
    root         /<insert your location>; 
 
    error_page 418 = @block_traffic; 
    location /stub_status { 
      #sidecar only 
      if ($block_outside_pod = 1) { 
        return 418; 
      } 
      stub_status; 
    }   
 
    error_page 418 = @block_traffic; 
    location /metrics { 
      #platform only 
      if ($block_outside_platform = 1) { 
        return 418; 
      } 
    }     
 
    location @block_traffic { 
      rewrite ^ /404$1; 
      proxy_pass http://localhost:8080;  
    }    
 
    location / { 
      index.html etc...
      add_header etc...
    } 

This means that anything coming in on 8080 via k8s to this pod from outside the platform will get a nice fancy 404 page from our website making it seem like it is not there =)

Thanks for the headsup from the person in the NGINX Slack!