nginx x-accel block conflict

Question

X-Accel-Redirect seems not to honor nginx block configuration rules and precedents. consider this example:

server {
   ...
   limit_conn limitedIP 10;

   location ^~ /files/ {
      internal
      alias /var/www/files/;
      limit_conn limitedIP 20;
   }

   location ~ \.php$ {
      ...
      limit_conn limitedIP 30;
   }
}

If I access /files directly (while removing internal ofcourse) the limit_conn 20 rule is used which is normal.

but If I use X-Accel-Redirect in a php script to /files the limit_conn 30 rule is used. even if I remove that line from php location block the limit_conn 10 rule from server block is used which is very odd. finally if I remove limit_conn 10 from server block limit_conn 20 rule is used which is I was expecting at first.

I tested this on nginx version 1.6.2.

limit_conn is just an example and several directives behave like this. any ideas?

Answer 1

This is just how limit_conn and limit_req modules work. These modules deal with connections between users and the server, and since internal redirects do not open any new such connections, Nginx only checks the limits once per request and does not consider it necessary to check them again.

In your example, user's request first matches the second location, which triggers the the limit_conn handler to be called for the first time. This is why the limit_conn 30 rule is used. The handler will be called the second time after the internal redirect, but it will end its work almost immediately without checking anything (return NGX_DECLINED).

If you comment the limit_conn 30 rule in the second location, the limit_conn 10 rule will be inherited from the server context, which is perfectly normal behavior for Nginx. In this case, again, the limit_conn handler will first be called for the main request, and the next call (after the internal redirect) will do nothing.

Finally, when you comment all limit_conn lines except for the limit_conn 20 rule, the limit_conn handler will first be called after the internal redirect and thus this rule will ultimately be used.

UPDATE:

Since the limit_conn rule is checked when a new connection is being established, opening a new connection will solve your problem. Conceptually, the configuration would look like this:

limit_conn_zone $binary_remote_addr zone=limitedIP:10m;

server {
    # You can use any random unused non-system port
    listen 127.0.0.1:8889;
    server_name internal.server;

    set_real_ip_from  127.0.0.1/32;
    real_ip_header    X-Forwarded-For;

    location ^~ /files/ {
        limit_conn limitedIP 20;
        alias /var/www/files/;
    }
}

server {
    ...

    location ^~ /files/ {
        internal;
        proxy_set_header Host "internal.server";
        proxy_set_header X-Forwarded-For $remote_addr;

        # Make sure you specify the same port as above
        proxy_pass http://127.0.0.1:8889;
    }

    location ~ \.php$ {
        ...
        limit_conn limitedIP 30;
    }
}

However, this approach will result in marked performance degradation, especially if your files are huge in size. Therefore, I would not recommend using this workaround.

Unfortunately, I cannot imagine any other way to solve the problem with Nginx only.