I have an NGINX server serving a web application where customers have their domains (CNAMES) that they access their websites.
Does NGINX have any way to limit the number of accesses to one of those domains for a period of time?
Example:
Restriction that I need: 2000 requests / domain / minute
So, in one specific period of time...
www.websiteA.com.br --- 1456 requests / minute OK!
www.websiteB.com.br --- 1822 requests / minute OK!
www.websiteC.com.br --- 2001 requests / minute LOCKED TEMPORARILY
Does anyone know how to make such a restriction?
I am using the rate-limiting features of nginx to define a number of rate limit rules and then apply these rules to particular hosts.
There are a few hacks in here (if-statements are used as well as redirect via error_page handlers) - I have not found
a better way of doing it yet. Please comment if you have better alternatives.
http {
# ...
# Define rate-limit key to use
map $http_authorization $rl_zone_key{
default $binary_remote_addr;
'~*.*' $http_authorization;
}
# Define Rate Limits
limit_req_zone $rl_zone_key zone=rateLimit_Standard:10m rate=1r/s;
limit_req_zone $rl_zone_key zone=rateLimit_Class_A:10m rate=5r/s;
limit_req_zone $rl_zone_key zone=rateLimit_Class_B:10m rate=10r/s;
limit_req_zone $rl_zone_key zone=rateLimit_Class_C:10m rate=100r/s;
# Define the rate Limit Category applied to the particular host
map $http_host $apply_rate_limit{
hostnames;
default rateLimit_Standard;
example.com rateLimit_Class_B;
*.example.com rateLimit_Class_A;
}
#upstream server definition for php fast-cgi using port 9000
upstream phpfcgi {
server 127.0.0.1:9000;
}
# ...
}
The key that is used to compare whether two requests are coming from the same place, uses the $binary_remote_addr nginx variable
unless an $http_authorization header exists, in which case it will use that. This means that until a user is authenticated (using digest auth),
rate limits are applied by ip - thereafter it is applied by logged-in user session.
Here I set the rate limit caches.
With the map directive, I check the hostname in $http_host to determine the target domain and then define $apply_rate_limit with the name of the rate limit class I want to apply.
server {
listen 80;
server_name example.com;
root /var/www/example.com;
location / {
index index.php;
try_files $uri =404;
location ~ ^/index\.php(/|$) {
error_page 420 =200 @rateLimit_Standard;
error_page 421 =200 @rateLimit_Class_A;
error_page 422 =200 @rateLimit_Class_B;
error_page 423 =200 @rateLimit_Class_C;
if ( $apply_rate_limit = 'rateLimit_Standard' ) {return 420;}
if ( $apply_rate_limit = 'rateLimit_Class_A' ) {return 421;}
if ( $apply_rate_limit = 'rateLimit_Class_B' ) {return 422;}
if ( $apply_rate_limit = 'rateLimit_Class_C' ) {return 423;}
}
}
location @rateLimit_Standard {
limit_req zone=rateLimit_Standard burst=5;
add_header X-Rate-Limit-Class $apply_rate_limit;
include fastcgi_params;
fastcgi_index index.php;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param HTTPS off;
fastcgi_pass phpfcgi;
}
location @rateLimit_Class_A {
limit_req zone=rateLimit_Class_A burst=10 nodelay;
add_header X-Rate-Limit-Class $apply_rate_limit;
include fastcgi_params;
fastcgi_index index.php;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param HTTPS off;
fastcgi_pass phpfcgi;
}
location @rateLimit_Class_B {
limit_req zone=rateLimit_Class_B burst=100 nodelay;
add_header X-Rate-Limit-Class $apply_rate_limit;
include fastcgi_params;
fastcgi_index index.php;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param HTTPS off;
fastcgi_pass phpfcgi;
}
location @rateLimit_Class_C {
limit_req zone=rateLimit_Class_C burst=1000;
add_header X-Rate-Limit-Class $apply_rate_limit;
include external_definition_of_fastcgi_parameters.conf
}
}
In my example vhost file, I use the rate limit classes previously defined. This could be done in a more straightforward manner
but in our scenario we have a single vhost file serving multiple domains, so defining which domain uses which rate limit class is better
done outside the vhost file.
I will provide a simplified example vhost file at the end that removes this abstraction and makes the vhost rate limit application more manual.
In nginx there is no convenient way to redirect to a named location outside of the try_files directive. A hack to get this done is to
define an error handler (for an error code not in common use) and then provide the named location as the error_page handler.
In this particular line, we are saying that when error code 420 is triggered, the error handler should redirect to the @rateLimit_Standard
location and reset the HTTP code to 200.
In our nginx.conf file, $apply_rate_limit was defined based on the $http_host header. If it was set to rateLimit_Standard, this if statement will complete by
returning from the http request with error code 420. This will be caught by the error_page handler we defined earlier and will be redirected to the named location @rateLimit_Standard
Once the request has been rerouted to @rateLimit_Standard, it will apply the rate limit and set the burst value: limit_req zone=rateLimit_Standard burst=5;
It will then proceed to process the php request as per normal.
For good measure I also add a header to track the applied rate limit: add_headerX-Rate-Limit-Class $apply_rate_limit;
You will notice for each of the named locations, the fcgi headers included are identical. There is no formal inheritance in nginx config
blocks, so this has to be repeated for each location in order forward this to php-fpm.
One can, however, define the common properties in an external file, and use the include statement above to fetch all those duplicate config options from
a single external file. I left this here because this is what you really should do.
The above example.conf file shows fully how I abstracted the host <-> rate limit class pairing as we are using it in our environment.
It can be a lot simpler if you are only using a single vhost entry for a single domain:
server {
listen 80;
server_name simple-example.com;
root /var/www/simple-example.com;
location / {
index index.php;
try_files $uri =404;
location ~ ^/index\.php(/|$) {
limit_req zone=rateLimit_Class_A burst=10 nodelay;
add_header X-Rate-Limit-Class rateLimit_Class_A;
include fastcgi_params;
fastcgi_index index.php;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param HTTPS off;
fastcgi_pass phpfcgi;
}
}
}
Disclaimer: All the code above was constructed from my own nginx config for the purpose of creating an example. This will probably not work out of the box and you will need to massage these snippets into your environment to get them working properly.
Please vote if you found this useful.