Search code examples
ubuntunginxluaload-testingconnection-refused

Load Testing: unstable error "Connection refused"


Enviroment:

  • Ubuntu 18.04.1, DigitalOcean: RAM 2Gb + 2CPU

  • Nginx (OpenResty) + Lua

What I'm doing:

I tried to send simple POST request like this by HTTPS: (this is format of YandexTank)

[Host: myserver.com]
[Content-Type: application/x-www-form-urlencoded]
80 /api
xxxxxx=777777777&yyyyyyy=666666666&zzzzzzzz=ce96c7c3-87e5-4b90-b1e5-9664697aa5b6

Schedule is: 100RPS during 30 seconds.

What I see:

First ≈ 1700 - 1900 request is success with status 200. But then... about 99% error requests with response: "111 Connection refused" maximum 1% success requests with code 200

If I wait 30 seconds and restart the load test, I won't have even 1600 success requests! The first requests already will be with error "111 Connection refused".

If I wait 10 minutes and restart the load test, the first case will repeat (1700 - 1900 - success, other - wrong).


MY QUESTION:

Any suggestions, which params should I configure for fix the error? It the problem in NGINX config? or OS?

The below numbers I was changing differently (worker_processes, worker_rlimit_nofile, worker_connections, keepalive_timeout).

Thank you!


CONFIGS:

Nginx

user  www-data;
worker_processes  4;

error_log  logs/error.log;
error_log  logs/error.log debug;

worker_rlimit_nofile 33000;

events {
    worker_connections  4000;
    use epoll;
    multi_accept on;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    reset_timedout_connection on;
    sendfile        on;
    keepalive_timeout 15;

    include '/etc/nginx/sites-enabled/*';
}

File (server + location)

...
    location ~ ^/api {
        content_by_lua '
            ngx.say("OK")
            return ngx.exit(200)
        ';
    }
...

ulimit -n

200000

Solution

  • The problem was in my home router

    It could not cope with such a load

    The problem was solved by replacing the router