Search code examples
dockerperformancenginxload-balancing

Very slow performance with Nginx and Traefik

We have the following infrasctuture to deploy a PHP application:

  • A VM running Nginx as a proxy
    • 3 nodes in a Docker Swarm cluster
      • Traefik containers deployed in this cluster (so we have 3 containers running Traefik, 1 per node)

So, when you try to access the application, you hit our Nginx VM, which has a load balancer that chooses one of those 3 nodes and send your request there.

As you hit a node, one of those Traefik containers will redirect you to the application container. The deployment application process generates a container per node, so we have 3 containers running our application and Traefik chooses one of them.

PROBLEM: this structure is TOO slow.

We have another structure, only with a Nginx proxy and Docker Swarm with one node. We hit our application running there in 80ms.

In our new structure which has load balancing and very nice features of DevOps, we hit our application in 2 seconds.

Any ideas on how we could improve it? When we set Nginx proxy to take the request to a specific node, we got 450ms as response time, which is still slow.

Solution

  • The problem was with the container OS: Alpine Linux.

    They have a Gitlab issue about it: https://gitlab.alpinelinux.org/alpine/aports/-/issues/14381

    And there is a post about this slow performance: https://www.linkedin.com/pulse/testing-alternative-c-memory-allocators-pt-2-musl-mystery-gomes/

    We changed our container to use a Debian image and now we have a normal performance.