How to scale the service better: Docker swarm service replication or use uvicorn workers
Colleagues, hello everyone!
I have a large web application on a microservice architecture. The entire infrastructure works in Docker Swarm.
All microservices are worked to Python and run on a uvicorn server.
Uwsgi has its own workers which can be scaled.
In Docker Swarm, you can also scale the application by increasing the replicas of the service.
The question is how to scale the service better:
1) 1 docker swarm replicator - 100 (any other number) uvicorn workers
2) 100 (any other number) replicas of docker swarm - 1 uvicorn worker
3) 10 replicas of docker swarm - 10 uvicorn workers
It depends on your application and your requirements/capabilities, so it's hard to say exact answer. The Docker replicas and Uvicorn workers are on different levels in the micro-service architecture.
The Docker handles the traffic (load) between the replicas and inside the replicas (containers) the Uvicorn handles the processes/requests between the workers.
Replica and Unicorn architecture (very high level):
- 1 docker swarm replicator - 100 (any other number) uvicorn workers
- In this case the Docker (container) can be overloaded because all traffic is handled by one replica (container). It might lead to resource contention, especially if the workers are resource-intensive, as they share the same container resources.
- 100 (any other number) replicas of docker swarm - 1 uvicorn worker
- If there is no load on the system, the this solution can be over kill. Furthermore, the default Docker overlay sub-net is limited how many containers can be started in the network (256), and the stopped containers are also counted. Furthermore the Unicorn can handle more workers so it's not logical to limit only 1 worker. BUT each instance operates independently, which can help in scenarios where one instance might become unresponsive or experience issues without affecting others. This can improve fault tolerance and stability.
- 10 replicas of docker swarm - 10 uvicorn workers
- I think this is the most efficient configuration. In this case the Docker part wouldn't be overloaded and wouldn't be over kill in case of low load. The Unicorn also can handle more processes/requests with this configuration. Ir you calculate a bit:
10 replicas * 10 Unicorn = 100 parallel request, so it's a quite good number for parallel request processing. This approach could be suitable for applications that benefit from a mix of parallel processing and independent instances.