I'm using docker swarm for deploying multiple services, which communicate between them via gRPC. Every service has multiple replicas, and I'm using client side load balancing to talk to them equally. I'm passing the services domain names as an environment variable like this: service-name-1,service-name-2,service-name-3 because if I rely on dockers default load balancer, I would only connect to one replica, as the channel remains open.
The problem I'm facing is that it's not very convenient to copy and paste the service (service-name-4) on my docker compose, in order to create a new replica. It would be better if I could just scale the services when needed with: docker service scale service-name=4. What I wanted to know is if there is a way of differentiating individual replicas in docker swarm.
I'm aware about the "look-aside server" for load balancing gRPC recommments. I just want to know if there's any straight forward solution to this problem.
If the number of clients is much smaller than the number of replicas, consider using a look-aside server. Otherwise, there's no problem leaving the job to docker and just increase the replica count like specified above.
I ended up using a look-aside server for my specific use case. However, what I described on the post above, wasn't such a big issue after all. Let me explain:
Let's say that there's service A, which talks to service B via gRPC. The issue would only exist if the number of clients of A is smaller than the number of available B replicas. For example: 2 replicas of A and 4 of B. It this were the case, A1 would connect (and keep an open connection) to B1, A2 would connect to B2, leaving B3 and B4 on idle.
On the other hand, if the number of clients A is greater or equal than the number of available B replicas, the services would distribute the load according to the load balancing strategy efficiently. For example 4 A replicas, and 2 B replicas. In this case, B1 would have open connections with A1 and A2; while leaving the rest to B2.
An argument could be made about dropped connections leaving uneven distribution on B services, however, that greatly depends on the load balancing strategy.