I have the following set up:
public network This network has internet access as the route tables routes the external traffic to a IGW I am deploying a small sample app webAppPublic in this network
private network A - no internet access
private network B - no internet access
Note network A and network B are in two different AZs
In private network A and network B I deployed a small sample app webAppPrivate used by webAppPublic so one instance per network.
I am using a internal load balancer for webAppPrivate I linked the load balancer to network A and network B and put the two instances as target group.
webAppPublic uses the dns name of the load balancer to communicate to webAppPrivate
This configuration works fine:
Furthermore I see in the console that if I call dig load-balancer-dns
it resolves to two IP addresses for each subnetwork namely private network A and private network B
So now I also have two load balancers node for each private network.
I also note that if I call dig multiple times I see the the order of the addresses changes so I guess AWS does this for dns load balancing among the load balancers instances on the network A and B
I asked my self the question, what should happen if I remove one instance of webAppPrivate ? In this case one of the load balancer would not find a instance in the same network and AZ to forward the request to, so most likely the request will fail.
However after doing this I notice the work balancing still works fine, as I don't get any issues and the one instance remaining is always reached.
I am not sure how this happens, if the DNS query resolves the dns name to the load balancer instance in the network and AZ from which I remove the instance I would expect this request to fail.
One possible explaination is that the instance of load balancer will, in case no instances are up and or healthy in its own network, contact AWS load balancer service elsewhere / or possibly the other instances of the laod balancer in different networks, before routing the actual traffic. So the node of load balancer in one cannot network cannot take the routing decision on its own ?
I think my mental model about internal load balancing and private network is somewhat wrong, can you help me understand what happens here ?
It looks like you are using Application Load Balancer (ALB).
Cross-zone load balancing is the default configuration for ALB. That means that each load balancer node distributes traffic across the registered targets in all enabled Availability Zones. When cross-zone load balancing is disabled, each load balancer node distributes traffic only across the registered targets within the same AZ as the load balancer is deployed.
As of late 2022, you can now disable cross-zone load balancing.
See more at the cross-zone load balancing section of How Elastic Load Balancing works.