I have an ECS cluster which has an Auto Scaling group with 2 EC2 instances. Also I have 3 services, and each service has its own task definition. Each EC2 instance run one docker container per service. Therefore I have 3 docker containers in each EC2 instance.
Each docker container simply run a spring boot app. Since I have 3 services, then I have 3 spring boot apps. Again, one container only runs one of these 3 spring boot apps. Each app exposes a rest ful API, with services like POST, GET etc, under URLs like /service1 or /service1/resource1. One important point here is I'm using dynamic port mapping in the container's host.
I have an external (internet-facing) ALB on port 443, which has 3 target group. Depending on the URL, the request will go to one of the 3 apps (or containers).
My problem is sometimes app A needs to make an http request to app B. My EC2 instances live in private subnet, while my ALB lives in a public subnet. So if I use my ALB to make http requests from inside a container to another, what's going to happen is the request will go through a NAT, and since the public ip of the NAT is not part of the security group of the ALB then it can't connect on port 443. I have 2 ways to make this work:
In the security groups of the ALB whitelist 0.0.0.0/0. I don't want to do that since the entire world will have access.
In the security group of the ALB whitelist the public IP of the NAT. I'm not sure about this approach. Is it recommendable?
A third option I'm trying to implement is to have a third load balancer, an internal one. But I'm lost here, as per AWS docs you can only assign 1 load balancer to your service. And since we are using dynamic port mapping I don't see a way to create manually an ALB and use the dynamic automatically assigned port.
How do you guys make this kind of connectivity between containers, where one container consumes a service that other provides?
As last comment, I use cloud formation for all. No manual setup from console.
Thanks,
You may try to whitelist you nat gateways public IP as a host with /32 mask. It is a quite normal approach because you already have exposed endpoints to the public internet via ALB. You only need to care about security rules updates in case of destroying or changing the NAT gateway, because it's IP may change.