Search code examples
amazon-web-servicesamazon-ec2portamazon-ecs

How to have AWS ECS automatically map ports of my container to the host machine(EC2)

Context:

I am using Circle CI's aws-ecs/deploy-service-update orb to deploy my docker container by pulling the latest image in AWS ECR and deploy it in AWS ECS with AWS EC2 instance. This container is a Machine Learning model that accepts API requests at TCP port 3000(I am using fastAPI for this) and returns the predictions. After I deployed it I couldn't send requests to the public IP of the container instance of the task that deploys the container at port 3000 (This IP is not my EC2 instance's public IP; it only has private IP and public IP is disable).

Debugging

  1. I checked my security group and made sure that the port 3000 is open to receive requests from all IPs(0.0.0.0), as part of the inbound rule.
  2. I stopped the task(which automatically will stop the container running in the EC2 instance) with the thought that something may have gone wrong from Circle CI. Then, according to the service configuration(1 desired task) and task definition of AWS ECS, a new task has started(hence the container) automatically. But, I couldn't send requests to this either.
  3. I SSHed into my EC2 instance to know if the port 3000 is open. This is when is when I learned that ports weren't mapped at all: enter image description here
    As you can see, PORTS column is empty for the container and the container has to accept requests at port 3000 from the command.

And here are the open ports of the EC2 instance: enter image description here As you can see, port 3000 is not listed here.

Here is the task with port mappings which deployed the container (to AWS ECS) that you see docker ps screenshot above: enter image description here
In the task definition, you can see the port mappings I have defined for the container.

Here is the task running on my EC2 instance with the task-definition shown above and the network mode I am using is 'awsvpc': enter image description here

Here's the "Networking" tab of ENI associated with the task, and also the inbound rule of the security group associated with the EC2 instance that the task is running inside, which accepts requests on port 3000 from all IPs. enter image description here

EDIT 1:

After I did

docker run -p 3000:3000 <my-image:my-tag>

inside the EC2 machine(by SSHing from my laptop), I could send API requests and receive proper response to the container to it's public IP, of the cluster of AWS ECS. This means that ports are being mapped only when I run the container manually.

I had no problems with ports when I used FARGATE, when I updated the service from Circle CI or even when I manually started tasks.

So, how to automatically map ports when a task is run from AWS ECS service dashboard or from Circle CI? If I run docker container manually, I will not be able to get logs automatically from AWS Cloudwatch and will not be able to stop it from AWS ECS dashboard. Another container by AWS that is running in EC2 instance will take care of those things. It will route the logs to Cloudwatch and accepts stop the existing one and start commands to start a new container with new image stored in AWS ECR, without having to SSH everytime I would want to look at logs or start/stop containers.

What has gone wrong here, which led to ports not being mapped and How do I fix it and map ports properly, so i will be able to send API requests to my container.

Solution

  • The mistake I had done was to use FARGATE optimised cluster and an EC2 instance inside it; which presumably had led to no network mode(default, bridge, host, awsvpc and none) work for my application. So, when I redid everything same as last time, but with EC2 optimised cluster type, bridge network mode worked just as expected.