Search code examples
amazon-web-servicesamazon-ecs

AWS Container with static IP

I am running a ECS task in a service. I would like to use it with a public domain name, so it needs a public static IP. I have already allocated an Elastic IP, and tried many ways how to eventually tie it with the task.

Here is what I have tried so far:

  • Run a task with a public IP - works great, only the IP is not static (task's public IP), so this solution is not acceptable.
  • Run a task without the public IP - task doesn't start because of this error.
  • Set up a NAT with the EIP, assigned it explicitly with the custom RT to the subnet where the task is - doesn't work.
  • Set up a NLB with the EIP and created a service with the task, mapped to the NLB, following this guide. Doesn't work. Tried ALB as well, no success.

VPC has a default RT in most cases.

What is the working ECS with EIP setup? Please provide a detailed guide if possible.

Details on ECS with NLB/EIP setup

(following the above guide)

  • Internet-facing, IPv4
  • Network mapping: Selected one AZ with the subnet where the task is; Use an Elastic IP address (the allocated one)
  • Security group: default with inbound HTTP allowed for all sources and outbound 443 for all sources (needed for access to ECR)
  • Listener: TCP:80 (default) / Target group: type IP addresses; all other settings are set to default: TCP:80; IPv4, network - my VPC;
  • Service setup: Fargate Latest; Application type: Service; task definition: my-server:latest; service type: Replica; Networking: only one subnet which is used for the NLB network mapping; Load Balancing - the target group which I created in the previous step.

Result:

  • I can access HTTP by the public dynamic IP of the task.
  • I can't access HTTP by the EIP assigned to the NLB.
  • The health check of the TG fails and the task is stopped.

Same attempt, but the service is configured without the public IP:

  • the service doesn't even start with this error: ResourceInitializationError: unable to pull secrets or registry auth (link above)

CloudFormation template

AWSTemplateFormatVersion: "2010-09-09"
Description: HelloWorld ECS Fargate Task with NLB and EIP

Resources:
  HelloWorldVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
    DependsOn: HelloWorldIGW

  HelloWorldRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref HelloWorldVPC

  HelloWorldRoute:
    Type: AWS::EC2::Route
    DependsOn: AttachGateway
    Properties:
      RouteTableId: !Ref HelloWorldRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref HelloWorldIGW

  HelloWorldIGW:
    Type: AWS::EC2::InternetGateway

  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref HelloWorldVPC
      InternetGatewayId: !Ref HelloWorldIGW

  HelloWorldSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref HelloWorldVPC
      CidrBlock: 10.0.0.0/24
      AvailabilityZone: us-east-1a

  HelloWorldSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref HelloWorldVPC
      CidrBlock: 10.0.1.0/24
      AvailabilityZone: us-east-1b

  Subnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref HelloWorldSubnet1
      RouteTableId: !Ref HelloWorldRouteTable

  Subnet2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref HelloWorldSubnet2
      RouteTableId: !Ref HelloWorldRouteTable

  HelloWorldNLB:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: HelloWorldNLB
      Subnets:
        - !Ref HelloWorldSubnet1
        - !Ref HelloWorldSubnet2
      Scheme: internet-facing
      LoadBalancerAttributes:
        - Key: load_balancing.cross_zone.enabled
          Value: true
      IpAddressType: ipv4
      SecurityGroups:
        - !GetAtt HelloWorldNLBSecurityGroup.GroupId

  HelloWorldNLBSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: NLBSecurityGroup
      VpcId: !Ref HelloWorldVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0

  HelloWorldECSCluster:
    Type: AWS::ECS::Cluster

  HelloWorldECSTaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: HelloWorldTask
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      Cpu: "256"
      Memory: "512"
      ExecutionRoleArn: "arn:aws:iam::USERID:role/ecsTaskExecutionRole"
      ContainerDefinitions:
        - Name: HelloWorldWebServerContainer
          Image: any-image-exposing-port-80:latest
          Essential: true
          PortMappings:
            - ContainerPort: 80

  HelloWorldECSService:
    Type: AWS::ECS::Service
    Properties:
      ServiceName: HelloWorldECSService
      Cluster: !Ref HelloWorldECSCluster
      TaskDefinition: !Ref HelloWorldECSTaskDefinition
      LaunchType: FARGATE
      DesiredCount: 2
      NetworkConfiguration:
        AwsvpcConfiguration:
          Subnets:
            - !Ref HelloWorldSubnet1
            - !Ref HelloWorldSubnet2
          SecurityGroups:
            - !GetAtt HelloWorldECSSecurityGroup.GroupId
          AssignPublicIp: ENABLED

  HelloWorldECSSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: ECSSecurityGroup
      VpcId: !Ref HelloWorldVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 0.0.0.0/0

Outputs:
  NLBDNSName:
    Description: HelloWorldNetworkLoadBalancer
    Value: !GetAtt HelloWorldNLB.DNSName

As the result, I have two tasks running in two subnets with public IP's, by which they can be accessed. NLB has a public domain name, which doesn't show anything on port 80.

Solution

  • Here is the CloudFormation template that produces a working stack:

    AWSTemplateFormatVersion: "2010-09-09"
Description: HelloWorld ECS Fargate Task with NLB, EIP, Target Group, and Targets

Resources:
  HelloWorldVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
    DependsOn: HelloWorldIGW

  HelloWorldIGW:
    Type: AWS::EC2::InternetGateway

  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref HelloWorldVPC
      InternetGatewayId: !Ref HelloWorldIGW

  HelloWorldSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref HelloWorldVPC
      CidrBlock: 10.0.0.0/24
      AvailabilityZone: us-east-1a

  HelloWorldRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref HelloWorldVPC

  HelloWorldRoute:
    Type: AWS::EC2::Route
    DependsOn: AttachGateway
    Properties:
      RouteTableId: !Ref HelloWorldRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref HelloWorldIGW

  Subnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref HelloWorldSubnet1
      RouteTableId: !Ref HelloWorldRouteTable

  HelloWorldNLB:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Type: network
      SubnetMappings:
        - SubnetId: !Ref HelloWorldSubnet1
          AllocationId: 'eipalloc-xxx' # Replace with your Elastic IP allocation id
      Scheme: internet-facing
      LoadBalancerAttributes:
        - Key: load_balancing.cross_zone.enabled
          Value: true
      IpAddressType: ipv4
      SecurityGroups:
        - !GetAtt HelloWorldNLBSecurityGroup.GroupId

  HelloWorldTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Protocol: TCP
      Port: 80
      VpcId: !Ref HelloWorldVPC
      TargetType: ip

  HelloWorldNLBListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref HelloWorldTargetGroup
      LoadBalancerArn: !Ref HelloWorldNLB
      Protocol: TCP
      Port: 80

  HelloWorldNLBSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: NLBSecurityGroup
      VpcId: !Ref HelloWorldVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0

  HelloWorldECSCluster:
    Type: AWS::ECS::Cluster

  HelloWorldECSTaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: HelloWorldTask
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      Cpu: "256"
      Memory: "512"
      # the role must have AmazonECSTaskExecutionRolePolicy and all ECR list/read permissions to be able to pull the image
      ExecutionRoleArn: "arn:aws:iam::USERID:role/ecsTaskExecutionRole" # replace with your user id
      ContainerDefinitions:
        - Name: HelloWorldWebServerContainer
          Image: USERID.dkr.ecr.us-east-1.amazonaws.com/ANY-IMAGE-WITH-PORT-80:latest # replace with your user id and with your Docker image
          Essential: true
          PortMappings:
            - ContainerPort: 80

  HelloWorldECSService:
    Type: AWS::ECS::Service
    DependsOn: HelloWorldNLBListener
    Properties:
      ServiceName: HelloWorldECSService
      Cluster: !Ref HelloWorldECSCluster
      TaskDefinition: !Ref HelloWorldECSTaskDefinition
      LaunchType: FARGATE
      DesiredCount: 1
      NetworkConfiguration:
        AwsvpcConfiguration:
          Subnets:
            - !Ref HelloWorldSubnet1
          SecurityGroups:
            - !GetAtt HelloWorldECSSecurityGroup.GroupId
          AssignPublicIp: ENABLED # important for the task to pull image from ECR
      LoadBalancers:
        - ContainerName: HelloWorldWebServerContainer
          ContainerPort: 80
          TargetGroupArn: !Ref HelloWorldTargetGroup

  HelloWorldECSSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: ECSSecurityGroup
      VpcId: !Ref HelloWorldVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
      SecurityGroupEgress: # important for the task to pull image from ECR
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 0.0.0.0/0

Outputs:
  NLBDNSName:
    Description: HelloWorldNetworkLoadBalancer
    Value: !GetAtt HelloWorldNLB.DNSName

    The service must be associated to the target group, which is associated to the listener, which is associated to the load balancer. The Elastic IP must be mapped to the subnet in the load balancer settings.

    It is not enough for the ecsTaskExecutionRole to have only AmazonECSTaskExecutionRolePolicy to pull image from ECR, I had to add another policy to it with ECR all list and read permissions.