Search code examples
amazon-web-servicesamazon-ec2amazon-ecsamazon-elb

AWS ELB: 503 Service Temporarily Unavailable


I am trying to deploy microservices to AWS ECS following this example repo. Here, a load balancer is utilized for networking between different docker services. I have adjusted the cloudformation templates to my needs and have only deployed one service yet, the webserver. I want public access to the web interface and add other services this service is talking to subsequently. However, I currently struggle to get the webserver going using the URL of the load balancer. If I go directly to the EC2 instance using the public IP, I get access to the interface. However, if I go to the load balancer DNS, I get 503 Service Temporarily Unavailable. I have checked the AWS docs, but the webservice target group shows a registered target (the EC2 instance) with status healthy. What am I missing?

Parameters:
  EnvironmentName:
    Description: An environment name that will be prefixed to resource names
    Type: String

  VPC:
    Type: AWS::EC2::VPC::Id
    Description: Choose which VPC the Application Load Balancer should be deployed to

  Subnets:
    Description: Choose which subnets the Application Load Balancer should be deployed to
    Type: AWS::EC2::Subnet::Id

  PublicSubnet:
    Description: Choose which public subnet the EC2 instance should be deployed to
    Type: AWS::EC2::Subnet::Id

Resources:

  LoadBalancerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${EnvironmentName}-loadbalancer
      GroupDescription: Access to the load balancer that sits in front of ECS
      VpcId: !Ref VPC
      SecurityGroupIngress:
        # Allow access from anywhere to our ECS services
        - CidrIp: 0.0.0.0/0
          IpProtocol: -1
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName}-LoadBalancers

  LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: !Ref EnvironmentName
      Subnets:
        - !Ref Subnets
      SecurityGroups:
        - !Ref LoadBalancerSecurityGroup
      Tags:
        - Key: Name
          Value: !Ref EnvironmentName

  LoadBalancerListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      LoadBalancerArn: !Ref LoadBalancer
      Port: 80
      Protocol: HTTP
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref DefaultTargetGroup

  DefaultTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: !Sub ${EnvironmentName}-default
      VpcId: !Ref VPC
      Port: 80
      Protocol: HTTP

  ECSHostSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${EnvironmentName}-ecs-hosts
      GroupDescription: Access to the ECS hosts and the tasks/containers that run on them
      VpcId: !Ref VPC
      SecurityGroupIngress:
        # Only allow inbound access to ECS from the ELB
        - SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup
          IpProtocol: -1
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName}-ECS-Hosts

  ECSCluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Ref EnvironmentName

  ECSRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      RoleName: !Sub ${EnvironmentName}-ecs-role
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role'
        - 'arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM'
        - 'arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy'

  ECSInstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - !Ref ECSRole

  EC2Webserver:
    Type: AWS::EC2::Instance
    Properties:
      AvailabilityZone: eu-central-1a
      ImageId: !Ref ECSAMI
      InstanceType: !Ref InstanceType
      IamInstanceProfile: !Ref ECSInstanceProfile
      UserData:
        Fn::Base64:
          !Sub |
          #!/bin/bash
          # Add to cluster:
          echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
          echo ECS_ENABLE_AWSLOGS_EXECUTIONROLE_OVERRIDE=true >> /etc/ecs/ecs.config
      SecurityGroupIds:
        - !Ref ECSHostSecurityGroup
      SubnetId: !Ref PublicSubnet
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName}

  Service:
    Type: AWS::ECS::Service
    DependsOn: ListenerRule
    Properties:
      Cluster: !Ref Cluster
      Role: !Ref ServiceRole
      DesiredCount: !Ref DesiredCount
      TaskDefinition: !Ref TaskDefinitionWebserver
      LoadBalancers:
        - ContainerName: !Sub ${EnvironmentName}-webserver
          ContainerPort: 8080
          TargetGroupArn: !Ref TargetGroup

  TaskDefinitionWebserver:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: !Sub ${EnvironmentName}-webserver
      ContainerDefinitions:
        - Name: !Sub ${EnvironmentName}-webserver
          Essential: true
          Image: !Ref Image
          Memory: 512
          PortMappings:
            - ContainerPort: 8080
              HostPort: 80

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: !Sub ${EnvironmentName}-webserver
      VpcId: !Ref VPC
      Port: 80
      Protocol: HTTP
      Matcher:
        HttpCode: 200-299
      HealthCheckIntervalSeconds: 30
      HealthCheckPath: /health
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 10
      HealthyThresholdCount: 5

  ListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      ListenerArn: !Ref LoadBalancerListener
      Priority: 1
      Conditions:
        - Field: path-pattern
          Values:
            - /
      Actions:
        - TargetGroupArn: !Ref TargetGroup
          Type: forward

  ServiceRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ecs-service-${AWS::StackName}
      Path: /
      AssumeRolePolicyDocument: |
        {
            "Statement": [{
                "Effect": "Allow",
                "Principal": { "Service": [ "ecs.amazonaws.com" ]},
                "Action": [ "sts:AssumeRole" ]
            }]
        }
      Policies:
        - PolicyName: !Sub ecs-service-${AWS::StackName}
          PolicyDocument:
            {
              "Version": "2012-10-17",
              "Statement":
                [
                {
                  "Effect": "Allow",
                  "Action":
                    [
                      "ec2:AuthorizeSecurityGroupIngress",
                      "ec2:Describe*",
                      "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
                      "elasticloadbalancing:Describe*",
                      "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
                      "elasticloadbalancing:DeregisterTargets",
                      "elasticloadbalancing:DescribeTargetGroups",
                      "elasticloadbalancing:DescribeTargetHealth",
                      "elasticloadbalancing:RegisterTargets"
                    ],
                  "Resource": "*"
                }
                ]
            }

Outputs:

  WebsiteServiceUrl:
    Description: The URL endpoint for the website service
    Value: !Join ["", [!GetAtt LoadBalancer.DNSName, "/"]]

Solution

  • Thanks everyone! I finally figured it out, what I had to do was adjusting the path as my service redirects on /. So I only changed the listener rule using a wildcard:

    ListenerRule:
        Type: AWS::ElasticLoadBalancingV2::ListenerRule
        Properties:
          ListenerArn: !Ref LoadBalancerListener
          Priority: 1
          Conditions:
            - Field: path-pattern
              Values:
                - [/*]
          Actions:
            - TargetGroupArn: !Ref TargetGroup
              Type: forward