Configure ECS Cluster Instance Configuration using YAML file

can any one provide the sample yaml file which is creating a ec2 instance in a ECS cluster.

As we can setup it through the aws console in below image. sample image :

But i want to create it through YAML file.

Also please explain what are the key points/things which connects a EC2 instance to a ECS cluster.

Solution

Check this out ecs-refarch-cloudformation

This reference architecture provides a set of YAML templates for deploying microservices to Amazon EC2 Container Service (Amazon ECS) with AWS CloudFormation.

In particular, check out this template, which configures an ECS cluster. It creates an autoscaling group which launches EC2 instances that containers in the cluster can be deployed into.

Description: >
  This template deploys an ECS cluster to the provided VPC and subnets
  using an Auto Scaling Group

Parameters:
  EnvironmentName:
    Description: An environment name that will be prefixed to resource names
    Type: String

  InstanceType:
    Description: Which instance type should we use to build the ECS cluster?
    Type: String
    Default: c4.large

  ClusterSize:
    Description: How many ECS hosts do you want to initially deploy?
    Type: Number
    Default: 4

  VPC:
    Description: Choose which VPC this ECS cluster should be deployed to
    Type: AWS::EC2::VPC::Id

  Subnets:
    Description: Choose which subnets this ECS cluster should be deployed to
    Type: List<AWS::EC2::Subnet::Id>

  SecurityGroup:
    Description: Select the Security Group to use for the ECS cluster hosts
    Type: AWS::EC2::SecurityGroup::Id

  ECSAMI:
    Description: ECS-Optimized AMI ID
    Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
    Default: /aws/service/ecs/optimized-ami/amazon-linux/recommended/image_id

Resources:
  ECSCluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Ref EnvironmentName

  ECSAutoScalingGroup:
    DependsOn: ECSCluster
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      VPCZoneIdentifier: !Ref Subnets
      LaunchConfigurationName: !Ref ECSLaunchConfiguration
      MinSize: !Ref ClusterSize
      MaxSize: !Ref ClusterSize
      DesiredCapacity: !Ref ClusterSize
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName} ECS host
          PropagateAtLaunch: true
    CreationPolicy:
      ResourceSignal:
        Timeout: PT15M
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MinInstancesInService: 1
        MaxBatchSize: 1
        PauseTime: PT15M
        SuspendProcesses:
          - HealthCheck
          - ReplaceUnhealthy
          - AZRebalance
          - AlarmNotification
          - ScheduledActions
        WaitOnResourceSignals: true

  ECSLaunchConfiguration:
    Type: AWS::AutoScaling::LaunchConfiguration
    Properties:
      ImageId: !Ref ECSAMI
      InstanceType: !Ref InstanceType
      SecurityGroups:
        - !Ref SecurityGroup
      IamInstanceProfile: !Ref ECSInstanceProfile
      UserData:
        "Fn::Base64": !Sub |
          #!/bin/bash
          yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm
          yum install -y https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
          yum install -y aws-cfn-bootstrap hibagent 
          /opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource ECSLaunchConfiguration
          /opt/aws/bin/cfn-signal -e $? --region ${AWS::Region} --stack ${AWS::StackName} --resource ECSAutoScalingGroup
          /usr/bin/enable-ec2-spot-hibernation

    Metadata:
      AWS::CloudFormation::Init:
        config:
          packages:
            yum:
              collectd: []

          commands:
            01_add_instance_to_cluster:
              command: !Sub echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
            02_enable_cloudwatch_agent:
              command: !Sub /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:${ECSCloudWatchParameter} -s
          files:
            /etc/cfn/cfn-hup.conf:
              mode: 000400
              owner: root
              group: root
              content: !Sub |
                [main]
                stack=${AWS::StackId}
                region=${AWS::Region}

            /etc/cfn/hooks.d/cfn-auto-reloader.conf:
              content: !Sub |
                [cfn-auto-reloader-hook]
                triggers=post.update
                path=Resources.ECSLaunchConfiguration.Metadata.AWS::CloudFormation::Init
                action=/opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource ECSLaunchConfiguration

          services:
            sysvinit:
              cfn-hup:
                enabled: true
                ensureRunning: true
                files:
                  - /etc/cfn/cfn-hup.conf
                  - /etc/cfn/hooks.d/cfn-auto-reloader.conf

  # This IAM Role is attached to all of the ECS hosts. It is based on the default role
  # published here:
  # http://docs.aws.amazon.com/AmazonECS/latest/developerguide/instance_IAM_role.html
  #
  # You can add other IAM policy statements here to allow access from your ECS hosts
  # to other AWS services. Please note that this role will be used by ALL containers
  # running on the ECS host.

  ECSRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      RoleName: !Sub ${EnvironmentName}-ECSRole-${AWS::Region}
      AssumeRolePolicyDocument: |
        {
            "Statement": [{
                "Action": "sts:AssumeRole",
                "Effect": "Allow",
                "Principal": {
                    "Service": "ec2.amazonaws.com"
                }
            }]
        }
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM
        - arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy
      Policies:
        - PolicyName: ecs-service
          PolicyDocument: |
            {
                "Statement": [{
                    "Effect": "Allow",
                    "Action": [
                        "ecs:CreateCluster",
                        "ecs:DeregisterContainerInstance",
                        "ecs:DiscoverPollEndpoint",
                        "ecs:Poll",
                        "ecs:RegisterContainerInstance",
                        "ecs:StartTelemetrySession",
                        "ecs:Submit*",
                        "ecr:BatchCheckLayerAvailability",
                        "ecr:BatchGetImage",
                        "ecr:GetDownloadUrlForLayer",
                        "ecr:GetAuthorizationToken"
                    ],
                    "Resource": "*"
                }]
            }

  ECSInstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - !Ref ECSRole

  ECSServiceAutoScalingRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          Action:
            - "sts:AssumeRole"
          Effect: Allow
          Principal:
            Service:
              - application-autoscaling.amazonaws.com
      Path: /
      Policies:
        - PolicyName: ecs-service-autoscaling
          PolicyDocument:
            Statement:
              Effect: Allow
              Action:
                - application-autoscaling:*
                - cloudwatch:DescribeAlarms
                - cloudwatch:PutMetricAlarm
                - ecs:DescribeServices
                - ecs:UpdateService
              Resource: "*"

  ECSCloudWatchParameter:
    Type: AWS::SSM::Parameter
    Properties:
      Description: ECS
      Name: !Sub "AmazonCloudWatch-${ECSCluster}-ECS"
      Type: String
      Value: !Sub |
        {
          "logs": {
            "force_flush_interval": 5,
            "logs_collected": {
              "files": {
                "collect_list": [
                  {
                    "file_path": "/var/log/messages",
                    "log_group_name": "${ECSCluster}-/var/log/messages",
                    "log_stream_name": "{instance_id}",
                    "timestamp_format": "%b %d %H:%M:%S"
                  },
                  {
                    "file_path": "/var/log/dmesg",
                    "log_group_name": "${ECSCluster}-/var/log/dmesg",
                    "log_stream_name": "{instance_id}"
                  },
                  {
                    "file_path": "/var/log/docker",
                    "log_group_name": "${ECSCluster}-/var/log/docker",
                    "log_stream_name": "{instance_id}",
                    "timestamp_format": "%Y-%m-%dT%H:%M:%S.%f"
                  },
                  {
                    "file_path": "/var/log/ecs/ecs-init.log",
                    "log_group_name": "${ECSCluster}-/var/log/ecs/ecs-init.log",
                    "log_stream_name": "{instance_id}",
                    "timestamp_format": "%Y-%m-%dT%H:%M:%SZ"
                  },
                  {
                    "file_path": "/var/log/ecs/ecs-agent.log.*",
                    "log_group_name": "${ECSCluster}-/var/log/ecs/ecs-agent.log",
                    "log_stream_name": "{instance_id}",
                    "timestamp_format": "%Y-%m-%dT%H:%M:%SZ"
                  },
                  {
                    "file_path": "/var/log/ecs/audit.log",
                    "log_group_name": "${ECSCluster}-/var/log/ecs/audit.log",
                    "log_stream_name": "{instance_id}",
                    "timestamp_format": "%Y-%m-%dT%H:%M:%SZ"
                  }
                ]
              }
            }
          },
          "metrics": {
            "append_dimensions": {
              "AutoScalingGroupName": "${!aws:AutoScalingGroupName}",
              "InstanceId": "${!aws:InstanceId}",
              "InstanceType": "${!aws:InstanceType}"
            },
            "metrics_collected": {
              "collectd": {
                "metrics_aggregation_interval": 60
              },
              "disk": {
                "measurement": [
                  "used_percent"
                ],
                "metrics_collection_interval": 60,
                "resources": [
                  "/"
                ]
              },
              "mem": {
                "measurement": [
                  "mem_used_percent"
                ],
                "metrics_collection_interval": 60
              },
              "statsd": {
                "metrics_aggregation_interval": 60,
                "metrics_collection_interval": 10,
                "service_address": ":8125"
              }
            }
          }
        }

Outputs:
  Cluster:
    Description: A reference to the ECS cluster
    Value: !Ref ECSCluster

  ECSServiceAutoScalingRole:
    Description: A reference to ECS service auto scaling role
    Value: !GetAtt ECSServiceAutoScalingRole.Arn

  ECSAutoScalingGroupName:
    Description: A reference to ECS AutoScaling Group Name
    Value: !Ref ECSAutoScalingGroup

Essentially, this template creates an autoscaling group that spawns up EC2 instances running an Amazon ECS-Optimized Amazon Linux AMI that is configured to work with ECS. The launch configuration for the auto scaling group further customizes the instances by specifying the particular ECS Cluster from which the instances run tasks. This is accomplished by updating settings in /etc/ecs/ecs.config. For example:

command: !Sub echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config

For more information about ECS instance configuration, check out these resoureces: