Search code examples
amazon-web-servicesamazon-ec2aws-cloudformationuser-datadatadog

Best practice to create a file in AWS EC2 instance using cloud formation

There are a lot of options to do that and I don't know which is the best one. I tried in the beginning to do is as following:

ServiceInstance:
Type: "AWS::EC2::Instance"
Properties:
  ImageId: !Ref AmiId, !Ref LatestOnescoutAmi ]
  InstanceType: !Ref InstanceType
  SubnetId: !ImportValue vpc-stack-PublicASubnet
  SecurityGroupIds:
    - !Ref ServiceSecurityGroup
  KeyName: !Ref KeyName
  UserData:
    'Fn::Base64': !Sub |
      #cloud-config
      write_files:
        - path: /etc/sysconfig/cloudformation
          permissions: 0644
          owner: root
          content: |
            STACK_NAME=${AWS::StackName}
            AWS_REGION=${AWS::Region}
        - path: /etc/datadog-agent/conf.d/mysql.d/conf.yaml
          permissions: 0644
          owner: dd-agent
          content: |
            init_config:
            instances:
            - server: some-db-host
              user: some-admin
              pass: some-password
              port: 3306
              tags:
              - dbinstanceidentifier:someide

      runcmd:
        ## enable datadog agent
        - systemctl start datadog-agent
        - systemctl start application.service

but then my /etc/datadog-agent/conf.d/mysql.d/conf.yaml grew and I have around 13 blocks and it's not good to put them hardcoded inside the template. Better to keep the template generic and pass the config file as a parameter. But, according to this answer here, it's not possible to pass a file or content of file to cloud formation.

The way above is the simplest I see among other two options I can think about.

  1. Store the config in SSM and then get it back at the ec2 start.

  2. Create an Autoscaling and launch group which accepts the file path but it's more complex than I need:

    LaunchConfig:
    Type: AWS::AutoScaling::LaunchConfiguration
    Metadata:
      AWS::CloudFormation::Init:
        configSets:
          service_configuration:
          - datadog_setup
        datadog_setup:
          files:
            /etc/datadog-agent/conf.d/mysql.d/conf.yaml:
              content: "@file://./config/conf-${Env}.yaml"
              mode: "000644"
              owner: "root"
              group: "root"
          commands:
            start_datadog:
              command: service datadog-agent start

Any idea how to do this in a simple, generic and secure way? An example given would be appreciated. Thanks in advance.

Solution

  • How I managed to do that with another way, I created S3 bucket and then created a role to my ec2 instance that can access this s3 bucket and can download files, then in my runcmd section, I downloaded this file.

      ServiceInstance:
    Type: "AWS::EC2::Instance"
    Properties:
      InstanceType: !Ref InstanceType
      IamInstanceProfile: !Ref InstanceProfile
      UserData:
        'Fn::Base64': !Sub |
          #cloud-config
          write_files:
          - path: /etc/sysconfig/cloudformation
            permissions: 0644
            owner: root
            content: |
              STACK_NAME=${AWS::StackName}
              AWS_REGION=${AWS::Region}
          runcmd:
          - aws s3 cp s3://${ArtifactsBucketName}/dd-get-secrets.py /home/ec2-user/dd-get-secrets.py

  InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles: [!Ref IAMRole]

  IAMRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service: [ec2.amazonaws.com]
          Action: ['sts:AssumeRole']
      Path: /
      Policies:

      - PolicyName: allow-downloading-dd-templates
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action:
            - s3:GetObject
            - s3:ListBucket
            Resource: !Sub "arn:aws:s3:::${ArtifactsBucketName}/*"