Search code examples
amazon-web-servicesaws-lambdaaws-cloudformationamazon-iamzappa

Only able to deploy CloudFormation template to us-east-1 using Boto3


I'm trying to deploy my CloudFormation template to other regions for testing. My template works fine with us-east-1 via Boto3 but if it try another region I get no error output.

Whilst trying different regions I got an email unexpectedly saying that the Canada region has been verified but since trying via Boto3 this has been unsuccessful. (billing console says all regions are now activated)

I'm running Boto3 from Lambda (No VPC) that has been deployed using Zappa to us-east-1. It has an IAM policy that does not specify a specific region.

Python:

cf_client = boto3.client(
            'cloudformation', region_name=request.POST['region'])

cf_client.create_stack(
                StackName=stack_name,
                TemplateURL='https://s3.amazonaws.com/#######/build_instance.yaml',
                Parameters=[
                    {"ParameterKey": "FQDN",
                        "ParameterValue": instance_domain},
                    {"ParameterKey": "BucketName",
                        "ParameterValue": bucket_name},
                    {"ParameterKey": "CreateSubdomain",
                        "ParameterValue": create_subdomain},
                    {"ParameterKey": "CustomerEmail",
                        "ParameterValue": request.user.email},
                    {"ParameterKey": "Region",
                        "ParameterValue": request.POST['region']},
                ],
                Capabilities=['CAPABILITY_NAMED_IAM'],
                Tags=[
                    {
                        'Key': 'Name',
                        'Value': instance_domain
                    },
                    {
                        'Key': 'env',
                        'Value': "prod"
                    }, ],
                EnableTerminationProtection=True
            )

CF:

---
AWSTemplateFormatVersion: "2010-09-09"
Description: ""

Parameters:
  FQDN:
    Type: String
    Description: Instance FQDN

  BucketName:
    Type: String
    Description: Name of S3 bucket

  CreateSubdomain:
    Type: String
    Default: false
    AllowedValues: [true, false]
    Description: Does the customer want to use our sub-domain?

  CustomerEmail:
    Type: String
    Description: Customer email to deliver credentials

  Region:
    Type: String
    Description: Customer region

Mappings:
  RegionMap:
    us-east-1:
      AMI: "ami-0affd4508a5d2481b"
    us-west-1:
      AMI: "ami-03ba3948f6c37a4b0"
    ca-central-1:
      AMI: "ami-0d0eaed20348a3389"
    eu-west-2:
      AMI: " ami-006a0174c6c25ac06"

Conditions:
  ShouldCreateSubDomain: !Equals [true, !Ref CreateSubdomain]

Resources:
  Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref BucketName
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
      MetricsConfigurations:
        - Id: EntireBucket
      LifecycleConfiguration:
        Rules:
          - Id: IntelligentTieringTransition
            Status: Enabled
            Transitions:
              - TransitionInDays: 30
                StorageClass: INTELLIGENT_TIERING
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        IgnorePublicAcls: false
        BlockPublicPolicy: true
        RestrictPublicBuckets: true

  User:
    Type: AWS::IAM::User
    Properties:
      UserName:
        Ref: FQDN
      Groups: ["Customers"]
    DependsOn: Bucket

  Key:
    Type: AWS::IAM::AccessKey
    Properties:
      UserName:
        Ref: User
    DependsOn: User

  BucketPolicy:
    Type: "AWS::S3::BucketPolicy"
    Properties:
      Bucket: !Ref BucketName
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Principal:
              AWS: !GetAtt User.Arn
            Action: "s3:*"
            Effect: Allow
            Resource:
              - !GetAtt Bucket.Arn
              - !Sub "${Bucket.Arn}/*"
    DependsOn: Key

  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !FindInMap [RegionMap, !Ref "AWS::Region", AMI]
      KeyName: aws-master
      InstanceType: t3.micro
      DisableApiTermination: true
      SecurityGroups: ["nextcloud-security"]
      BlockDeviceMappings:
        - DeviceName: /dev/sda1
          Ebs:
            VolumeSize: 10
            DeleteOnTermination: false
      Tags:
        - Key: "Name"
          Value: !Ref FQDN
        - Key: "env"
          Value: "prod"

      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -xe
          exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
            curl -s ####### | /bin/bash
            git clone #######

            cd nextcloud

            sed -i 's/# fqdn       = nc.example.org/fqdn       = ${FQDN}/g' inventory
            sed -i "s/ssl_certificate_type  = 'selfsigned'/#ssl_certificate_type  = 'selfsigned'/g" inventory
            sed -i "s/# ssl_certificate_type  = 'letsencrypt'/ssl_certificate_type  = 'letsencrypt'/g" inventory
            sed -i 's/# cert_email = [email protected]/cert_email = ####/g' inventory
            sed -i "s/# nc_db_type          = 'mysql'/nc_db_type          = 'mysql'/g" inventory
            sed -i "s/nc_db_type           = 'pgsql'/#nc_db_type           = 'pgsql'/g" inventory
            sed -i 's/nc_configure_mail    = false/nc_configure_mail    = true/g' inventory
            sed -i 's/nc_mail_from         =/nc_mail_from         = contact/g' inventory
            sed -i 's/nc_mail_domain       =/nc_mail_domain       = ######/g' inventory
            sed -i 's/nc_mail_smtpname     =/nc_mail_smtpname     = #######/g' inventory
            sed -i 's/nc_mail_smtphost     =/nc_mail_smtphost     = smtp.gmail.com/g' inventory
            sed -i 's/nc_mail_smtppwd      =/nc_mail_smtppwd      = #####/g' inventory
            sed -i 's/s3_key               =/s3_key               = ${Key}/g' inventory
            sed -i 's|s3_secret            =|s3_secret            = ${Key.SecretAccessKey}|g' inventory
            sed -i 's/s3_bucket            =/s3_bucket            = ${BucketName}/g' inventory
            sed -i 's/s3_region            =/s3_region            = ${Region}/g' inventory
            sed -i 's/talk_install         = false/talk_install         = true/g' inventory

            ./nextcloud.yml

  IPAddress:
    Type: AWS::EC2::EIP
    Properties:
      Tags:
        - Key: "Name"
          Value: !Ref FQDN
        - Key: "env"
          Value: "prod"

  IPAssoc:
    Type: AWS::EC2::EIPAssociation
    Properties:
      InstanceId: !Ref "EC2Instance"
      EIP: !Ref "IPAddress"

  Route53Record:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneName: "########"
      Name: !Join ["", [!Ref FQDN, "."]]
      Type: A
      TTL: "300"
      ResourceRecords:
        - !Ref "IPAddress"
    Condition: ShouldCreateSubDomain

Outputs:
  InstanceId:
    Description: InstanceId of the newly created EC2 instance
    Value: !Ref "EC2Instance"
  InstanceIPAddress:
    Description: IP address of the newly created EC2 instance
    Value: !Ref "IPAddress"

IAM:

 "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeImages",
                "ec2:DescribeInstances",
                "ec2:DescribeAddresses",
                "ec2:DescribeTags",
                "ec2:CreateTags",
                "ec2:RunInstances",
                "ec2:DescribeKeyPairs",
                "ec2:AssociateAddress",
                "ec2:AllocateAddress"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "iam:AddUserToGroup",
                "cloudformation:CreateStack",
                "route53:ChangeResourceRecordSets",
                "iam:GetUser",
                "iam:CreateUser",
                "iam:CreateAccessKey"
            ],
            "Resource": [
                "arn:aws:iam::*:user/*",
                "arn:aws:iam::#######:group/Customers",
                "arn:aws:cloudformation:*:*:stack/*/*",
                "arn:aws:route53:::hostedzone/#####"
            ]
        }
    ]
}

Solution

  • From Selecting a Stack Template - AWS CloudFormation:

    Amazon S3 URL: The URL must point to a template with a maximum size of 460,800 bytes that is stored in an S3 bucket that you have read permissions to and that is located in the same region as the stack.

    I suspect that your stack is failing because the template is in an Amazon S3 bucket that is in a different region to where the stack is being launched. You will need to copy the template into a bucket in the same region, then provide it in the create_stack() command.

    You can test this by using the AWS Console to launch the template, rather than having to go via boto3.