Using Encrypted EBS Volumes in Auto Scaling Groups with CMK owned by a different AWS account

Question

I'm trying to use Auto Scaling groups in AWS to create and manage instances created from AMIs with encrypted snapshots, which have been encrypted by a CMK owned by a different AWS account.

I keep getting the error "Client.InternalError: Client error on launch". According to Scenario 2 at https://docs.aws.amazon.com/autoscaling/ec2/userguide/ts-as-instancelaunchfailure.html#ts-as-instancelaunchfailure-12, I need to create a grant to the CMK with the Auto Scaling group service-linked role as the grantee principal.

I tried following the guidelines in the AWS documentation and at https://forums.aws.amazon.com/thread.jspa?threadID=277523 for setting up the grant.

However, I keep getting an AccessDeniedException saying that my user is not authorised to perform kms:CreateGrant on the CMK.

I feel like I've followed the instructions perfectly, but it's not working. I'm hoping someone might be able to provide some insight.

Answer 1

I chatted with an AWS employee who ran into the same problem until he re-read the forum post. The key line in Case 2 Step 4 is "The kms:GrantIsForAWSResource condition is not included to allow an IAM user or role in account 111122223333 to create the grant in the next step.".

In other words, you need to remove this condition from the default key policy for a customer managed CMK.

The instructions could've made that requirement much more explicit, but technically it's there and it resolves the problem.

Edit: To clarify, I'm going to include the default and amended JSON below.

The following is the default key policy as shown at https://docs.aws.amazon.com/kms/latest/developerguide/key-policies.html#key-policy-default

    {
      "Version": "2012-10-17",
      "Id": "key-consolepolicy-2",
      "Statement": [
        {
          "Sid": "Enable IAM User Permissions",
          "Effect": "Allow",
          "Principal": {"AWS": "arn:aws:iam::111122223333:root"},
          "Action": "kms:*",
          "Resource": "*"
        },
        {
          "Sid": "Allow access for Key Administrators",
          "Effect": "Allow",
          "Principal": {"AWS": [
            "arn:aws:iam::111122223333:user/KMSAdminUser",
            "arn:aws:iam::111122223333:role/KMSAdminRole"
          ]},
          "Action": [
            "kms:Create*",
            "kms:Describe*",
            "kms:Enable*",
            "kms:List*",
            "kms:Put*",
            "kms:Update*",
            "kms:Revoke*",
            "kms:Disable*",
            "kms:Get*",
            "kms:Delete*",
            "kms:TagResource",
            "kms:UntagResource",
            "kms:ScheduleKeyDeletion",
            "kms:CancelKeyDeletion"
          ],
          "Resource": "*"
        },
        {
          "Sid": "Allow use of the key",
          "Effect": "Allow",
          "Principal": {"AWS": [
            "arn:aws:iam::111122223333:user/KMSUser",
            "arn:aws:iam::111122223333:role/KMSRole",
            "arn:aws:iam::444455556666:root"
          ]},
          "Action": [
            "kms:Encrypt",
            "kms:Decrypt",
            "kms:ReEncrypt*",
            "kms:GenerateDataKey*",
            "kms:DescribeKey"
          ],
          "Resource": "*"
        },
        {
          "Sid": "Allow attachment of persistent resources",
          "Effect": "Allow",
          "Principal": {"AWS": [
            "arn:aws:iam::111122223333:user/KMSUser",
            "arn:aws:iam::111122223333:role/KMSRole",
            "arn:aws:iam::444455556666:root"
          ]},
          "Action": [
            "kms:CreateGrant",
            "kms:ListGrants",
            "kms:RevokeGrant"
          ],
          "Resource": "*",
          "Condition": {"Bool": {"kms:GrantIsForAWSResource": "true"}}
        }
      ]
    }

The key is to remove the Condition for "kms:GrantIsForAWSResource" as below.

    {
      "Version": "2012-10-17",
      "Id": "key-consolepolicy-2",
      "Statement": [
        {
          "Sid": "Enable IAM User Permissions",
          "Effect": "Allow",
          "Principal": {"AWS": "arn:aws:iam::111122223333:root"},
          "Action": "kms:*",
          "Resource": "*"
        },
        {
          "Sid": "Allow access for Key Administrators",
          "Effect": "Allow",
          "Principal": {"AWS": [
            "arn:aws:iam::111122223333:user/KMSAdminUser",
            "arn:aws:iam::111122223333:role/KMSAdminRole"
          ]},
          "Action": [
            "kms:Create*",
            "kms:Describe*",
            "kms:Enable*",
            "kms:List*",
            "kms:Put*",
            "kms:Update*",
            "kms:Revoke*",
            "kms:Disable*",
            "kms:Get*",
            "kms:Delete*",
            "kms:TagResource",
            "kms:UntagResource",
            "kms:ScheduleKeyDeletion",
            "kms:CancelKeyDeletion"
          ],
          "Resource": "*"
        },
        {
          "Sid": "Allow use of the key",
          "Effect": "Allow",
          "Principal": {"AWS": [
            "arn:aws:iam::111122223333:user/KMSUser",
            "arn:aws:iam::111122223333:role/KMSRole",
            "arn:aws:iam::444455556666:root"
          ]},
          "Action": [
            "kms:Encrypt",
            "kms:Decrypt",
            "kms:ReEncrypt*",
            "kms:GenerateDataKey*",
            "kms:DescribeKey"
          ],
          "Resource": "*"
        },
        {
          "Sid": "Allow attachment of persistent resources",
          "Effect": "Allow",
          "Principal": {"AWS": [
            "arn:aws:iam::111122223333:user/KMSUser",
            "arn:aws:iam::111122223333:role/KMSRole",
            "arn:aws:iam::444455556666:root"
          ]},
          "Action": [
            "kms:CreateGrant",
            "kms:ListGrants",
            "kms:RevokeGrant"
          ],
          "Resource": "*"
        }
      ]
    }