Search code examples
jsonamazon-web-servicesamazon-dynamodbaws-cloudformation

Is there a way to populate an Amazon DynamoDB table from a CloudFormation template?


I need to create a DynamoDB table were one of the attribute values its passed as a parameter in a CloudFormation template:

The DynamoDB table should look like this:

PhoneNumber     |       OrderNumber

223546421               11545154
784578745               11547854
223458784               11547487
XXXXXXXXX               11578451

The attribute value "XXXXXXXXX" needs to be passed as a parameter from a CloudFormation template were the DynamoDB table is going to be created and populated with the information above.

This is the current CF template which builds the DynamoDB table:

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Resources": {
        "OrdersTable": {
            "Type": "AWS::DynamoDB::Table",
            "Properties": {
                "TableName": "ClientOrders",
                "AttributeDefinitions": [
                    {
                        "AttributeName": "PhoneNumber",
                        "AttributeType": "S"
                    },
                    {
                        "AttributeName": "OrderNumber",
                        "AttributeType": "S"
                    }
                ],
                "KeySchema": [
                    {
                        "AttributeName": "PhoneNumber",
                        "KeyType": "HASH"
                    },
                    {
                        "AttributeName": "OrderNumber",
                        "KeyType": "RANGE"
                    }
                ],
                "TimeToLiveSpecification": {
                    "AttributeName": "ExpirationTime",
                    "Enabled": true
                },
                "ProvisionedThroughput": {
                    "ReadCapacityUnits": "10",
                    "WriteCapacityUnits": "5"
                }
            },
            "DependsOn": [
                "DynamoDBQueryPolicy"
            ]
        },
        "DynamoDBQueryPolicy": {
            "Type": "AWS::IAM::Policy",
            "Properties": {
                "PolicyName": "DynamoDBQueryPolicy",
                "PolicyDocument": {
                    "Version": "2012-10-17",
                    "Statement": [
                        {
                            "Effect": "Allow",
                            "Action": "dynamodb:Query",
                            "Resource": "*"
                        }
                    ]
                },
                "Roles": [
                    {
                        "Ref": "OrdersTableQueryRole"
                    }
                ]
            }
        },
        "OrdersTableQueryRole": {
            "Type": "AWS::IAM::Role",
            "Properties": {
                "AssumeRolePolicyDocument": {
                    "Version": "2012-10-17",
                    "Statement": [
                        {
                            "Effect": "Allow",
                            "Principal": {
                                "Service": [
                                    "dynamodb.amazonaws.com"
                                ]
                            },
                            "Action": [
                                "sts:AssumeRole"
                            ]
                        }
                    ]
                },
                "Path": "/"
            }
        }
    }
}

I need help to populate the table with the mentioned values above and how to pass mentioned attribute value as a parameter.

Any help on this would be appreciated.

Thanks


Solution

  • We need custom cloudformation resource to add entries into Dynamo Table.

    From Docs:

    With Lambda functions and custom resources, you can run custom code in response to stack events (create, update, and delete).

    This custom resource invokes a Lambda function and sends it the StackName property as input. The function uses this property to get outputs from the appropriate stack.

    Here is a sample I created for 1 key, need some minor changes:

    We need two things.

    One, a stack which holds the Lambda function. With this we not have a cloudformation resource which calls this Lambda behind the scenes when we add/remove/update a resource. This typically goes into separate stack all together.

    AWSTemplateFormatVersion: '2010-09-09'
    Resources:
        DynamoCfnLambdaRole:
            Type: AWS::IAM::Role
            Properties:
                AssumeRolePolicyDocument:
                    Version: '2012-10-17'
                    Statement:
                        - Effect: Allow
                          Principal:
                              Service:
                                  - lambda.amazonaws.com
                          Action:
                              - sts:AssumeRole
                Path: '/'
                Policies:
                    - PolicyName: dynamodbAccessRole
                      PolicyDocument:
                          Version: '2012-10-17'
                          Statement:
                              - Effect: Allow
                                Action:
                                    - dynamodb:*
                                Resource: '*'
                              - Effect: Allow
                                Action:
                                    - logs:*
                                Resource: '*'
        CfnCrtUpdDltDynamodbDocumentLambda:
            Type: AWS::Lambda::Function
            Properties:
                FunctionName: 'cfn-crt-upd-dlt-dynamodb-document'
                Code:
                    ZipFile: >
                        const AWS = require("aws-sdk");
                        const response = require("./cfn-response");
                        const docClient = new AWS.DynamoDB.DocumentClient();
                        exports.handler = function(event, context) {
                          console.log(JSON.stringify(event, null, 2));
                          var item = JSON.parse(event.ResourceProperties.DynamoItem);
                          var keyProperty = event.ResourceProperties.DynamoKeyProperty;
                          var tableName = event.ResourceProperties.DynamoTableName;
                          if (event.RequestType == "Create" || event.RequestType == "Update") {
                            console.log("item:", item);
                            var params = {
                              TableName: tableName,
                              Item: item
                            };
                            console.log("Creating or Updating Document");
                            docClient.put(params, function(err, data) {                
                              if (err) {
                                console.log('error creating/updating document', err);
                                response.send(event, context, "FAILED", {}, tableName + '_' + item[keyProperty]);
                              } else {
                                response.send(event, context, "SUCCESS", {}, tableName + '_' + item[keyProperty]);
                              }
                            });
                          }
    
                          if (event.RequestType == "Delete") {
                            console.log("Deleting a Document");
                            var params = {
                              TableName: tableName,
                              Key: {
                                [keyProperty]: item[keyProperty]
                              }
                            };
                            docClient.delete(params, function(err, data) {
                              if (err) {
                                response.send(event, context, "FAILED", {});
                              } else {
                                response.send(event, context, "SUCCESS", {});
                              }
                            });
                          }
                        };
                Handler: index.handler
                Role: !GetAtt DynamoCfnLambdaRole.Arn
                Runtime: nodejs10.x
                Timeout: 60
    

    Second, we need to add resource blocks Custom::ThisCouldBeAnything with ServiceToken pointing to Lambda Arn we created above. Whenever we

    • Add a new block in any cloudformation, Lambda will be called with Create, which creates an entry in to Dynamo table.
    • Remove the block from Cloudformation, Lambda will be called with Delete, which delete entry from Dynamo Table.
    • Change something in resource block, example change something in DynamoItem, Lambda will be called with Update.

    We need one resource block for every record that we need to insert into Dynamo.

    Resources:
        MyPhone223546421:
            Type: Custom::CrtUpdDltDynamodbDocumentLambda
            Properties:
                ServiceToken: !GetAtt CfnCrtUpdDltDynamodbDocumentLambda.Arn
                DynamoTableName: My-Table
                DynamoKeyProperty: 'PhoneNumber'
                DynamoItem: |
                    {
                      "PhoneNumber": "223546421",
                      "OrderNumber": "11545154",
                      "someKey": "someValue"
                    }
        MyPhone784578745:
            Type: Custom::CrtUpdDltDynamodbDocumentLambda
            Properties:
                ServiceToken: !GetAtt CfnCrtUpdDltDynamodbDocumentLambda.Arn
                DynamoTableName: My-Table
                DynamoKeyProperty: 'PhoneNumber'
                DynamoItem: |
                    {
                      "PhoneNumber": "784578745",
                      "OrderNumber": "11547854",
                      "someKey": "someValue"
                    }