Search code examples
amazon-dynamodbaws-step-functions

Storing optional attributes in DynamoDB's putItem via step functions


I have defined a state machine in AWS step functions and one of my states is storing an item to DynamoDB

...
"Store item": {
  "End": true,
  "Type": "Task",
  "Resource": "arn:aws:states:::dynamodb:putItem",
  "Parameters": {
    "Item": {
      "foo": {
        "S.$": "$.data.foo"
      },
      "bar": {
        "S.$": "$.data.bar"
      },
      "baz": {
        "S.$": "$.data.baz"
      },
    },
    "TableName": "nrp_items"
  }
},
...

The problem starts from the fact that baz property is optional, ie not exist in some cases. On those cases, the putItem task fails:

An error occurred while executing the state 'Store item' (entered at the event id #71). > The JSONPath '$.data.baz' specified for the field 'S.$' could not be found in the input

My backup plan is to use a lambda to perform that type of operation, but can I do it directly using the putItem task in steps function?

I was wondering if:

  1. Is possible to somehow inject via JSONPath my whole $.data item to the "Item" property, something like:
...
"Store item": {
  "End": true,
  "Type": "Task",
  "Resource": "arn:aws:states:::dynamodb:putItem",
  "Parameters": {
    "Item": "$.data",
    "TableName": "nrp_items"
  }
},
...

OR
2) Define that the baz property is optional


Solution

  • TL;DR We can deal with optional variables with a "Variable": "$.baz", "IsPresent": true Choice condition to handle no-baz cases.

    The Amazon States Language spec does not have optional properties: Step Functions will throw an error if $.baz does not exist in the input. We can avoid undefined paths by inserting a two-branch Choice State, one branch of which handles baz-exists cases, the other no-baz cases. Each branch continues with a Pass State that reworks the data input into dynamo-format Item syntax, using Parameters. The put-item task's "Item.$": "$.data" (as in your #1) contains only foo-bar when baz is not defined, but all three otherwise.

    {
      "StartAt": "HasBazChoice",
      "States": {
        "HasBazChoice": {
          "Type": "Choice",
          "Choices": [
            {
              "Variable": "$.baz",
              "IsPresent": true,
              "Next": "MakeHasBazItem"
            }
          ],
          "Default": "MakeNoBazItem"
        },
        "MakeHasBazItem": {
          "Type": "Pass",
          "Parameters": {
            "data": {
              "foo": { "S.$": "$.foo"},
              "bar": { "S.$": "$.bar"},
              "baz": { "S.$": "$.baz"}
            }
          },
          "Next": "PutItemTask"
        },
        "MakeNoBazItem": {
          "Type": "Pass",
          "Parameters": {
            "data": {
              "foo": {"S.$": "$.foo"},
              "bar": {"S.$": "$.bar"}
            }
          },
          "Next": "PutItemTask"
        },
        "PutItemTask": {
          ...
          "Parameters": {
            "TableName": "my-table",
            "Item.$": "$.data"
          }
        },
      }
    }
    

    If you have more than one optional field, your lambda backup plan is the better option - the above workaround would become unwieldy.