Copy all keys or remove some keys in AWS Step Function Task - Input and Output Processing

Question

I have an AWS Step Function that is receiving the following payload.

{
  "appID": "1234",
  "eventType": "HOME",
  "name": {
    "firstName": "John/Jane",
    "lastName": "Doe"
  },
  "phone":"555-555-1212" <--- This field is optional, and this is what is causing a problem
}

The first two (Pass) tasks augment the payload and add Generated and DestinationVariables

{
  "appID": "1234",
  "eventType": "HOME",
  "name": {
    "firstName": "John/Jane",
    "lastName": "Doe"
  },
  "Generated": {
    "Request": {
      "appID": "1234"
    },
    "Response": {
      "eventID": "a036b28e-8ff8-4924-9cf1-86ca99570648"
    }
  },
  "DestinationVariables": {
    "Region": "us-east-2",
    "Environment": "Test",
    "DynamoDBTable": "Orders"
  }
}

I have subsequent tasks that extract and act on the Generated and DestinationVariables data which work fine, but I have an additional step further down in the workflow where I need to work on just the original input.

I tried to edit payload in another Pass task with the following (Transform Input with Parameters)

{
  "appID.$": "$.appID",
  "eventType.$": "$.eventType",
  "name.$": "$.name",
  "phone.$": "$.phone"
}

but when 'phone' is missing I get this error;

An error occurred while executing the state 'Pass (1)' (entered at the event id #10). The JSONPath '$.phone' specified for the field 'phone.$' could not be found in the input '{"AppID":"1234","eventType":"TEST","name":{"firstName":"John/Jane","lastName":"Doe"},"Generated":{"Request":{"appID":"1234"},"Response":{"eventID":"a036b28e-8ff8-4924-9cf1-86ca99570648"}},"DestinationVariables":{"Region":"us-east-2","Environment":"Test","DynamoDBTable":"Orders"}}'

The solutions I'm thinking about, none of which I can figure out how to do are;

1. Define a Task to either remove Generated and DestinationVariables from the input.
2. Define a Task to copy everything except Generated and DestinationVariables to another json key, such as originalInput
3. Define a Task before Generated and DestinationVariables get added to copy the entire input to originalInput so it's available later.

Plan 4 would be a Lambda function but that seems like such overkill. Does anyone have any thoughts? Thanks!

Answer 1

You can reference the original input to your workflow from any state using the Context Object. Specifically, $$.Execution.Input.

You might also want to use a Choice state to generate a placeholder for phone if it's not provided in the input. This is another way to manage cases of optional input when you don't want to have logic throughout your workflow to compensate.

Below is an example that both generates a default empty string for phone if it's not provided, then references that later. And follows that by a reference to the original input.

{
  "StartAt": "Check for Phone Number",
  "States": {
    "Check for Phone Number": {
      "Type": "Choice",
      "Choices": [
        {
          "Not": {
            "Variable": "$.phone",
            "IsPresent": true
          },
          "Next": "Provide Default Phone Number"
        }
      ],
      "Default": "Reference Phone Number from State Payload"
    },
    "Provide Default Phone Number": {
      "Type": "Pass",
      "ResultPath": "$.phone",
      "Result": "",
      "Next": "Reference Phone Number from State Payload"
    },
    "Reference Phone Number from State Payload": {
      "Type": "Pass",
      "Parameters": {
        "phone.$": "$.phone"
      },
      "Next": "Reference Phone Number from Context Object"
    },
    "Reference Phone Number from Context Object": {
      "Type": "Pass",
      "End": true,
      "Parameters": {
        "phone.$": "$$.Execution.Input.phone"
      }
    }
  }
}