Search code examples
amazon-web-servicesaws-step-functions

How can I use Choice to test for if an input array contains a given filter?


Given a Map state thats output is an array similar to the following:

[
  {
    "ProcessState": {
      "Status": "SUCCESS"
    }
  },
  {
    "ProcessState": {
      "Status": "SUCCESS"
    }
  },
  {
    "ProcessState": {
      "Status": "FAILURE"
    }
  }
]

I would like to be able to test if there is an element with Status = 'FAILURE'. I attempted to use a Choice with a choice as follows:

{
  "Variable": "$..ProcessState[?(Status == FAILURE)]",
  "IsPresent": true,
  "Next": "Items Contained Failure"
}

When attempting this I get Value is not a Reference Path: Illegal '..' ...

I'm thinking to attempt to use a Pass as an intermediate step, but I think that's just going to fail that it can't find anything if no entries match.


Solution

  • This has been solved by having the Map state ResultSelector to perform the filtering.

    "ResultSelector": {
      "QueueFailures.$": "$[?(@.ProcessState.Status == 'FAILED')]"
    },
    "ResultPath": "$.ProcessResult"
    

    The Choice state can now just test for the presence of the first failure item.

    "Test Queue Failures": {
      "Type": "Choice",
      "Default": "Mark Decision Run Ready",
      "Choices": [
        {
          "Variable": "$.ProcessResult.QueueFailures[0]",
          "IsPresent": true,
          "Next": "Queue Contains Failures"
        }
      ]
    }
    

    While illustrating the solution but omitting much of the business case, the following puts this in the fuller context:

    {
      // register our exit states
      "Failure Path": { "Type": "Fail"  },
      "Happy Path":   { "Type": "Succeed" },
      
      // map process (much removed)
      "My Map Task": {
        "Type": "Map",
        "Next": "Test Queue Failures",
        "ItemsPath": "$.Queue.Items",
        "Parameters": {
          "Item.$": "$$.Map.Item.Value"
        },
    
        "ResultSelector": {
          // this selector takes the error info from a batch job task.
          "QueueFailures.$": "$[?(@.ErrorInfo)]",
          "QueueItems.$": "$"
        },
        "ResultPath": "$.ProcessResult",
    
        "Iterator": {
          "StartAt": "First Map Item Task",
          "States": {
            "First Map Item Task": {
              "Type": "Task",
              "Resource": "arn:aws:states:::batch:submitJob.sync",
              "Next": "Complete Entry",
              // this is the key to storing the error info from the batch job
              "Catch": [
                {
                  "ResultPath": "$.ErrorInfo",
                  "ErrorEquals": ["States.ALL"],
                  "Next": "Fail Entry"
                }
              ]
              // result selector and path omitted
            },
            
            // the following two lambdas allow me to capture state via a lambda
    
            // happy path for a single item iteration.
            "Complete Entry": {
              "Type": "Task",
              "Resource": "arn:aws:states:::lambda:invoke",
              "End": true,
              "OutputPath": "$.Payload"
            },
    
            // sad path for a single item iteration
            "Fail Entry": {
              "Type": "Task",
              "Resource": "arn:aws:states:::lambda:invoke",
              "End": true,
              "OutputPath": "$.Payload"
            }
          }
        }
      },
      
      // here we check for our fail state by finding at least 1 array being present.
      "Test Queue Failures": {
        "Type": "Choice",
        "Default": "Happy Path",
        "Choices": [
          {
            "Variable": "$.ProcessResult.QueueFailures[0]",
            "IsPresent": true,
            "Next": "Queue Contains Failures"
          }
        ]
      }
    
    }