Step Function with EventBridge task `"NotAuthorizedForSourceException"`

Question

Background

I'm creating a Step Function state machine that starts an AWS CodeBuild once a defined AWS CodePipeline has an execution status of SUCCEED. I'm using the .waitForTaskToken feature within the Step Function to wait on the CodePipeline to succeed via a CloudWatch event. Once the pipeline succeeds, the event sends back the token to the step function and runs the CodeBuild.

Here's the step function definition:

{
  "StartAt": "PollCP",
  "States": {
    "PollCP": {
      "Next": "UpdateCP",
      "Parameters": {
        "Entries": [
          {
            "Detail": {
              "Pipeline": [
                "bar-pipeline"
              ],
              "State": [
                "SUCCEEDED"
              ],
              "TaskToken.$": "$$.Task.Token"
            },
            "DetailType": "CodePipeline Pipeline Execution State Change",
            "Source": "aws.codepipeline"
          }
        ]
      },
      "Resource": "arn:aws:states:::events:putEvents.waitForTaskToken",
      "Type": "Task"
    },
    "UpdateCP": {
      "End": true,
      "Parameters": {
        "ProjectName": "foo-project"
      },
      "Resource": "arn:aws:states:::codebuild:startBuild.sync",
      "Type": "Task"
    }
  }
}

The permissions for the step function:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": "codebuild:StartBuild",
            "Resource": "*"
        },
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": "codepipeline:*",
            "Resource": "*"
        }
    ]
}

and arn:aws:iam::aws:policy/CloudWatchEventsFullAccess

Problem

The cloudwatch event within the step function returns the error:

Error

EventBridge.FailedEntry
Cause

{
  "Entries": [
    {
      "ErrorCode": "NotAuthorizedForSourceException",
      "ErrorMessage": "Not authorized for the source."
    }
  ],
  "FailedEntryCount": 1
}

Attempts:

1. Modify the associated Codepipeline and Codebuild role to have step function permission to send task statuses. Specifically, the permission is:

{
            "Effect": "Allow",
            "Action": [
                "states:SendTaskSuccess",
                "states:SendTaskFailure",
                "states:SendTaskHeartbeat"
            ],
            "Resource": "*"
}

Got the same original error mentioned above.

2. Modify the associated Step Function machine's permission to have full access to all step function actions and resources. Got the same original error mentioned above.

3. Test the event rule specified in the PollCP step function task with the default AWS EventBridge bus. The event was:

{
  "version": "0",
  "detail-type": "CodePipeline Pipeline Execution State Change",
  "source": "aws.codepipeline",
  "account": "123456789012",
  "time": "2021-06-14T00:44:41Z",
  "region": "us-west-2",
  "resources": [],
  "detail": {
    "pipeline": "<pipeline-arn>",
    "state": "SUCCEED"
  }
}

The event outputted the same error mentioned above. This probably means the error is strictly related to the event entry mentioned in the code snippet above.

Answer 1

Are you trying to trigger your State Machine using CloudWatch event when a CodePipeline pipeline completes/succeeds?

If so, you cannot define your trigger in your state machine. The integration with EventBridge is not so that state machine can be triggered by events. Rather it is to Publish events to an event bus from your state machine or workflow.

Read more here: https://aws.amazon.com/blogs/compute/introducing-the-amazon-eventbridge-service-integration-for-aws-step-functions/

So I suggest you create a CloudWatch rule and target your state machine instead.

If you want to use the waitForTaskToken pattern. You will have to explicitly return that token with a send_task_success API call (sample below for python/boto3).

sfn.send_task_success(
    taskToken=task_token,
    output=json.dumps(some_optional_payload)
)

This means that, when the step executes, it will publish the event to EventBridge bus. You will have to detect this event outside your state machine, most likely with an CloudWatch event rule. Then trigger a lambda function from the rule. The lambda function performs the send_task_success API call which restarts/continues your workflow/state machine.

In my opinion, that is just unnecessary. Like I said, you can simply watch for pipeline execution state changes uing CW event rule, trigger your state machine, and your state machine starts with the CodeBuild stage.

Side note: It's nice to see people using Step Functions for CI/CD pipeline. It's just got more flexibility and ability to do complex branching strategies. Will probably do a blog post around this soon.