How to have a Python glue job return when called in step function?

Question

I have a glue job, in python, that I call from a step function. The step function successfully starts the job. The job successfully finishes. But the step function never moves to the next step. Is there some required configuration/permission for the step function to respond to job success? Something to do in the python script?

Here is the step function (state machine) definition:

"MyGlueTask": {
  "Type": "Task",
  "Resource": "arn:aws:states:::glue:startJobRun.sync",
  "Parameters": {
    "JobName": "my_glue_job"
  },
  "ResultPath": "$.MyGlueTask",
  "Next": "NextGlueJob"
}

Answer 1

The solution to my actual problem was permissions. You need four permissions when running a startJogRun.sync:

• glue:StartJobRun
• glue:GetJobRun
• glue:GetJobRuns
• glue:BatchStopJobRun

Those are actually the Terraform values, but should help anybody struggling with this.