Search code examples
amazon-cloudwatchaws-glue

AWS Glue - Cloudwatch events - How to get custom parameters on Job event triggers


I am creating a cloudwatch event rule which triggers when there is a "Glue job state change". All works good but the payload from cloud watch event doesnt seem to have the custom parameters I have added as part of the job.

aws glue start-job-run --job-name test_dev_eventbridge --arguments keyName1=testValue1,keyName2=testValue2 --region us-east-1 --output json

Sample event I see from cloud watch:

{
"version": "0",
"id": "7e26551f-ef2b-3b1f-6bbc-e89dcd189f72",
"detail-type": "Glue Job State Change",
"source": "aws.glue",
"account": "xxxxxxxxxx",
"time": "2020-03-29T00:43:05Z",
"region": "us-east-1",
"resources": [],
"detail": {
"jobName": "test_dev_eventbridge",
"severity": "INFO",
"state": "SUCCEEDED",
"jobRunId": "c0cf9f8d1ed",
"message": "Job run succeeded"
}
}

Appreciate if some one could let me know a way to receive customer parameters as part of the event payload from cloud watch trigger.


Solution

  • If this is attached to a lambda, you can use this event and using the boto3 glue client, run the get_job_run method. The result of this call should give u everything u need including arguments