I need to write code of AWS CloudFormation Template for AWS Glue, where I am creating single template for multiple Glue job. I added predicate condition, for second job onwards. Unfortunately though predicate accept multiple conditions, it accepts only one state for previous job, as in if previous job state succeeded or failed or stopped, still next job should run. But I need it to run in multiple state as in even if previous job has any of the succeeded/failed/stopped state next job should add.
If I give multiple conditions, it gives error saying duplicate job exist, as I am giving same job name in all condition
I wrote below code snippet in template
'''
Resources:
WF:
Type: AWS::Glue::Workflow
Properties:
Description: workflow for Parallel Jobs
Name: Sequence_Job_Triggers
SequenceTrigger:
Type: AWS::Glue::Trigger
Properties:
Name: Job_Trigger_1
Type: ON_DEMAND
Description: Description
WorkflowName: !Ref WF
Actions:
- JobName: job1
SequenceTrigger:
Type: AWS::Glue::Trigger
Properties:
Name: Job_Trigger_2
Type: CONDITIONAL
Description: Description
WorkflowName: !Ref WF
Predicate:
Conditions:
- LogicalOperator: EQUALS
JobName: job1
State: SUCCEEDED
Actions:
- JobName: job2
'''
Can't you just create two triggers, at least this works for me:
SucceededSequenceTrigger:
Type: AWS::Glue::Trigger
Properties:
Name: Succeeded_Job_Trigger_2
Type: CONDITIONAL
Description: Description
WorkflowName: !Ref WF
Predicate:
Conditions:
- LogicalOperator: EQUALS
JobName: job1
State: SUCCEEDED
Actions:
- JobName: job2
FailedSequenceTrigger:
Type: AWS::Glue::Trigger
Properties:
Name: Failed_Job_Trigger_2
Type: CONDITIONAL
Description: Description
WorkflowName: !Ref WF
Predicate:
Conditions:
- LogicalOperator: EQUALS
JobName: job1
State: FAILED
Actions:
- JobName: job2