I have a DAG which has 5 parallel task. I don't want to execute them all in one go. I want to pass some flag/value to the task suppose if the flag is set to True it runs and if the flag is set to False it gets skip. Can we do this in Airflow2. The default arguments are very basic one and I have not scheduled the DAG.
My DAG Flow looks something like this
starttask>>5paralleltask>>endtask
And these 5 parallel task I have created using a for loop
Thank you so much in advance
The 5 parallel task that I have are bash operators.
To skip few task and to run few task I forced the selected task to fail and to show is as skipped I used exit 99
with my bash command like:
'echo "running";exit 99'
There are two steps:
To force fail the task I have added a list of 5 flags(True/False)
if the flag is True the task runs if It is false it get forced failed and then skipped. for skipping the task I used exit 99
.
And it is working as I expected in my case.