Search code examples
google-cloud-platformairflowschedulerdirected-acyclic-graphs

Continuously run a DAG after 2 minutes of completion of the previous run in Airflow

I want to know if we can schedule a DAG to run continuously after 2 minutes of completion of the same DAG in Airflow.

Edit:

My DAG should run in such a way that every time it completes its run, it has to wait for 2 minutes and start running again. I don't want to schedule my DAG to run for every 2 minutes instead it should continuously run right after 2 minutes of completion of the same DAG.

Solution

  • You could schedule your dag at an arbitrary time in a day and use the TriggerDagRunOperator to trigger itself again. To wait for 2 minutes before triggering itself, you could simply introduce a sleep task.

    DAG:

    Task 1 >> Task 2 >> Task 3 BashOperator(bash_command="sleep 120") >> Task 4 TriggerDagRunOperator(trigger_dag_id="this-dag-id")