Search code examples
pythonairflowetldirected-acyclic-graphs

how to run automatically apache airflow tasks


I'm using Apache Airflow to schedule ETL jobs which are python scripts. When I create dags on airflow, it sets dags status on off. My code are like this.

import airflow
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta

default_args = {
    'owner': 'oguz',
    'depends_on_past': False,
    'start_date': datetime(2018, 1, 25),
    'email': ['airflow@airflow.com'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
    'end_date': datetime(9999, 1, 1),
}

dag = DAG('%s', default_args=default_args, schedule_interval='@daily')

# t1 and t2 are examples of tasks created by instantiating operators
t1 = BashOperator(
    task_id='%s',
    bash_command='python /bookmark/ETL/extract/incremental/%s.py',
    dag=dag)

t2 = BashOperator(
    task_id='%s',
    bash_command='python /bookmark/ETL/load/incremental/%s.py',
    retries=3,
    dag=dag)

t2.set_upstream(t1)

I've searched on airflow documentation but I couldn't find anything.

How can I run automatically airflow dags?

Thanks,


Solution

  • If you toggle the status to "ON" in the UI, it should start running your DAG based on the given interval (daily in your case). If you want new DAGs to be enabled by default, you can update set dags_are_paused_at_creation = False under [core] in your airflow config.