When building an Airflow dag, I typically specify a simple schedule to run periodically - I expect this is the most common use.
dag = DAG('my_dag',
description='this is what it does',
schedule_interval='0 12 * * *',
start_date=datetime(2017, 10, 1),
catchup=False)
I then need to use the 'date' as a parameter in my actual process, so I just check the current date.
date = datetime.date.today()
# do some date-sensitive stuff
operator = MyOperator(..., params=[date, ...])
My understanding is that setting catchup=True
will have Airflow schedule my dag for every schedule interval between start_date
and now (or end_date
); e.g. every day.
How do I get the scheduled_date
for use within my dag instance?
I think you mean execution date here, You can use Macros inside your operators, more detail can be found here: https://airflow.apache.org/docs/apache-airflow/2.10.3/templates-ref.html. So airflow will respect it so you don't need to have your date
been generated dynamically
Inside of Operator, you can call {{ ds }}
in a str directly
Outside of Operator, for example PythonOperator, you will need provide_context=True
first then to pass **kwargs
as last arguments to your function then you can call kwargs['ds']