Search code examples
airflow

How can I retrieve the 'scheduled time' for catchup jobs in Airflow?


When building an Airflow dag, I typically specify a simple schedule to run periodically - I expect this is the most common use.

dag = DAG('my_dag',
      description='this is what it does',
      schedule_interval='0 12 * * *',
      start_date=datetime(2017, 10, 1),
      catchup=False)

I then need to use the 'date' as a parameter in my actual process, so I just check the current date.

date = datetime.date.today()
# do some date-sensitive stuff
operator = MyOperator(..., params=[date, ...])

My understanding is that setting catchup=True will have Airflow schedule my dag for every schedule interval between start_date and now (or end_date); e.g. every day.

How do I get the scheduled_date for use within my dag instance?


Solution

  • I think you mean execution date here, You can use Macros inside your operators, more detail can be found here: https://airflow.apache.org/docs/apache-airflow/2.10.3/templates-ref.html. So airflow will respect it so you don't need to have your date been generated dynamically

    Inside of Operator, you can call {{ ds }} in a str directly

    Outside of Operator, for example PythonOperator, you will need provide_context=True first then to pass **kwargs as last arguments to your function then you can call kwargs['ds']