Search code examples
google-bigqueryairflowgoogle-cloud-composerairflow-2.x

airflow: templated argument with BigQueryCheckOperator


I would like to use the BigQueryCheckOperator in airflow where the query needs to have the execution date as parameter. I have written the following operator that check the current date, but I do not understand how to use the execution_date of the dag.

BigQueryCheckOperator(
        task_id='mytask',
        retry_delay=10,
        retries=2,
        sql="""
           SELECT
            COUNT(*)
           FROM
            mydataset.mytable AS tb
           WHERE
            DATE(tb.date) = '{check_date}'
        """.format(
            check_date=(datetime.now()).strftime("%Y-%m-%d")
        ),
        bigquery_conn_id='my_google_cloud_conn',
        dag=dag)

Could you help me?


Solution

  • You can use Jinja template and variable = {{ ds }}

    BigQueryCheckOperator(
            task_id='mytask',
            retry_delay=10,
            retries=2,
            sql="""
               SELECT
                COUNT(*)
               FROM
                mydataset.mytable AS tb
               WHERE
                DATE(tb.date) = DATE('{{ ds }}')""",
            bigquery_conn_id='my_google_cloud_conn',
            dag=dag)