Search code examples
airflowcicd

Is there any way to check for import errors on airflow dag deploy

I have a python script that copies dag to airflow server via ssh and in 10 minutes or so it can be found in airflow ui. But if the dag has errors then it's not shown in "All" tab and a warning banner with "DAG import errors" appears. Is there any way to force dag validation on deploy so i can fail my deploy script if there're any errors?

Solution

  • Not sure if you're using the Airflow CLI or Astro CLI, but the Astro CLI contains pytest commands, read here and the Astro CLI will initialize all projects with a sample pytest file that checks for import errors when your Airflow deployment is deployed.

    More recent versions of Airflow also have the dag.test() command that can be added to your DAG file. You can read about this here.

    To use the dag.test() just add the following to the bottom of your file

    if __name__=="__main__":
    dag.test()

    then run in bash

    > python your_dag_file.py

    This should pick up any import errors.