Search code examples
dockerairflowairflow-scheduler

new dags not shown on docker airflow

Problem: new dags not shown on docker airflow, no error when running airflow dags list-import-errors

Docker image: official airflow image

Dags path inside docker-compose.yaml (this is the default path):

  volumes:
    - ./dags:/opt/airflow/dags

I put the dag file inside the dags folder on the main directory, as shown below: enter image description here

However, the dags is still not shown on both webserver UI and airflow dags list. Running airflow dags list-import-errors also yield no result.

When I open the docker terminal, I can see my dag inside the dags folder via ls command. I also tried to make the owner root by using chown, but both of my dag still not shown up on the list.

The airflow run successfully (via docker compose) as I can see example dags, but not my own dags.

Any help will be appreciated. Thanks!

Solution

  • I would try a few things:

    • rename files to sth like gcp_dag.py and python_dag.py
    • ensure import airflow is present in each file
    • ensure you create DAG object in each file
    • add __init__.py (empty) file to dags folder

    It would be also helpful to see contents of at least one of those files.