Search code examples
kubernetesairflowkubernetes-helm

How to run airflow CLI commands with airflow/kubernetes installed from Helm stable/airflow?

Difficulty running airflow commands when running Airflow on Kubernetes that I installed from the Helm stable/airflow repo. For instance I try to exec into the scheduler pod and run airflow list and I get the following error:

airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the KubernetesExecutor airlow

Ok so I switch to the celery executor.

Same thing

airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the CeleryExecutor

So what is the correct way to run airflow CLI commands when running on K8s?

Solution

  • Make sure you are using bash. /home/airflow/.bashrc imports the environment variables from /home/airflow/airflow_env.sh to setup the connection. The following are some examples:

    kubectl exec -ti airflow-scheduler-nnn-nnn -- /bin/bash
$ airflow list_dags

    Or with shell you can import the env vars yourself:

    kubectl exec -ti airflow-scheduler-nnn-nnn -- sh -c ". /home/airflow/airflow_env.sh && airflow list_dags"