Search code examples
ubuntuserviceupstartairflow

Airflow upstart script enters stop/waiting state immediately after start/running state


I'm trying to run this upstart script from the official repo for airflow: https://github.com/apache/incubator-airflow/blob/master/scripts/upstart/airflow-webserver.conf

start on started networking
stop on (deconfiguring-networking or runlevel [016])

respawn
respawn limit 5 30

setuid airflow
setgid airflow

exec usr/local/bin/airflow webserver

When I run it this is the output:

$ sudo service airflow-webserver start
airflow-webserver start/running, process 7612

$ sudo service airflow-webserver status
airflow-webserver stop/waiting

But just doing exec /usr/local/bin/airflow webserver will run it perfectly without any hiccups, which is puzzling. Does anyone know why this is happening?


* You can get airflow with pip install airflow to test in case you need to see for yourselves.


Solution

  • Upstart runs in a clean environment, meaning that it won't use the variables in /etc/environment. This means you have to set AIRFLOW_HOME and AIRFLOW_CONFIG again.

    description "Airflow webserver daemon"
    
    start on started networking
    stop on (deconfiguring-networking or runlevel [016])
    
    respawn
    respawn limit 5 30
    
    setuid airflow
    setgid airflow
    
    # I omitted all of the below because I assumed 
    # it would pick it up from the already defined env
    env AIRFLOW_CONFIG=/path/to/airflow/airflow.cfg
    env AIRFLOW_HOME=/path/to/airflow
    export AIRFLOW_CONFIG
    export AIRFLOW_HOME
    
    exec usr/local/bin/airflow webserver