Search code examples
webserverairflow

How to run airflow webserver on port 80

When I set airflow webserver to run on port 80, the service is not executed and fails with following error:

...
[2017-08-30 06:26:35,286] {__init__.py:57} INFO - Using executor CeleryExecutor
[2017-08-30 06:26:35,421] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python3.5/lib2to3/Grammar.txt
[2017-08-30 06:26:35,463] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python3.5/lib2to3/PatternGrammar.txt
[2017-08-30 06:26:35 +0000] [9401] [INFO] Starting gunicorn 19.3.0
[2017-08-30 06:26:35 +0000] [9401] [ERROR] Retrying in 1 second.
[2017-08-30 06:26:36 +0000] [9401] [ERROR] Retrying in 1 second.
[2017-08-30 06:26:37 +0000] [9401] [ERROR] Retrying in 1 second.
[2017-08-30 06:26:38 +0000] [9401] [ERROR] Retrying in 1 second.
[2017-08-30 06:26:39 +0000] [9401] [ERROR] Retrying in 1 second.
[2017-08-30 06:26:40 +0000] [9401] [ERROR] Can't connect to ('0.0.0.0', 80)
...

Using systemd on Ubuntu 16.04 hosted on AWS. The whole setup works well if run on port 8080.

Relevant config part:

$ grep web_server_port /home/ubuntu/airflow/airflow.cfg
web_server_port = 80

Service configuration:

$ cat /usr/lib/systemd/system/airflow-webserver.service 
[Unit]
Description=Airflow webserver daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
User=ubuntu
Group=ubuntu
Type=simple
ExecStart=/usr/local/bin/airflow webserver --pid /home/ubuntu/airflow/webserver.pid
Restart=on-failure
RestartSec=5s
PrivateTmp=true

[Install]
WantedBy=multi-user.target
Solution

  • The problem was that port 80 can be used by root only. The only changes in /usr/lib/systemd/system/airflow-webserver.service needed to fix the problem were:

    • removing User and Group: root is the default value
    • setting airflow home environment variable: because otherwise the service looked for airflow in /root/airflow

    New service configuration:

    $ cat /usr/lib/systemd/system/airflow-webserver.service 
[Unit]
Description=Airflow webserver daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
Environment=AIRFLOW_HOME=/home/ubuntu/airflow
Type=simple
ExecStart=/usr/local/bin/airflow webserver --pid /home/ubuntu/airflow/webserver.pid
Restart=on-failure
RestartSec=5s
PrivateTmp=true

[Install]
WantedBy=multi-user.target