I am running Ubuntu 20.04 on an EC2 instance in AWS. For some reason when the server is rebooted, Nginx starts but there is no service listing on port 80. When using SSH I can run sudo service Nginx reload and everything starts running. Ideas, help and suggestions are appreciated.
See output after a fresh reboot.
ubuntu@ip-xxx-xxx-xxx-xxx:~$ ps aux | grep nginx
root 504 0.0 0.1 57184 1460 ? Ss 10:21 0:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
www-data 508 0.0 0.5 57944 5368 ? S 10:21 0:00 nginx: worker process
ubuntu 1039 0.0 0.2 8160 2572 pts/0 S+ 10:44 0:00 grep --color=auto nginx
ubuntu@ip-xxx-xxx-xxx-xxx:~$ sudo service nginx status
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2021-02-09 10:21:59 UTC; 8min ago
Docs: man:nginx(8)
Process: 460 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
Process: 503 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
Main PID: 504 (nginx)
Tasks: 2 (limit: 1164)
Memory: 9.2M
CGroup: /system.slice/nginx.service
├─504 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
└─508 nginx: worker process
Feb 09 10:21:59 ip-xxx-xxx-xxx-xxx systemd[1]: Starting A high performance web server and a reverse proxy server...
Feb 09 10:21:59 ip-xxx-xxx-xxx-xxx systemd[1]: Started A high performance web server and a reverse proxy server.
ubuntu@ip-xxx-xxx-xxx-xxx:~$ sudo netstat -tanpl|grep nginx
ubuntu@ip-xxx-xxx-xxx-xxx:~$ sudo netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 397/systemd-resolve
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 628/sshd: /usr/sbin
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/init
tcp6 0 0 :::22 :::* LISTEN 628/sshd: /usr/sbin
tcp6 0 0 :::111 :::* LISTEN 1/init
udp 0 0 xxx.xxx.xxx.xxx 0.0.0.0:* 397/systemd-resolve
udp 0 0 xxx.xxx.xxx.xxx:68 0.0.0.0:* 394/systemd-network
udp 0 0 0.0.0.0:111 0.0.0.0:* 1/init
udp6 0 0 :::111 :::* 1/init
Update 1
It looks like this is an issue with EFS. Nginx configs were saved on an EFS volume and Nginx is loading before the EFS volume has been mounted. I will continue to investigate and read, updates to follow.
Nginx loads services based on configs set in /lib/systemd/system/ (Path may change depending on the OS version).
It would seem Nginx is starting before Ubuntu mounts the network drives on startup. Because Nginx was told to load all configs from a directory on the EFS volume, it could not load the configs when launched as they do not exist until the filesystems finish mounting.
This can be solved by telling Nginx to wait for the specific file system to be mounted before starting.
Edit the service file by running:
Sudo nano /lib/systemd/system/nginx.service
Add the following line under the [Service] section, amending your mount target as needed:
After=network.target var-www\x2defs.mount
If you do not know the mount target you can do the following to find out, replace dir with the dir you mount the drive to.
systemctl list-units --type=mount | grep /var/www
Article / Post where I found the information to solve the issue: https://unix.stackexchange.com/questions/246935/set-systemd-service-to-execute-after-fstab-mount