Search code examples
ruby-on-railsubuntu-20.04passenger-nginx

Ubuntu 20.04 + Nginix 1.20.2 + Passenger 6.0.12 dynamic module + Rails

summary of issue: after compiling nginx/dynamic passenger module by hand, I can't tell if passenger is starting up or what the problem is serving my application, but no matter what page I try I get 403 forbidden or 404 not found.

details: I had a working rails app under the passenger+nginx bundle but due to security alert on nginx 1.18 and my company's security policy, I had to clear out my nginx setup and start over, manually compiling everything according to this link. After a half day of struggling, nginx will start however when i try to access my site via browser, I get this in the error log:

2022/02/24 00:03:06 [error] 156967#156967: *3 directory index of "/home/<app name>/staging/current/public/" is forbidden, client: <client ip addr>, server: <server ip addr>, request: "GET / HTTP/1.1", host: "<server ip addr>"

What I think is happening is that passenger is starting but is having some error when it tries to run the rails app. But I can't be sure since

  1. the error message is completely uninformative and I suppose that if passenger were not starting, nginx would fail with a similar message.
  2. i can't run passenger-status, I get an error as per this question that passenger can't find its instance registry folder. Setting the appropriate env. variable PASSENGER_INSTANCE_REGISTRY_DIR to /tmp didn't fix this for me even though I do see passenger files are created there.

I have these settings in my nginx.conf:

load_module modules/ngx_http_passenger_module.so

http {
:
passenger_root /home/<app name>/.rvm/gems/ruby-2.6.5/gems/passenger-6.0.12
passenger_ruby /home/<app name>/.rvm/gems/ruby-2.6.5/wrappers/ruby
:

I have these settings in sites-enabled/default:

root /home/<app name>/staging/current/public;
passenger_enabled on;
passenger_app_env staging;

And I am able to run passenger standalone by going to the rails apps directory and running RAILS_ENV=staging passenger start. No apparent problems starting the app. Nor with the rails server.

How can I diagnose what's going on? How can I know if passenger is even the source of the error log entry? I am totally stuck after 2 days of trying everything I can find in SO.

thanks in advance for any help.

edits After my initial question, I noticed that sudo service nginx status was complaining that Passenger was not correctly installed/compiled. However, I cleared out everything (nginx, passenger) and set it up again from scratch, and I am back to the above problems, but there is NO error about passenger when I now check the status.

I also became convinced Passenger is not being called at all because if I put a "hello, world" in index.html in my app's public directory, it gets served by Nginx. Nor do I see Passenger processes when I check passenger=memory-stats. So I think Passenger is not starting up.

Solution

  • I answered my own question.

    A moment after the last edit, I fixed a minor problem (I had temporarily commented out passenger_root and passenger_ruby directives in my nginx.conf file) and then everything was working.

    So now I will outline the whole procedure I used to upgrade both Nginx and Phusion passenger to latest stable versions on Ubuntu 20.04 and hopefully it will help someone out in the future. In my case it was because Nginx 1.20+ was mandated but Ubuntu's apt only provided 1.18, which had a known vulnerability from about 6 months ago.

    This procedure is needed when (as it has been for several months) the passenger-nginx package supplied by Phusion is using an outdated, vendor-supplied Nginx.

    Step 1. Purge existing ubuntu-supplied nginx and passenger:

    sudo apt-get remove nginx*
sudo apt-get remove passenger

    Step 2. Download latest stable nginx (in my case 1.20.2) from nginx.org's apt repository instead of Ubuntu. I followed the instructions at this link under the Ubuntu heading.

    Step 3. I already had ruby managed by rvm, but you should install that or whatever app environment that you need. My instructions going forward are for ruby/rails.

    Step 4. Install passenger through ruby gem command:

    gem install passenger

    Step 5. Go to passenger directory and rake it:

    cd ~/.rvm/gems/ruby-2.6.5/gems/passenger-6.0.12
rvmsudo rake  nginx:as_dynamic_module CACHING=false

    Note use of rvmsudo here. Because passenger was installed in the rvm directory then rvmsudo should be used instead of sudo with respect to passenger executables.

    Step 6. Download nginx source code to match the stable version you installed in step 2: As per this page you get it from this URI:

    https://nginx.org/download/nginx-${NGINX_VERSION_HERE}.tar.gz

    and unpack it with tar -xf nginx-${NGINX_VERSION_HERE}.tar.gz

    Step 7. As per the above link, get the compatible directives for your upcoming compile. Run sudo nginx -V which returns:

    nginx version: nginx/1.20.2
built by gcc 9.3.0 (Ubuntu 9.3.0-10ubuntu2) 
built with OpenSSL 1.1.1f  31 Mar 2020
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --wi...

    EDIT copy everything from "--prefix" onwards.

    Step 8. Configure the compile. Go to the nginx source directory and execute:

    ./configure {paste from clipboard} --add-dynamic-module=$(passenger-config --nginx-addon-dir)
make
sudo make install

    Step 9. Configure nginx:

    cd /etc/nginx
sudo mkdir modules-enabled  #if directory not already there
cd modules-enabled
sudo ln -s /usr/lib/nginx/modules/ngx_http_passenger_module.so ngx_http_passenger_module.so

    the last line puts a symbolic link to the created dynamic library for passenger, that was created at the location defined by --modules-path parameter of step 7.

    then, make sure nginx sees the module, and knows where to find itself and the ruby interpreter needed by Passenger. In nginx.conf, make sure these lines are there:

    load_module modules/ngx_http_passenger_module.so

http {
:
passenger_root /home/<app name>/.rvm/gems/ruby-2.6.5/gems/passenger-6.0.12
passenger_ruby /home/<app name>/.rvm/gems/ruby-2.6.5/wrappers/ruby
:

    of course the exact location depends on your rvm version, and passenger version.

    Step 10. Configure your site. This is beyond the scope of this, but the key lines to have are:

       :
   root /home/<app name>/staging/current/public;
   passenger_enabled on;
   passenger_app_env staging;  #rails environment (not needed for production, only development or staging)
   :
   }