Prevent site being down during updates using nginx and Python

Question

I have an active site that's hosted on Ubuntu, uses nginx, and the site is written in Python (CherryPy is the server, Bottle is the framework).

I have a shell script that copies python files that I upload over the existing live site ones which then of course results in CherryPy restarting the server so it's running the latest code (how I want it). The problem is, in between the time it's stopping and started a default static page is displayed to any unlucky person who tries to view a page on the site at that time (hope they aren't submitting a form). I've seen this page a bunch of times while updating.

My current setup is two copies of the site running on two ports reverse proxied with nginx. So I figured if I update one, wait a few seconds, then update the other then the site will be up 100% of the time, but this doesn't appear to be the case?

Lets say I have reverse proxy on ports 8095 and 8096, both show the same site but two identical copies of it on the hard drive. I update the python files for port 8095 which causes that port to go down while CherryPy restarts it. Shouldn't everyone then be hitting 8096? It doesn't seem to be working like this. I have an 8 second delay in my file copy script and according to CherryPy logs the 2nd stopped to restart 6 seconds after the 1st was already finished restarting, yet I saw the default static offline page that's displayed when the server is down. I'm confused. According to logs there was always one port up.

Here's part of my nginx.conf:

upstream app_servers {
    server 127.0.0.1:8095;
    server 127.0.0.1:8096;
}

server {
    server_name www.mydomain.com; 
    listen 80;

error_page 502 503 /offline/offline.html;

location /offline {
    alias   /usr/share/nginx/html/mysite/1/views/;
}

    location / {
            proxy_pass         http://app_servers;
            proxy_redirect     off;
            proxy_set_header   Host $host;
            proxy_set_header   X-Real-IP $remote_addr;
            proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header   X-Forwarded-Host $server_name;
            proxy_set_header        X-Scheme $scheme;
            proxy_connect_timeout 10;
            proxy_read_timeout 10;
    }
}

Answer 1

Try this: From the manual

upstream app_servers {
server 127.0.0.1:8095 max_fails=1 fail_timeout=1;
server 127.0.0.1:8096 max_fails=1 fail_timeout=1;;
}