How to minimise downtime and user facing errors while restarting Mysql Server and Rails servers on Cloud66?

Question

What are the best practises for restarting mysql and rails servers on Cloud66, to minimise the possibilities of user's seeing errors?

Background Sometimes logging into a server via ssh, there is a message system restart required Sometimes there are security upgrades that require a restart.

Current Approach

My current approach for mysql is to:

1. Put the app into maintenance mode
2. use bluepill commands to stop the mysql
3. restart server with "shutdown -r now" command
4. After server has booted up, confirm that bluepill mysql service is running
5. put the app back into non-maintenance mode

My current approach for rails servers is to:

1. use bluepill commands to stop the web service (this is due to the fact that I have multiple web servers so the app can keep on running using alternative web servers)
2. restart server with "shutdown -r now" command
3. After server has booted up, confirm that bluepill web service is running

How do you do it?

Answer 1

Your approach sounds very reasonable. In most cases, a restart is required when an update to the Linux kernel has been installed, which usually contains security updates and only come into effect after a reboot. So restarting is good practice, you just need to determine the best time for doing this.

To determine what's being updated (and as such the urgency of reboot), you could always run cat /var/run/reboot-required.pkgs. Apart from being notified of a required restart when you SSH into the server, you can also check for the existence of the reboot-require file by running ls /var/run/reboot-required.

To add to your approach, you may also want to stop any background tasks that will be connecting to your database when you restart it.