I am currently running a large rails application on a virtual server. My setup uses Unicorn to serve the Rails App with nginx as the proxy server in front.
Occasionally I get notifications that My CPU usage has pegged for a long amount of time. Then I login and use htop to find the badly behaving process to kill it. Those misbehaving processes always seem to be unicorn workers.
What I am wanting to know is, if there is a way to setup monit, or god to monitor these 'child processes' of unicorn and kill them if they hang at high cpu usage for a long period of time. I know how to monitor the unicorn master process because it writes it's process id to a PID file. But it is never the master process, only the child processes the misbehave.
You might want to try the process outlined here: Where Unicorns go to die: Watching unicorn workers with monit
I haven't done it yet, but I'm going to give it a shot shortly.
Edit: It works really well. I'm using the methods there in production now.