Search code examples
linuxbashsupervisord

How to check if supervisor process is running or stopped using bash script


I run multiple supervisor processes sometimes due to server overload some processes Stop indefinitely until you manually restart them. Is there a way I can write a bash script that can be regularly executed by a crontab to check the process that has stopped and restart it.

This how I can check status, stop or restart a process on a terminal

root@cloud:~# supervisorctl status birthday-sms:birthday-sms_00
birthday-sms:birthday-sms_00     RUNNING   pid 2696895, uptime 0:02:08
root@cloud:~# supervisorctl stop birthday-sms:birthday-sms_00
birthday-sms:birthday-sms_00: stopped
root@cloud:~# supervisorctl status birthday-sms:birthday-sms_00
birthday-sms:birthday-sms_00     STOPPED   May 07 11:07 AM

I don't want use crontab to restart all at a certain interval */5 * * * * /usr/bin/supervisorctl restart all but I want to check and restart stopped processes only


Solution

  • First make and test a script without crontab that performs the actions you want.
    When the script is working, check that it still runs without the settings in your .bashrc, such as the path to supervisorctl. Also check that your script doesn't write to stdout, perhaps introduce a logfile. Next add the script to crontab.

    #!/bin/bash
    for proc in 'birthday-sms:birthday-sms_00' 'process2' 'process3'; do
      # Only during development, you don't want output from cron
      echo "Checking ${proc}"
      status=$(supervisorctl status "${proc}" 2>&1)
      echo "$status"
      if [[ "$status" == *STOPPED* ]]; then
        echo "Restarting ${proc}"
        supervisorctl restart "${proc}"
      fi
    done
    

    or using an array and shorter testing

    #!/bin/bash
    processes=('birthday-sms:birthday-sms_00' 'process2' 'process3')
    for proc in ${processes[@]}; do
      supervisorctl status "${proc}" 2>&1 |
        grep -q STOPPED &&
        supervisorctl restart "${proc}"
    done