Search code examples
linuxbashinotify

Bash trap not killing children, causes unexpected ctrl-c behavior


edit

For future readers. The root of this problem really came down to running the function in an interactive shell vs. putting it in a separate script.

Also, there are many things that could be improved in the code I originally posted. Please see comments for things that could/should have been done better.

/edit

I have a bash function intended to rerun a process in the background when files in a directory change (think like Grunt, but for general purposes). The script functions as desired while running:

  • The subprocess is correctly started (including any children)
  • On file change, the sub is killed (including children) and started again

However, on exit (ctrl-c) none of the processes are killed. Additionally, pressing ctrl-c a second time will kill the current terminal session. I'm assuming this is a problem with my trap, but have been unable to identify a reason for the issue.

Here is the code of rerun.sh

#!/bin/bash
# rerun.sh

_kill_children() {
    isTop=$1
    curPid=$2
        # Get pids of children
    children=`ps -o pid --no-headers --ppid ${curPid}`
    for child in $children
    do
            # Call this function to get grandchildren as well
            _kill_children 0 $child
    done
    # Parent calls this with 1, all other with 0 so only children are killed
    if [[ $isTop -eq 0 ]]; then
            kill -9 $curPid 2> /dev/null
    fi
}

rerun() {
    trap " _kill_children 1 $$; exit 0" SIGINT SIGTERM
    FORMAT=$(echo -e "\033[1;33m%w%f\033[0m written")
    #Command that should be repeatedly run is passed as args
    args=$@
    $args &

    #When a file changes in the directory, rerun the process
    while inotifywait -qre close_write --format "$FORMAT" .
    do
        #Kill current bg proc and it's children
        _kill_children 1 $$
        $args & #Rerun the proc
    done
}

#This is sourced in my bash profile so I can run it any time

To test this, create a pair of executable files parent.sh and child.sh as follows:

#!/bin/bash
#parent.sh
./child.sh

#!/bin/bash
#child.sh
sleep 86400

Then source the rerun.sh file and run rerun ./parent.sh. In another terminal window I watch "ps -ef | grep pts/4" to see all processes for the rerun (in this example on pts/4). Touching a file in the directory triggers a restart of parent.sh and children. [ctrl-c] exits, but leaves the pids running. [ctrl-c] again kills bash and all other processes on pts/4.

Desired behavior: on [ctrl-c], kill children and exit to shell normally. Help?

-- Code sources:

Inotify idea from: https://exyr.org/2011/inotify-run/

Kill children from: http://riccomini.name/posts/linux/2012-09-25-kill-subprocesses-linux-bash/


Solution

  • This isn't a good practice to follow in the first place. Track your children explicitly:

    children=( )
    foo & children+=( "$!" )
    

    ...then, you can kill or wait for them explicitly, referring to "${children[@]}" for the list. If you want to get grandchildren as well, this is a good user for fuser -k and a lockfile:

    lockfile_name="$(mktemp /tmp/lockfile.XXXXXX)" # change appropriately
    trap 'rm -f "$lockfile_name"' 0
    
    exec 3>"$lockfile_name" # open lockfile on FD 3
    kill_children() {
        # close our own handle on the lockfile
        exec 3>&-
    
        # kill everything that still has it open (our children and their children)
        fuser -k "$lockfile_name" >/dev/null
    
        # ...then open it again.
        exec 3>"$lockfile_name"
    }
    
    rerun() {
        trap 'kill_children; exit 0' SIGINT SIGTERM
        printf -v format '%b' "\033[1;33m%w%f\033[0m written"
    
        "$@" &
    
        #When a file changes in the directory, rerun the process
        while inotifywait -qre close_write --format "$format" .; do
            kill_children
            "$@" &
        done
    }