Search code examples
dockerdockerfilegitlab-ci-runneralpine-linuxsigkill

Catching SIGTERM from alpine image


I was trying to catch SIGTERM signal from a docker instance (basically when docker stop is called) but couldn't find a way since I have different results for each try I performed.

Following is the setup I have

Dockerfile

FROM gitlab/gitlab-runner:alpine

COPY ./start.sh /start.sh

ENTRYPOINT ["/start.sh"]

start.sh

#!/bin/bash

deregister_runner() {
    echo "even if nothing happened, something happened"
    exit
}
trap deregister_runner SIGTERM

while true; do
    sleep 10
done

Now I build the docker image

$ docker build -t dockertrapcatch .
Sending build context to Docker daemon  51.71kB
Step 1/3 : FROM gitlab/gitlab-runner:alpine
 ---> 9f8c39873bee
Step 2/3 : COPY ./start.sh /start.sh
 ---> Using cache
 ---> ebb3cac0c509
Step 3/3 : ENTRYPOINT ["/start.sh"]
 ---> Using cache
 ---> 7ab67fe5a714
Successfully built 7ab67fe5a714
Successfully tagged dockertrapcatch:latest

Run the docker

$ docker run -it dockertrapcatch

Now when I run docker stop <<container_id_here>> or docker kill --signal=SIGTERM <<container_id_here>>, my deregister_runner function is not called.

After that I changed the start.sh script as following (SIGKILL ==> EXIT)

#!/bin/bash

deregister_runner() {
    echo "even if nothing happened, something happened"
    exit
}
trap deregister_runner EXIT

while true; do
    sleep 10
done

After this change and creating the docker image and running it docker stop <<container_id_here>> still does not work but docker kill --signal=SIGTERM <<container_id_here>> works!

$ docker run -it dockertrapcatch
even if nothing happened, something happened
$ docker kill --signal=SIGTERM 6b667af4ac6c
6b667af4ac6c

I read that actually docker stop sends a SIGTERM but I think this time it is not working? Any idea?


Solution

  • I can reproduce the issue you raise, while it does not show up when I replace the base image with debian:10, for example.

    It happens the issue is not due to alpine but to the gitlab/gitlab-runner:alpine image itself, namely this Dockerfile contains the following line:

    STOPSIGNAL SIGQUIT

    To be more precise, the line above means docker stop will send a SIGQUIT signal to the running containers (and wait for a "graceful termination time" before killing the containers, as if a docker kill were issued in the end).

    If this Dockerfile directive is not used, the default signal sent by docker stop is SIGTERM.

    Beware that SIGKILL would be a very poor choice for STOPSIGNAL, given that the KILL signal cannot be trapped.

    So, your first example should work if you use the following line:

    trap deregister_runner SIGINT SIGQUIT SIGTERM
    

    This way, your cleanup function deregister_runner will be triggered anytime you issue a docker stop, or use the Ctrl-C keybinding (thanks to SIGINT).

    Finally, two additional notes related to this question of Docker, bash and signals:

    • The "graceful termination time" (between stop and kill) can be customized, and there are some pitfalls when using a Bash entrypoint (regarding the "signal propagation"). I explained both issues in more detail in this SO answer: Speed up docker-compose shutdown.

    • Beware that in many alpine images, bash is not pre-installed, e.g.:

      $ sudo docker run --rm -it alpine /bin/bash
        /usr/bin/docker: Error response from daemon: OCI runtime create failed:
        container_linux.go:346: starting container process caused
        "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown.
      

      (fortunately, this was not the case of gitlab/gitlab-runner:alpine, which indeed contains the bash package :)