When I ran the KCL as ./bin/kcl-bootstrap -p app.properties -j /usr/bin/java -e at the terminal, and then I did ps -ef | grep java and got the pid for it. I issued kill -s TERM $PID_ABOVE. I noticed the node recordProcessor managed to log like this
2018-03-31T11:13:46.998 INFO recordProcessor - Shutdown requested
2018-03-31T11:13:48.024 INFO recordProcessor - Shutting down...
When I ran the equivalent via a docker run command as such
docker run -v /tmp:/tmp -v ~/.aws:/root/.aws -ti cde3946e2cf9, issuing Ctrl-C or docker stop cde3946e2cf9 or docker kill --signal=SIGTERM cde3946e2cf9 to terminate the docker didn't result in the logs being produced above. e,g the recordProcessor didn't get notified about the shutdown request.
Our docker is deployed in Kubernetes cluster, and when we are redeploying the docker, I can see that Process terminanted, will initiate shutdown. is being logged by the main processor, but nothing from the worker shutting down request being reported.
I would like to trap these events when re-deploying docker to make sure we handle shutdown gracefully.
Anyone knows how to make sure the worker does get notified when redeploy docker in cluster?
Thanks
For anyone who is facing a similar issue, here are the steps I solved mine:
Add a preStop hook in kubernetes template to execute a shell command
The command is kill -s TERM $(ps -ef | grep "MultiLangDaemon" | grep -v \"grep\" | awk '{print $1}')