I have a swarm of few services, and in the compose file there are few volumes created with the vieux/sshfs driver, which are used by the services.
The containers spawned by the services execute a single script, after which the container finishes/exits and a new one is created on its place - basically the services are spawning new containers all the time.
All works smooth, except that there is exceptionally large amount of zombie processes accumulated in the host machine. The zombies go away when the docker daemon is re stared - it must be docker who makes the zombies.
"ps aux | grep 'Z'" is
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 3040 0.0 0.0 0 0 ? Zs 14:13 0:00 [ssh] <defunct>
root 3042 0.0 0.0 0 0 ? Zs 14:13 0:00 [sshfs] <defunct>
root 3052 0.0 0.0 0 0 ? Zs 14:13 0:00 [ssh] <defunct>
root 3055 0.0 0.0 0 0 ? Zs 14:13 0:00 [sshfs] <defunct>
...
As far as I understand, the volumes are created only once, and the services are just using the local copy of the volume - not creating a new ssh connection and reading straight form the remote machine - and this should not be creating another ssh connection process that will become zombie.
I have trouble finding info on the topic, which makes me think that I am doing something fundamentally wrong. Please help.
I have just resolved the issue by enabling Tini for the services in the docker-compose file as follows -
init: true
Few zombies (<10) pop up, but then they get killed in a second - no accumulation.
I still don't get what the zombies had to do with the ssh. If anyone can answer that I would be grateful.
PS: I have checked few days after I have enabled Tini. There are some accumulated zombies (~300, before there ware ~2000). Problem seems mitigated, but it is still there.
Recent update: The setup I had was bad. It was a bad idea to shut down the containers after the process is done and start a new one. Instead whatever it is running on the container, runs and waits for the next "event". So do not do what I did.