I'm trying to execute a shell script with systemd.
If I run my script from the bash everything works fine. But if I run the same script via systemd it never finishes. The command where it seems to hang is:
random="$(LC_ALL=C tr -cd '[:alnum:]' < /dev/urandom | fold -w128 | head -n1)"
If I'm replacing this line with random="1234"
it also runs with systemd. I guess the 'hanging' command is the tr
- its process never finishes.
And this is the systemd unit file I'm using:
[Unit]
Description=my script
[Service]
Type=forking
Restart=on-failure
ExecStart=/mypath/script.sh start
ExecStop=/bin/kill $MAINPID
ExecStopPost=/bin/rm -f /mypath/RUNNING_PID
[Install]
WantedBy=multi-user.target
Edit: Made the explanation clearer and added new information.
Short answer: systemd
is filtering SIGPIPE
s, so
set IgnoreSIGPIPE=false
under [Service]
in the .service
file. From the systemd.exec
manual:
IgnoreSIGPIPE= Takes a boolean argument. If true, causes SIGPIPE to be ignored in the executed process. Defaults to true because SIGPIPE generally is useful only in shell pipelines.
Long explanation:
random="$(LC_ALL=C tr -cd '[:alnum:]' < /dev/urandom | fold -w128 | head -n1)"
When the head
command exits after the first newline received from fold
, its open file descriptors are closed.
Thus when the fold
command later tries to write to the pipe, it will receive a SIGPIPE
signal (as described in pipe(7)
).
The default action for this signal is a termination of the process.
That mechanism repeats towards the start of the pipeline, so the fold
command would be terminated next, and eventually the tr
command.
However, when the pipeline is run under systemd
, systemd
sets the default action for SIGPIPE
to SIG_IGN
, which makes the processes in the pipeline ignore the signal.
When fold
(and the other components of the pipeline) inherits that signal handler, it won't be notified (killed) via SIGPIPE
when writing to the now closed output pipe (Instead each write to the pipe will return an EPIPE
error).
As fold
command does not check the return value of those write
calls (at least not in coreutils-8.26
), this makes fold
continue reading from the pipe (stdin) and to write to the closed pipe (stdout), even though an error is reported each time.
So fold
keeps its input pipe from tr
open, and tr
will happily write to the output pipe feeding fold
forever.