Search code examples
bashpipezshpipelinexclip

Output redirection hangs if process has spawned a background fork of xclip

I've got a script which calls to xclip inside. Something like this:

#!/bin/bash
RESULT="some data"
echo $RESULT | xclip
echo $RESULT

xclip puts puts the data from its stdin to the clipboard, spawns a background thread to preserve that buffer (that's how clipboard works in X11) and detaches it from tty.

If I run a script directly, it works as expected:

$ ./script.sh
some data
$

However, if I'll try to pass it's output into a pipeline, it hangs until xclip background process ends (basically, until someone else puts data into the clipboard).

$ ./script.sh | ./another_script.sh # it hangs

I found that xclip has a -f flag which seem to fix that problem.

From the man page:

when xclip is invoked in the in mode with output level set to silent (the defaults), the filter option will cause xclip to print the text piped to standard in back to standard out unmodified

However, I'm trying to understand why does it work that way.

I've created another example which behaves the same way:

echo $(sleep 5 &)

Also, I've been told that in fish my example works without -f flag.

So, the question is:
1. Is this a bug of a shell (bash and zsh) or an expected behaviour?
2. How does -f flag affects that behaviour? From the man page, it does not seems very related.
3. How to make other scripts (e.g. sleep) work the way xclip works with -f flag?

Solution

  • By default,

    1. all open files are passed from a parent process to child process. AND
    2. When a pipe is created (A->B), process B will not get the EOF indicator on stdin, until ALL processes from step A close their STDOUT.

    Combining the above, when you execute /script.sh | ./another_script.sh, the backgrounded xclip will inherit the 'script.sh' stdout (which is the input to ./another_script.sh), at this time, there are two processes (script.sh, xclip) which are connected to the pipe. Only when both will close STDOUT (or will terminate), the ./another_script.sh will see the EOF on the pipe, and will be able to exit (assuming it's waiting for EOF on input).

    The solution for those cases is for the back-grounded process to close stdout, therefore allowing pipelines to complete. Usually, the stdout will be redirected to stderr, if output is needed. Most likely, the '-f' closes the STDOUT.

    Per man xclip, possible to write echo $RESULT | xclip -f (no need for extra echo) to avoid the problem, and allow script to run as part of a pipeline

    man fork and man pipe provides the details. In particular

    man 2 fork:

    • The child inherits copies of the parent's set of open file descriptors. Each file descriptor in the child refers to the same open file description (see open(2)) as the corresponding file descriptor in the parent. This means that the two file descrip‐ tors share open file status flags, file offset, and signal-driven I/O attributes (see the description of F_SETOWN and F_SETSIG in fcntl(2)).

    man 7 pipe:

    If all file descriptors referring to the write end of a pipe have been closed, then an attempt to read(2) from the pipe will see end-of-file (read(2) will return 0).