Can GNU Parallel also group output of other file descriptors other than stdout and stderr?

Question

I have a simple bash script curlit.sh (simplified):

# stdout and stderr are used for other output
echo "hello"
>&2 echo "world"
# curl outputs result and headers respectively to fd 3 and 4
curl --output > >(cat >&3) --dump-header > >(cat >&4) "$1"

Now if I parallelize my script with millions of domain names:

cat MillionDomains.out |
  parallel -j0 ./curlit.sh {} > stdout.out 2> stderr.out 3> html.out 4> header.out

parallel groups stdout.out and stderr.out but the output for html.out and header.out is interleaved and basically useless.

Can I configure parallel to also group output from fd 3 and 4, \ie buffer it and print it per process?

Answer 1

Currently GNU Parallel cannot do that.

I tried autodetecting which file descriptor were open, and buffer those.

Detection is pretty much just:

for $fd (1..255) {
    open($fd{$fd}, ">&=$fd") || delete $fd{$fd};
}

But I use open3 to run the jobs and do the redirection. If you can show how to make an openN (as opposed to open3), then I am open to building that.

Perl: open3 but for n file descriptors