Search code examples
seekgnu-parallel

Is it possible to use process substitution with GNU parallel?

I am trying to do the following:

   $echo "hi" | parallel --pipe head
hi

but using the different -a flag and --pipepart because it's supposedly faster according to the manual.

With the following, it doesn't work:

   $parallel --pipepart --block 1M -a <(echo "hi") head
Died at -e line 1.

With another attempt, it doesn't work:

   $parallel --pipepart --block 1M -a "<(echo \"hi\")" head
parallel: Error: Cannot open input file `<(echo "hi")': No such file or directory.
  • Does this have to do with "The file must be a physical (seekable) file "(from the manual page)?
  • If so, stream is not "seekable"?
  • What does "seekable" mean?
Solution

  • Your 3 bullets are spot on.

    A stream has to be read from the first byte to the last byte. A seekable file you can start reading from the middle: you can fseek to the middle and read. You can even have multiple processes read from different places in the file at the same time. It is this last property that GNU Parallel uses.

    So no: <(process) is not a seekable file.