Search code examples
gzipgnu-paralleltee

How to gzip tee output while running in gnu parallel?


Suppose I am running tee inside a command executed by parallel.

I would like to gzip the output from tee:

... | tee --gzip the_file | and_continue

Solution

  • bash process substitution is useful for cases like this. Something like:

    ... | tee >(gzip -c the_file) | and_continue
    

    If you're choosing different files in a parallel run and need to format the name differently each time, take a look at GNU Parallel argument placeholder in bash process substitution for how that has to change (to defer the process substitution to act per parallel job).