Search code examples
bashcurlgnu-parallel

GNU Parallel: Run bash code that reads (seq number) from pipe?


I would like parallel to read the (seq numbers) pipe, so I would like running something like that:

seq 2000 | parallel --max-args 0 --jobs 10 "{ read test; echo $test; }"

Would be equivalent to running:

echo 1
echo 2
echo 3
echo 4
...
echo 2000

But unfortunately, the pipe was not read by parallel, meaning that it was instead ran like:

echo
echo
echo
...
echo

And the output is empty.

Does anyone know how to make parallel read (seq numbers) pipe? Thanks.


Solution

  • An alternative with GNU xargs that does not require GNU parallel:

    seq 2000 | xargs -P 10 -I {} "echo" "hello world {}"
    

    Output:

    hello world 1
    hello world 2
    hello world 3
    hello world 4
    hello world 5
    .
    .
    .
    

    From man xargs:

    -P max-procs: Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xargs will run as many processes as possible at a time.

    -I replace-str: Replace occurrences of replace-str in the initial-arguments with names read from standard input.