Search code examples
gnu-parallel

How to pass variables to parallel command


How do I pass different values to a single parallel command:

I tried this:

parallel ::: 'X="aa"' 'X="bb"' ::: echo ${X}

which only prints two empty lines, as if the value of X is not set. I tried with echo $X with the same result.

I expected it to print

aa
bb

as if I had run sequentially:

echo "aa"
echo "bb"

Solution

  • It is not entirely clear what you want run.

    You clearly do not want:

    X="aa" echo ${X}
    X="bb" echo ${X}
    

    because that outputs nothing.

    Do you want:

    X="aa"; echo ${X}
    X="bb"; echo ${X}
    

    In that case you can do one of:

    parallel ::: 'X="aa";' 'X="bb";' ::: 'echo ${X}'
    parallel {1}\;{2} ::: 'X="aa"' 'X="bb"' ::: 'echo ${X}'
    parallel '{}; echo ${X}' ::: 'X="aa"' 'X="bb"'
    

    However, the GNU Parallel way to do it would be to use the GNU Parallel replacement strings instead of $X:

    parallel echo {} ::: aa bb
    

    So if you want to run:

    CXX="c++"; ID="c++"; cd ${ID}; ${CXX} main.cpp
    CXX="/local/bin/nvcc"; ID="nvcc"; cd ${ID}; ${CXX} main.cpp
    

    You would use :::+ :

    parallel 'cd {2}; {1} main.cpp' ::: c++ /local/bin/nvcc :::+ c++ nvcc
    

    Also: --dry-run is your friend, as it will show you what would be run, and that can often help understand the effect.

    If you are new to GNU Parallel and prefer reading a book, buy GNU Parallel 2018 at http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html or download it at: https://doi.org/10.5281/zenodo.1146014

    Read at least chapter 1+2. It should take you less than 20 minutes.