Search code examples
bashshellparallel-processingargumentsgnu-parallel

Parallel: How to reference multiple arguments from a function


I have this function and I need it to reference multiple arguments from a function using GNU parallel.

foo () {

    cd ${HOME}/sh/xxx/xxx/xxx/folder_with_scripts
    bash -H $1 #replace with echo in test run {echo $1 is being echoed}
    bash -H $2 #replace with echo in test run {echo $2 is being echoed}
}

export -f foo
parallel foo ::: *script.sh bash*.sh

folder_with_scripts content

$ ls
firstscript.sh
secondscript.sh
thirdscript.sh
bashhim.sh
bashscript.sh
bashher.sh

parallel foo basically executes all scripts following *script.sh inside by referencing it as an argument inside foo. Which is $1. What I'm trying to do is have it also execute bash*.sh* which are inside folders_with_scriptsdirectory by using $2.

According to man parallel, the syntax is:

 parallel [options] [command [arguments]] ( ::: arguments | :::: argfile(s) )

Since ::: arguments is plural, I'm assuming this is very possible.

result

For convenience sake, I'm replacing bash with echo

$ ./foo.sh
firstscript.sh is being echoed
secondscript.sh is being echoed
thirdscript.sh is being echoed
              is being echoed
              is being echoed
              is being echoed

wanted result

firstscript.sh is being echoed
secondscript.sh is being echoed
thirdscript.sh is being echoed
bashhim.sh is being echoed
bashscript.sh is being echoed
bashher.sh is being echoed

For my case, parallel foo ::: *.sh won't cut it.

Swapping the positions of $1 with *script.sh won't cut it


Solution

  • Assuming you want to run all bash*.sh for each *script.sh:

    parallel foo ::: *script.sh ::: bash*.sh
    

    If each bash*.sh is linked to a *script.sh (:::+ is introduced in version 20160422):

    parallel foo ::: *script.sh :::+ bash*.sh
    

    Test it with --dry-run:

    parallel --dry-run foo ::: *script.sh ::: bash*.sh
    parallel --dry-run foo ::: *script.sh :::+ bash*.sh
    

    If this does not answer your question, please update the question with what commands you want GNU Parallel to run, e.g.:

    foo firstscript.sh bashhim.sh
    foo secondscript.sh thirdscript.sh
    foo bashscript.sh bashher.sh
    

    and show the output of echo *script.sh bash*.sh in the dir where you run parallel.

    The data behind ::: is called an input source. Dealing with input sources is covered in chapter 4 of https://zenodo.org/record/1146014 (Printed: http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html)