Search code examples
arraysbashparallel-processingassociative-arraygnu-parallel

Accessing Associative Arrays in GNU Parallel


Assume the following in Bash:

declare -A ar='([one]="1" [two]="2" )'
declare -a ari='([0]="one" [1]="two")'
for i in ${!ari[@]}; do 
  echo $i ${ari[i]} ${ar[${ari[i]}]}
done
0 one 1
1 two 2

Can the same be done with GNU Parallel, making sure to use the index of the associative array, not the sequence? Does the fact that arrays can't be exported make this difficult, if not impossible?


Solution

  • A lot has happened in 4 years. GNU Parallel 20190222 comes with env_parallel. This is a shell function that makes it possible to export the most of the environment to the commands run by GNU Parallel.

    It is supported in ash, bash, csh, dash, fish, ksh, mksh, pdksh, sh, tcsh, and zsh. The support varies from shell to shell (see details on https://www.gnu.org/software/parallel/env_parallel.html). For bash you would do:

    # Load the env_parallel function
    . `which env_parallel.bash`
    # Ignore variables currently defined
    env_parallel --session
    [... define your arrays, functions, aliases, and variables here ...]
    env_parallel my_command ::: values
    # The environment is also exported to remote systems if they use the same shell
    (echo value1; echo value2) | env_parallel -Sserver1,server2 my_command
    # Optional cleanup
    env_parallel --end-session
    

    So in your case something like this:

    env_parallel --session
    declare -A ar='([one]="1" [two]="2" )'
    declare -a ari='([0]="one" [1]="two")'
    foo() {
      for i in ${!ari[@]}; do 
        echo $i ${ari[i]} ${ar[${ari[i]}]}
      done;
    }
    env_parallel foo ::: dummy
    env_parallel --end-session
    

    As you might expect env_parallel is a bit slower than pure parallel.