How do I execute multiple commands in parallel on an array of parameters with bash, and fail if at least one of them failed

Question

I have a bash script with a function that needs to run in parallel with different arguments. I need to know if at least one of the executions failed (returned non-zero) - doesn't matter how many failed.

The command accepts an array of parameters for the execution. I need to limit the concurrency to 4 concurrent runs due to high load. I also need to print the logs in the parent process (the one that runs the bash script)

this is the function I'm running:

function run_and_retry {
  EXIT_STATUS=0
  $COMMAND || EXIT_STATUS=$?

  if [ $EXIT_STATUS -ne 0 ]; then
    EXIT_STATUS=0
    $COMMAND || EXIT_STATUS=$?

  fi

  return $EXIT_STATUS
}

I've tried using GNU parallel and xargs and encountered issues with both.

With xargs: (couldn't get the exit status out of it, and it also didn't work when I ran it in TravisCI)

PARAMETERS=(first-parameter second-parameter third-parameter)
export -f run_and_retry
echo "${PARAMETERS[@]}" | xargs -P 4 -n 1 -I {} bash -c "run_and_retry {}"

With GNU parallel:

PARAMETERS=(first-parameter second-parameter third-parameter)
export -f run_and_retry
parallel -j 4 -k --lb 2 run_and_retry {} ::: echo "${PARAMETERS[@]}" 

Answer 1

You are so close to getting the syntax of GNU Parallel correct:

COMMAND=echo
PARAMETERS=(first-parameter second-parameter third-parameter)
parallel -j 4 -k --retries 2 "$COMMAND" {} ::: "${PARAMETERS[@]}" ||
  echo $? commands failed. More than 99 if $? = 100

Or if you really insist on doing the retrying yourself:

PARAMETERS=(first-parameter second-parameter third-parameter)
export -f run_and_retry
parallel -j 4 -k run_and_retry {} ::: "${PARAMETERS[@]}" ||
  echo One or more commands failed