Parallel nested for loop in bash

Question

I am trying to run a c executable through bash. The executable will take a different argument in each iteration, and I want to do it in parallel since I have 12 cores available.

I tried

w=1;
for i in {1..100}
do
l=$(($i-1));
for j in {12*l..12*i}
do
./run $w/100 > "$w"_out &
done
expr=$w % 12;
if ["$expr" -eq "0"]
then wait;
fi;
done

run is the c executable. I want to run it with increasing argument w in each step, and I want to wait until all processes are done if 12 of the cores are in use. SO basically, I will run 12 executables at the same time, then wait until they are completed, and then move to the next 12.

Hope I made my point clear.

Cheers.

Answer 1

Use gnu parallel instead:

parallel ./myscript {1} ::: {1..100}

You can specify the number of parallel processes with the -P option, but it defaults to the number of cores in the system.

You can also specify -k to keep the output order and redirect the file.

To redirect the output to individual files, you can specify the output redirection, but you have to quote it, so that it is not parsed by the shell. For example:

parallel ./run {1} '>' {1}_out ::: {1..10}

is equivalent to running ./run 1 > 1_out to ./run 10 > 10_out