How to execute a bunch of commands from a file with concurrency control?

Question

I have a file with following items:

echo 1 ; sleep 2
echo 2 ; sleep 2
echo 3 ; sleep 2
echo 4 ; sleep 2
echo 5 ; sleep 2
echo 6 ; sleep 2
echo 7 ; sleep 2

What is the best way of iterating through it executing each line with concurrency control (max 2 items at a time). I was thinking about using xargs but no luck so far with it.

Answer 1

$ cat file.txt
echo 1 ; sleep 1
echo 2 ; sleep 1
echo 3 ; sleep 1
echo 4 ; sleep 1
...
echo 13 ; sleep 1
echo 14 ; sleep 1

Running it with xargs parallelization:

xargs -a file.txt -I CMD --max-procs=2 sh -c CMD

This seems to produce some output, but is mostly sequential although once in a while, you get independent output:

1
3
2
4
...
13
14

• -I option replaces each command read with a replacement, in this case CMD
• sh -c CMD basically says what to with each command.
• -c tells sh that the command will come from a command string

Perhaps running it with commands that require a bit more processing will show a hint of parallelization