Search code examples
batch-fileconcurrencywait

Start concurrent batches with filename mask and wait for them to finish

I'm trying to start a fixed number of concurrent batch processes that have similar filenames, all in the same directory:

TestMe1.bat
TestMe2.bat
TestMe3.bat

All batches should start at the same time, and all should complete before the batch continues, e.g. a master.bat:

echo Starting batches

(
start "task1" cmd /C "TestMe1.bat"
start "task2" cmd /C "TestMe2.bat"
start "task3" cmd /C "TestMe3.bat"
) | pause

echo All batches have stopped and we can safely continue

I'm trying to find a way to start all batches in the directory that match TestMe*.bat, so that I don't have to craft a new master.bat file each time. Something like this, but, you know, working:

echo Starting batches

(
for /f "delims=" %%x in ('dir /b /a-d TestMe*.bat') do start "task" cmd /C "%%x"
) | pause

echo All batches have stopped and we can safely continue

Thanks to this and this for getting me this far.

Advice and ideas gratefully received!

Solution

  • First you have to understand how this special method of using pipe with the pause command works and why it can be used for waiting for multiple parallel processes.

    Taking your first working code sample as the starting point

    (
start "task1" cmd /C "TestMe1.bat"
start "task2" cmd /C "TestMe2.bat"
start "task3" cmd /C "TestMe3.bat"
) | pause

    It works because each new instance of CMD which is invoked by the start command will be started in a new console with its stdout and stderr redirected to to that new console, so the outputs of the child CMDs will not be redirected to the pipe and so will not be consumed by the pause command which is waiting for input from the pipe.

    BUT,

    Still each of the processes have inherited the pipe handle, so the handle to the pipe will remain open as long as the child processes are alive.

    As a consequence the pause command in the right side of the pipe will remain active, waiting for input from the left side of the pipe until it receives input from the pipe(which will never happen) or all child processes have terminated and closed the handle to the pipe.

    So the main CMD instance which is executing your master batch file, is actually waiting for the pause command (right side of the pipe) to terminate which in turn is waiting for child processes (potential pipe writers) to terminate.

    Now it becomes clear why your second attempted code involving the FOR loop is not working.

    (
for /f "delims=" %%x in ('dir /b /a-d TestMe*.bat') do start "task" cmd /C "%%x"
) | pause

    The FOR command inside the pipe is executed by the child CMD in command line mode, In this mode the command echoing in on by default, so every command inside the FOR body will be echoed to standard output(the pipe) before execution, which in turn feeds the pause command and terminate its process before even the first child process is created, Therefor the batch file execution continues without ever waiting for the child processes to finish.

    This can be easily resolved by putting the @ after do will turn the command echoing off inside the FOR body.

    (
for /f "delims=" %%x in ('dir /b /a-d TestMe*.bat') do @start "task" cmd /C "%%x"
) | pause>nul

    You may also want to hide the output of pause command by redirecting it to nul