Read stdin line by line

Question

I need to pipe a command into a batch file and do some processing, while preserving the output of the original command. So that for example on running the following command, the output would still be as if there was no piping at all:

ping 127.0.0.1 -n 4 | my_process

The best workaround I have found so far is https://stackoverflow.com/a/6980605/6094503. But my problem is that I need the output line by line. Using that solution the output is flushed only after the ping command is done executing. I found https://stackoverflow.com/a/21567535/6094503 which says it is because of the in (inside the for in loop).

This is a line by line example:

ping 127.0.0.1 -n 4 | findstr $

Actually if Windows was an open source project we could probably find the answer inside findstr or similar commands.

Answer 1

You can flush the pipe line by line with more into a file and with a second cmd.exe instance read from that file.

@echo off
REM *** This is a trampoline to jump to a function when a child process shall be invoked
for /F "tokens=3 delims=:" %%L in ("%~0") do goto %%L

setlocal DisableDelayedExpansion

break > pipe.tmp
REM *** Create a new cmd.exe process, and calling :async in this batch file, uses the trampoline
start "" /b "%~d0\:async:\..%~pnx0"
( 
    more
    echo END
) >> pipe.tmp

REM Wait for a clean exit of the async thread
ping localhost -n 2 > nul
echo END
exit /b

:async
echo async

set lineCnt=0
< pipe.tmp (
    for /L %%n in ( infinite ) do (
    set "line="
    set /p line=
    if defined line (
        set /a lineCnt+=1
        setlocal EnableDelayedExpansion
        if "!line:~0,3!" == "END" (
            exit
        )
        echo( READ[!lineCnt!]: !line!
        endlocal
    )
    )
)

findstr fails to read from a pipe and store the output into a file asynchronously. But it works when reading from a file, but then you need two asynchronous processes.

@echo off
REM *** This is a trampoline to jump to a function when a child process shall be invoked
for /F "tokens=3 delims=:" %%L in ("%~0") do goto %%L

setlocal DisableDelayedExpansion

break > pipe1.tmp
break > pipe2.tmp
REM *** piperun.tmp is used as a signal for :async1 to detect when to stop the infinite loop
break > piperun.tmp

REM *** Create a new cmd.exe process, and calling :async1 in this batch file, uses the trampoline
start "" /b "%~d0\:async1:\..%~pnx0"
start "" /b "%~d0\:async2:\..%~pnx0"

more >> pipe1.tmp
del piperun.tmp

REM Wait for a clean exit of the async thread
ping localhost -n 2 > nul
del pipe1.tmp
del pipe2.tmp

echo END
exit /b


:async1
< pipe1.tmp > pipe2.tmp (
    for /L %%n in ( infinite ) do (
        findstr /n "^"
        if not exist piperun.tmp (
            REM *** The "raw" END is the signal for :async2 to stop the infinite loop
            echo END
            @REM echo EXIT %0 > CON
            exit
        )
    )
)
exit /b

:async2
set lineCnt=0
< pipe2.tmp (
    for /L %%n in ( infinite ) do (
    set "line="
    set /p line=
    if defined line (
        set /a lineCnt+=1
        setlocal EnableDelayedExpansion
        if "!line:~0,3!" == "END" (
            @REM echo EXIT %0 > CON
            exit
        )
        @REM set "line=!line:*:=!"
        echo( READ[!lineCnt!]: !line!
        endlocal
    )
    )
)
exit /b