Java - Redirecting Process I/O (piping) Is Stalling

Question

I am trying to redirect the output of a process to the input of another. I.e., piping. When doing this in the Windows DOS shell it looks like

C:\> dir /s/b . | findstr dat$

However I am trying to a command like this in Java and so far it looks like:

    Stopwatch sw = Stopwatch.createStarted();
    ProcessBuilder source = new ProcessBuilder("cmd", "/S/D/c", "dir", "/s/b", ".");
    ProcessBuilder target = new ProcessBuilder("cmd", "/S/D/c", "findstr", "dat$");
    source.directory(new File("C:/"));
    target.directory(source.directory());
    // I am messing with the below lines, nothing is working
    source.redirectInput(target.redirectInput());
    source.redirectOutput(ProcessBuilder.Redirect.PIPE);
    source.redirectOutput(target.redirectInput());
    source.redirectInput(target.redirectOutput());
    target.redirectOutput(source.redirectInput());

    Process pSource = source.start();
    Process pTarget = target.start();
    log.debug("Running {} | {}", source.command(), target.command());
    try (BufferedReader br = new BufferedReader(new InputStreamReader(pTarget.getInputStream()))) {

        String line;
        while ((line = br.readLine()) != null)
            log.debug("{}", line);
    } finally {
        log.debug("Ending process {} with exist code {} in time {}", target.command(),
                pTarget.destroyForcibly().exitValue(), sw);
    }

But I am finding the code stalls on the readLine, so something isn't working here. How do I properly use IO redirects?

Answer 1

The objects accepted or returned by redirectInput() resp, redirectOutput() describe a certain policy; they do not represent actual channels.
So by the statement source.redirectInput(target.redirectInput()) you are just specifying that both processes should have the same policy, you’re not linking channels.

In fact, directly linking the channels of two processes is impossible in Java 8. The best you can do to achieve a similar effect, is to start a background thread which will read the first process’ output and write it to the second process’ input:

static List<Process> doPipeJava8() throws IOException {
    Process pSource = new ProcessBuilder("cmd", "/S/D/c", "dir", "/s/b", ".")
                    .redirectInput(ProcessBuilder.Redirect.INHERIT)
                    .redirectError(ProcessBuilder.Redirect.INHERIT)
                    .start();
    Process pTarget;
    try {
        pTarget     = new ProcessBuilder("cmd", "/S/D/c", "findstr", "dat$")
                    .redirectErrorStream(true)
                    .redirectOutput(ProcessBuilder.Redirect.INHERIT)
                    .start();
    } catch(Throwable t) {
        pSource.destroyForcibly();
        throw t;
    }
    new Thread(() -> {
        try(InputStream srcOut = pSource.getInputStream();
            OutputStream dstIn = pTarget.getOutputStream()) {
            byte[] buffer = new byte[1024];
            while(pSource.isAlive() && pTarget.isAlive()) {
                int r = srcOut.read(buffer);
                if(r > 0) dstIn.write(buffer, 0, r);
            }
        } catch(IOException ex) {}
    }).start();
    return Arrays.asList(pSource, pTarget);
}

This configures the error channels, the input channel of the first process, and the output channel of the last process to INHERIT so they will use our initiating process’ console. The first process’ output and the second process’ input are kept at the default PIPE which means establishing a pipe to our initiating process, so it’s our duty to copy the data from one pipe to another.

The method can be used as

List<Process> sub = doPipeJava8();
Process pSource = sub.get(0), pTarget = sub.get(1);
pSource.waitFor();
pTarget.waitFor();

If we remove the .redirectOutput(ProcessBuilder.Redirect.INHERIT) from the builder of the pTarget process, we could read the final output:

List<Process> sub = doPipeJava8();
Process pSource = sub.get(0), pTarget = sub.get(1);
List<String> result = new BufferedReader(new InputStreamReader(pTarget.getInputStream()))
    .lines().collect(Collectors.toList());

---

Java 9 is the first Java version with support for establishing a pipeline between sub-processes. It simplifies the solution to

static List<Process> doPipeJava9() throws IOException {
    return ProcessBuilder.startPipeline(
        List.of(new ProcessBuilder("cmd", "/S/D/c", "dir", "/s/b", ".")
                    .redirectInput(ProcessBuilder.Redirect.INHERIT)
                    .redirectError(ProcessBuilder.Redirect.INHERIT),
                new ProcessBuilder("cmd", "/S/D/c", "findstr", "dat$")
                    .redirectErrorStream(true)
                    .redirectOutput(ProcessBuilder.Redirect.INHERIT)) );
}

It does the same as the other solution¹; the example above is configured to let the first process read from the console (if needed) and the last process write to the console. Again, if we omit the .redirectOutput(ProcessBuilder.Redirect.INHERIT) from the last process builder, we can read the last process’ output.

¹ except that it will use the system’s native piping capability when possible