java block while run shell command using process

Question

I have a Java project which help to find all videos information in youtube playlist using youtube-dl. Here is the Main.java

import java.io.*;


public class Main {

public static void main(String[] args) throws Exception {
    String command1 = "/usr/local/bin/youtube-dl --flat-playlist --dump-single-json https://www.youtube.com/playlist?list=PLFcOH1YaRqUo1yEjY5ly09RFbIpUePF7G";
    String command2 = "/usr/local/bin/youtube-dl --flat-playlist --dump-single-json https://www.youtube.com/playlist?list=PLC6A0625DCA9AAE2D";
    System.out.println(executeCommand(command1));

}

private static String executeCommand(String command) throws IOException, InterruptedException {
    int exitCode = 0;
    String result = "";
    Process process;
    ProcessBuilder builder = new ProcessBuilder(command.replaceAll("[ ]+", " ").split(" "));
    builder.directory(new File("/tmp/test"));
    process = builder.start();
    exitCode = process.waitFor();
    return getStringFromInputStream(process.getInputStream());
}
private static String getStringFromInputStream(InputStream is) {

    BufferedReader br = null;
    StringBuilder sb = new StringBuilder();

    String line;
    try {

        br = new BufferedReader(new InputStreamReader(is));
        while ((line = br.readLine()) != null) {
            sb.append(line);
        }

    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        if (br != null) {
            try {
                br.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
    return sb.toString();
}
}

The command1 and command2 are identical except for the Youbute playlist parameter. The playlist in command2 has 409 videos and has 200 videos in command1. I can get result successfully for both command in terminal. Just the command2 takes more time, but only several seconds. But when I run the Main.java (javac Main.java; java Main), for command1 it prints result successfully, but for command2 it hangs there for several minutes without any result. Here is the jstack for this process

"main" #1 prio=5 os_prio=0 tid=0x00007f828c009800 nid=0xce7 in Object.wait() [0x00007f8293cf7000]
java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x0000000771df9fe0> (a java.lang.UNIXProcess)
    at java.lang.Object.wait(Object.java:502)
    at java.lang.UNIXProcess.waitFor(UNIXProcess.java:396)
    - locked <0x0000000771df9fe0> (a java.lang.UNIXProcess)
    at Main.executeCommand(Main.java:18)
    at Main.main(Main.java:8)

It hangs at exitCode = process.waitFor();. I have no idea about it. Can anyone help me? Many thanks.

Answer 1

As mentioned in the comments, by default the output of the subprocess is sent to a pipe which can be read using Process.getInputStream(). If the subprocess generates lots of output and the Java program doesn't consume it, the pipe's buffer will fill up and the subprocess will block on writing.

The easiest solution is to call .inheritIO() on the ProcessBuilder. That will send the output to the console instead of buffering it in memory (same for input and error streams).