Speed-up external call in Java

Question

I have a program in JAVA that needs to call two other external executables (written in C++) thousands of times per launch. Actual time for calculations in each of them is effectively milliseconds, so the bottleneck of the whole program is calls and write/read to temporary file (external_exe_2 can read its input only from file). Also I need to check if there are no errors occurred.

I have a following benchmark test program to measure the performance of external calls:

long launchtime = (new Date()).getTime();

int n=1000;
String[][] sargs = new String[][]
    {{"external_exe_1", "arg1", "-o", "file.tmp"},
    {"external_exe_2", "arg2", "-i", "file.tmp"}};
String line, message;
Process process;
InputStream stderr;
BufferedReader reader;
StringBuilder bstring;
for (int i=0; i<n; i++)
    for (int j=0; j<2; j++) {
        process = Runtime.getRuntime().exec(sargs[j]);
        stderr = process.getErrorStream();
        reader = new BufferedReader (new InputStreamReader(stderr));
        bstring = new StringBuilder();
        while ((line=reader.readLine()) != null)
            bstring.append(line);
        message=bstring.toString();
        if (!message.isEmpty()) {
            System.err.println("Error: "+message);
            reader.close();
            return;
        }
    }

long exec_time = (new Date()).getTime() - launchtime;
System.out.printf("%d repeats done in %.3f sec\n", n, exec_time*0.001);

However, the speed is not satisfying at all. In fact, it's even slower than the Python analog. Is there something that I can improve to boost it?

Answer 1

If running on Linux/Mac, you're better off spawning a shell process (i.e., /bin/bash -i) and feed each command call into the shell to execute. This avoids recreating the streams for each iteration. For this to work, you'll need to execute some end-of-output marker for each execution (i.e. by sending

"external_exe_1 arg1 -o file.tmp ; echo 'EOI EOI'"

to the shell).