process.waitFor() never returns

Question

Process process = Runtime.getRuntime().exec("tasklist");
BufferedReader reader = 
 new BufferedReader(new InputStreamReader(process.getInputStream()));
process.waitFor();

• Ikaso: Please note that on JAVA 8 there is a waitFor overload that let's you specify the specify a timeout. This might be a better choice to refrain from a case where the waitFor never returns.
• SRJ: In my case i added waitFor() before reading output stream and that caused deadlock situation.if(!process.waitFor(15, TimeUnit.MINUTES)) { process.destroy(); } else { process.getOutputStream().close(); BufferedReader stdout = new BufferedReader(new InputStreamReader(process.getInputStream()));

Answer 1

There are many reasons that waitFor() doesn't return.

But it usually boils down to the fact that the executed command doesn't quit.

This, again, can have many reasons.

One common reason is that the process produces some output and you don't read from the appropriate streams. This means that the process is blocked as soon as the buffer is full and waits for your process to continue reading. Your process in turn waits for the other process to finish (which it won't because it waits for your process, ...). This is a classical deadlock situation.

You need to continually read from the processes input stream to ensure that it doesn't block.

There's a nice article that explains all the pitfalls of Runtime.exec() and shows ways around them called "When Runtime.exec() won't" (yes, the article is from 2000, but the content still applies!)

• ForguesR: This answer is correct but it misses a code sample to troubleshoot the problem. Have a look at Peter Lawrey answer for useful code to find out why waitFor() doesn't return.
• d-b: If the process isn't supposed to quit? If you run ls it will quit but what if you start httpd?
• Joachim Sauer: @d-b: I'm not sure what your question is. Yes, that's another reason why waitFor()doesn't return.
• d-b: The question is: how do you handle that situation? You want httpd or whatever to stay open.
• Joachim Sauer: @d-b: in that case simply don't call waitFor().

Answer 2

It appears you are not reading the output before waiting for it to finish. This is fine only if the output doesn't fill the buffer. If it does, it will wait until you read the output, catch-22.

Perhaps you have some errors which you are not reading. This would cause the application to stop and waitFor to wait forever. A simple way around this is to re-direct the errors to the regular output.

ProcessBuilder pb = new ProcessBuilder("tasklist");
pb.redirectErrorStream(true);
Process process = pb.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
while ((line = reader.readLine()) != null)
 System.out.println("tasklist: " + line);
process.waitFor();

• Jean-François Savard: For info : ProcessBuilder being a real builder, you can directly write ProcessBuilder pb = new ProcessBuilder("tasklist").redirectErrorStream(true)
• Toochka: I'd rather use pb.redirectError(new File("/dev/null"));
• ZhekaKozlov: @Toochka Just for information, redirectError is only available since Java 1.7
• Gerben Rampaart: Should be the accepted answer I believe, I replaced my code with this and it immediately worked.
• Sybuser: why do you need to use waitFor since you're already reading err/out until you receive EOF ?

Answer 3

Also from Java doc:

java.lang

Class Process

Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.

Fail to clear the buffer of input stream (which pipes to the output stream of subprocess) from Process may lead to a subprocess blocking.

Try this:

Process process = Runtime.getRuntime().exec("tasklist");
BufferedReader reader =
new BufferedReader(new InputStreamReader(process.getInputStream()));
while ((reader.readLine()) != null) {}
process.waitFor();

• kevinarpe: Two caveats: (1) Use ProcessBuilder + redirectErrorStream(true), then you are safe. Else, (2) you need one thread to read from Process.getInputStream() and another to read from Process.getErrorStream(). Just spent about four hours figuring this out(!), aka, "The Hard Way".
• Matthew Wise: You could use Apache Commons Exec functionality to consume the stdout and stderr streams simultaneously: DefaultExecutor executor = new DefaultExecutor(); PumpStreamHandler pumpStreamHandler = new PumpStreamHandler(stdoutOS, stderrOS); executor.setStreamHandler(pumpStreamHandler); executor.execute(cmdLine);, where stoutOS and stderrOS are BufferedOutputStreams I've created to write out to appropriate files.
• shaILU: In my case I was calling a batch file from Spring that internally opens one editor. My code was hanging even after applying code of process.close(). But when I open input stream as suggested above and close immediately -- problem goes away. So in my case Spring was waiting for stream close signal. Even though I am using Java 8 auto closable.
• Gili: @kevinarpe You can use InputStream.available() to consume both streams from a single thread without blocking. So long as the method returns a positive number you are guaranteed a read will not block.

Answer 4

I would like to add something to the previous answers but since I don't have the rep to comment, I will just add an answer. This is directed towards android users which are programming in Java.

Per the post from RollingBoy, this code almost worked for me:

Process process = Runtime.getRuntime().exec("tasklist");
BufferedReader reader =
new BufferedReader(new InputStreamReader(process.getInputStream()));
while ((reader.readLine()) != null) {}
process.waitFor();

In my case, the waitFor() was not releasing because I was executing a statement with no return ("ip adddr flush eth0"). An easy way to fix this is to simply ensure you always return something in your statement. For me, that meant executing the following: "ip adddr flush eth0 && echo done". You can read the buffer all day, but if there is nothing ever returned, your thread will never release its wait.

Hope that helps someone!

• anon: When you don't have the rep to comment, don't work around it and comment anyway. Make this an answer on its own and get rep from it!
• osundblad: I don't think it is the process.waitFor() that hangs it is the reader.readLine() that hangs if you have no output. I tried to use the waitFor(long,TimeUnit) to timeout if something goes wrong and discovered that it was the read the hung. Which makes the timeouted version require another thread to do the reading...

Answer 5

There are several possibilities:

1. You haven't consumed all the output on the process's stdout.
2. You haven't consumed all the output on the process's stderr.
3. The process is waiting for input from you and you haven't provided it, or you haven't closed the process's stdin.
4. The process is spinning in a hard loop.

Answer 6

As others have mentioned you have to consume stderr and stdout.

Compared to the other answers, since Java 1.7 it is even more easy. You do not have to create threads yourself anymore to read stderr and stdout.

Just use the ProcessBuilder and use the methods redirectOutput in combination with either redirectError or redirectErrorStream.

String directory = "/working/dir";
File out = new File(...); // File to write stdout to
File err = new File(...); // File to write stderr to
ProcessBuilder builder = new ProcessBuilder();
builder.directory(new File(directory));
builder.command(command);
builder.redirectOutput(out); // Redirect stdout to file
if(out == err) { 
 builder.redirectErrorStream(true); // Combine stderr into stdout
} else { 
 builder.redirectError(err); // Redirect stderr to file
}
Process process = builder.start();

Answer 7

For the same reason you can also use inheritIO() to map Java console with external app console like:

ProcessBuilder pb = new ProcessBuilder(appPath, arguments);
pb.directory(new File(appFile.getParent()));
pb.inheritIO();
Process process = pb.start();
int success = process.waitFor();

Answer 8

You should try consume output and error in the same while

 private void runCMD(String CMD) throws IOException, InterruptedException {
 System.out.println("Standard output: " + CMD);
 Process process = Runtime.getRuntime().exec(CMD);
 // Get input streams
 BufferedReader stdInput = new BufferedReader(new InputStreamReader(process.getInputStream()));
 BufferedReader stdError = new BufferedReader(new InputStreamReader(process.getErrorStream()));
 String line = "";
 String newLineCharacter = System.getProperty("line.separator");
 boolean isOutReady = false;
 boolean isErrorReady = false;
 boolean isProcessAlive = false;
 boolean isErrorOut = true;
 boolean isErrorError = true;
 System.out.println("Read command ");
 while (process.isAlive()) {
 //Read the stdOut
 do {
 isOutReady = stdInput.ready();
 //System.out.println("OUT READY " + isOutReady);
 isErrorOut = true;
 isErrorError = true;
 if (isOutReady) {
 line = stdInput.readLine();
 isErrorOut = false;
 System.out.println("=====================================================================================" + line + newLineCharacter);
 }
 isErrorReady = stdError.ready();
 //System.out.println("ERROR READY " + isErrorReady);
 if (isErrorReady) {
 line = stdError.readLine();
 isErrorError = false;
 System.out.println("ERROR::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::" + line + newLineCharacter);
 }
 isProcessAlive = process.isAlive();
 //System.out.println("Process Alive " + isProcessAlive);
 if (!isProcessAlive) {
 System.out.println(":::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Process DIE " + line + newLineCharacter);
 line = null;
 isErrorError = false;
 process.waitFor(1000, TimeUnit.MILLISECONDS);
 }
 } while (line != null);
 //Nothing else to read, lets pause for a bit before trying again
 System.out.println("PROCESS WAIT FOR");
 process.waitFor(100, TimeUnit.MILLISECONDS);
 }
 System.out.println("Command finished");
}

Answer 9

I think I observed a similar problem: some processes started, seemed to run successfully but never completed. The function waitFor() was waiting forever except if I killed the process in Task Manager.
However, everything worked well in cases the length of the command line was 127 characters or shorter. If long file names are inevitable you may want to use environmental variables, which may allow you keeping the command line string short. You can generate a batch file (using FileWriter) in which you set your environmental variables before calling the program you actually want to run. The content of such a batch could look like:

 set INPUTFILE="C:\Directory 0\Subdirectory 1\AnyFileName"
 set OUTPUTFILE="C:\Directory 2\Subdirectory 3\AnotherFileName"
 set MYPROG="C:\Directory 4\Subdirectory 5\ExecutableFileName.exe"
 %MYPROG% %INPUTFILE% %OUTPUTFILE%

Last step is running this batch file using Runtime.

Answer 10

Here is a method that works for me. NOTE: There is some code within this method that may not apply to you, so try and ignore it. For example "logStandardOut(...), git-bash, etc".

private String exeShellCommand(String doCommand, String inDir, boolean ignoreErrors) {
logStandardOut("> %s", doCommand);
ProcessBuilder builder = new ProcessBuilder();
StringBuilder stdOut = new StringBuilder();
StringBuilder stdErr = new StringBuilder();
boolean isWindows = System.getProperty("os.name").toLowerCase().startsWith("windows");
if (isWindows) {
 String gitBashPathForWindows = "C:\\Program Files\\Git\\bin\\bash";
 builder.command(gitBashPathForWindows, "-c", doCommand);
} else {
 builder.command("bash", "-c", doCommand);
}
//Do we need to change dirs?
if (inDir != null) {
 builder.directory(new File(inDir));
}
//Execute it
Process process = null;
BufferedReader brStdOut;
BufferedReader brStdErr;
try {
 //Start the command line process
 process = builder.start();
 //This hangs on a large file
 // https://stackoverflow.com/questions/5483830/process-waitfor-never-returns
 //exitCode = process.waitFor();
 //This will have both StdIn and StdErr
 brStdOut = new BufferedReader(new InputStreamReader(process.getInputStream()));
 brStdErr = new BufferedReader(new InputStreamReader(process.getErrorStream()));
 //Get the process output
 String line = null;
 String newLineCharacter = System.getProperty("line.separator");
 while (process.isAlive()) {
 //Read the stdOut
 while ((line = brStdOut.readLine()) != null) {
 stdOut.append(line + newLineCharacter);
 }
 //Read the stdErr
 while ((line = brStdErr.readLine()) != null) {
 stdErr.append(line + newLineCharacter);
 }
 //Nothing else to read, lets pause for a bit before trying again
 process.waitFor(100, TimeUnit.MILLISECONDS);
 }
 //Read anything left, after the process exited
 while ((line = brStdOut.readLine()) != null) {
 stdOut.append(line + newLineCharacter);
 }
 //Read anything left, after the process exited
 while ((line = brStdErr.readLine()) != null) {
 stdErr.append(line + newLineCharacter);
 }
 //cleanup
 if (brStdOut != null) {
 brStdOut.close();
 }
 if (brStdErr != null) {
 brStdOut.close();
 }
 //Log non-zero exit values
 if (!ignoreErrors && process.exitValue() != 0) {
 String exMsg = String.format("%s%nprocess.exitValue=%s", stdErr, process.exitValue());
 throw new ExecuteCommandException(exMsg);
 }
} catch (ExecuteCommandException e) {
 throw e;
} catch (Exception e) {
 throw new ExecuteCommandException(stdErr.toString(), e);
} finally {
 //Log the results
 logStandardOut(stdOut.toString());
 logStandardError(stdErr.toString());
}
return stdOut.toString();

}

Answer 11

Asynchronous reading of stream combined with avoiding Wait with a timeout will solve the problem.

You can find a page explaining this here http://simplebasics.net/.net/process-waitforexit-with-a-timeout-will-not-be-able-to-collect-the-output-message/

Answer 12

For the ones interested in a solution using kevinarpe's Thread suggestion:

If not using ProcessBuilder + redirectErrorStream(true), you need one thread to read from Process.getInputStream() and another to read from Process.getErrorStream().

(Code in Kotlin)

val process = ProcessBuilder(cmd).start()
val output = StringBuilder()
val errors = StringBuilder()
// See https://docs.oracle.com/javase/8/docs/api/java/lang/Process.html
val outputThread = Thread {
 process.inputStream.bufferedReader().use { it.forEachLine { line -> output.append(line).append("\n") } }
}
val errorThread = Thread {
 process.errorStream.bufferedReader().use { it.forEachLine { line -> errors.append(line).append("\n") } }
}
outputThread.start()
errorThread.start()
val exitCode = process.waitFor()
outputThread.join()
errorThread.join()

Answer 13

public static void main(String[] args) throws PyException, IOException, InterruptedException

these should be the exceptions thrown

• Community: Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.