Recovering from Out of Memory Exception within the JVM and reset JVM if possible

Question

I am creating a test suite to run functional tests for an application that my team is developing. The functional tests are written by individual developers, and I observed that some testcases are throwing a OutOfMemoryError due to various reasons. This keeps happening intermittently.

To handle this situation I want to catch OutOfMemoryError in my test suite. I am hoping that catching this error would allow me to run subsequent testcases without having to quit the application all together. But I came across a post saying Out Of Memory Error is thrown after garbage collection is complete. In that case, Can I do something in my catch block of OutOfMemoryError to cleanup/re-instate JVM and then continue the execution?

 Map<String, String> testcaseStatuses = new HashMap<String, String>();
 String testcaseStatus = "FAIL";
 for(Testcase testcase : testcases)
 {
 try
 {
 runTestcase(testcase);
 testcaseStatus = "SUCCESS";
 }
 catch(OutOfMemoryError e)
 {
 logError(e);
 testcaseStatus = "SYSTEM_ERROR";
 // Do something creative to re-initialize JVM
 }
 catch (Exception e)
 {
 logError("Testcase failed with exception : " + e);
 }
 finally
 {
 testcaseStatuses.put(testcase.getName(), testcaseStatus);
 }
 }
 // Do something with testcaseStatuses 

I wouldn't want to stop running the remaining testcases if the first one fails. Is this something feasible?

NOTE: I did come across several posts here on SO, on using

-XX:OnOutOfMemoryError="New script to run"

This is something I am considering for the moment. But in order to follow this path, I would have to re-design the entire way I am executing the testcases which I want to avoid If I can.

Thanks in advance.

• Gimby: Please. Please. Please. With sugar on top. Do the right thing, and fix the cause for the OutOfMemory errors. You are trying to tame chaos here.
• Sree Sake: I am not resposible for the testcases. I am only responsible for running the testcases. My concern is that, if five developers come up with five testcases and the first testcase fails with Out of Memory Error, I don't want the other 4 to wait until the first testcase is fixed, to see if their testcases would pass.
• CryptoFool: I am curious for an answer to this. In a perfect world, @Gimby would have the whole answer. But often, you get what you get in our industry. Maybe you don't control all code involved. Maybe you don't have time to find all the memory problems. Maybe you figure this will happen again, and want to not break your test run next time too. I think your question has merit. I hope someone can provide an solution. All I can see to do is store the point where tests failed, and then restart your test and skip forward to that point and resume.
• CryptoFool: ...trying to save a JVM that has run out of memory seems questionable to me.

Answer 1

Impossible from within the JVM that ran out of memory.

Of course in theory you should fix the cause(s) of those OutOfMemoryErrors, or make sure that those who are responsible fix them. However given the real life scenario that you are in, I'd suggest redesigning your test application to encapsulate each test in its own JVM, e.g. like so:

• Create one application that can execute the tests, but only ever one at a time, chosen by a runtime argument.

• Create an application that runs the former application in a separate process, once for each test. Then use console output, logging data and/or the process return value to determine the state of the test, including if it ran out of memory.

• Sree Sake: I already implemented your second part of solution at a higher level. My actual project is little complicated. To start with I have 10 different processes each with varying number of testcases anywhere from 1 to 50. There is potential to have additional processes in the furture too. There are other requirements like initialization of environment, DB connections etc which will be performed outside JVM and which have a huge overhead and also time-consuming. Hence the requirement to just recover jvm if possible.
• Max: @SreeSake Not possible, especially with all that overhead, shared DB connections, environment etc. You cannot from within the JVM that ran out of memory reliably and safely recover, all hell can break lose if you try. Even if you could, after "recovering" you would have to throw away and re-initialize anything touched by the test that caused the error, to be safe that it still works as it should, as it may be in a corrupt state now. So you won't even win anything in that regard. Simply do not go down that path. The earlier you redesign your test suite to accomodate this fact, the better.