Clearing static state before testing each method

Question

My SUT class depends on external static state (which generally should be avoided, I know).

How do I make sure my tests do not depend on their order of execution?

I mean other by introducing some reset()/clear() method that is invoked @BeforeEach test. I'm not 100% comfortable introducing changes in business code solely for testing (the business logic does not require such a method, at least not yet).

 /* This is the essence of it:
 MyClass is one of Cache's clients: if it can get 
 a cached MyData instance, it doesn't use its dao */
 @Test
 void doSomething_ifDataCached_noDbCalls() {
 MyData data = new MyData("some expensive data");
 Cache.cache(data);
 MyDao dao = mock();
 MyClass myClass = new MyClass(daoMock);
 myClass.doSomething();
 then(daoMock).shouldHaveNoInteractions();
 }
 @Test
 void doSomething_ifDataNotCached_noDbCalls() {
 MyData data = new MyData("some expensive data");
 // no Cache.cache(data) call this time
 MyDao dao = mock();
 given(dao.findData()).willReturn(Arrays.asList(data));
 MyClass myClass = new MyClass(daoMock);
 myClass.doSomething();
 // fails given this test order: the data's still in cache from the pervious test
 then(daoMock).should().findData();
 }

// works but a bit questionable
 @BeforeEach
 void setUp() {
 Cache.clear();
 }

It's not a "how do I do X" question. Rather, it's about how I can ensure a solid test design.

JUnit 5, Mockito, Java 8.

• Doc Brown: A close vote because some misinterpreted this as "coding/debugging help"? Seriously?
• Doc Brown: ... Sergey, I am under the impression the title lacked the word "testing", I took the freedom to insert it. If you think Its against your intention, let me know.
• Flater: "(which generally should be avoided, I know)" It's one thing to at least acknowledge the issue, but it is another thing to then sidestep the issue and pretend like you can just carry on without being negatively impacted by the issue. This is precisely why shared static state is such a bad idea. Do you really want to invest in working around that, with all the added complexity of how you could even manage to test your work; as opposed to tackling the issue that you already acknowledge is an issue?

Answer 1

how I can ensure a solid test design.

Well, like you said:

external static state (which generally should be avoided, I know)

But, if you're determined/doomed to work with a bad design here's some things I've learned the hard way.

// works but a bit questionable
 @BeforeEach
 void setUp() {
 Cache.clear();
 }

is a bad idea. Doing the clear before is fine. Just not each. This one size fits all approach will seem to work fine right up until it doesn't.

A better solution is to put the clear call at the start of each test that needs that particular form of clear. This way you're making the tests needs obvious. To make this work well, don't fix things the test doesn't care about. Doing that makes what the test cares about clear.

Some argue that you should put the clear after the test to clean up nicely for the next test. I argue against that for the same reason. The test should know it's own needs better than anything else. Also, when done "before" style, a failure to clear makes the test that needs fixing fail. Not some other one.

Yes I know this is more work. This isn't the easy way. Just better.

• Basilevs: Global state is implicit by nature, tests doing partial and specific resets would be brittle.
• candied_orange: @Basilevs dependence on global state is test specific. Don't pretend the test cares when it doesn't.
• Basilevs: Dependence on global state is a property of component under test, not the test itself. By doing specific actions in test, you make the test depend on the particulars on components implementation - doing whitebox testing. The approach you suggest makes the test care more, not less.
• candied_orange: @Basilevs the component under test is part of the test. By making its actual needs clear (rather than over engineering it) you enable refactoring's to pair down those needs. Hiding them doesn't help.
• Doc Brown: When the result of your suggestion results in a class which contains Cache.clear(); in each and every function, maybe a dozen or more, then using @BeforeEach may start looking not as bad as this answer describes it. Alternative,, when the test class contains cerrtain function which require Cache.clear(); and other's which may work wrong with that kind of initialization, then avoiding @BeforeEach is a no-brainer. Or in other words: it depends.

Answer 2

How do I make sure my tests do not depend on their order of execution?

The usual answer is to design your test subjects so that they can be configured to be isolated.

Michael Feathers (Working Effectively with Legacy Code) refers to this sort of configuration as a "seam" -- the idea being that we can vary the way your test subject works by changing code somewhere else (ex: in the test).

My SUT class depends on external static state (which generally should be avoided, I know).

So what we would normally do is to introduce a seam such that the dependency on shared state can be controlled from the test.

(In effect, what we are doing is making "... and it needs to be possible to isolate the SUT..." part of the requirements -- it's a design constraint.)

I'm not 100% comfortable introducing changes in business code solely for testing

Yup - I get it. The "advanced" answer is "get over it". The basic idea is analogous to working in hardware - being able to connect to the test pins to communicate more directly with the device is necessary to verifying that it does the right thing, so that gets built into the design.

The Doctrine of Useful Objects covers a lot of these ideas (with a bias toward the idea that any object that is complicated enough to need it's logic tested should default to being isolated; you need to "opt-in" if you want to connect the logic to something on the outside).

• Basilevs: More examples, less buzzwords?
• Basilevs: The link on Useful Objects describes a bunch of antipatterns glued together. Is that a troll article?

Answer 3

I am a little confused, because you already seem to do what I will suggest, but I will spell it out anyway:

You inject your cache just like your data access. Your third party does not support that? Well, then wrap it.

Create a ICache interface with the methods you need (get/put).

Inject this interface everywhere you need a cache, just like you inject your data access everywhere you need it.

Implement one class for ICache that is the actual cache third party library. Implement one class for ICache that you can use for unit testing and that is some kind of memory cache.

Unit test the actual implementation. For all other tests using an ICache, use your non-static, in-memory implmentatation.

I am not a big fan of mocking frameworks. If you are into that, you can obviously use your mocking framework to setup up a mock, instead of writing a memory cache.

So your unit tests become (sorry if there is a minor syntax error, I'm not a Java programmer):

/* This is the essence of it:
 MyClass is one of Cache's clients: if it can get 
 a cached MyData instance, it doesn't use its dao */
@Test
void doSomething_ifDataCached_noDbCalls() {
 MyData data = new MyData("some expensive data");
 ICache cache = new UnitTestInMemoryCache();
 cache.put(data);
 MyDao dao = mock();
 MyClass myClass = new MyClass(cache, daoMock);
 myClass.doSomething();
 then(daoMock).shouldHaveNoInteractions();
}
@Test
void doSomething_ifDataNotCached_noDbCalls() {
 MyData data = new MyData("some expensive data");
 ICache cache = new UnitTestInMemoryCache();
 // no cache.put(data); call this time
 MyDao dao = mock();
 given(dao.findData()).willReturn(Arrays.asList(data));
 MyClass myClass = new MyClass(daoMock);
 myClass.doSomething();
 then(daoMock).should().findData();
}

• nvoigt: Feel free to leave a comment if you think this is not useful. I can try my best to improve it given constructive criticism, but I can only leave it as it is without.
• Basilevs: The question was about static state, your solutions are about injected state.
• nvoigt: Yes, indeed, my solution is to mock out the static state in testing and only use it in production. Just because the third party uses static state doesn't mean one has to sprinkle it through the whole program. You can still put it behind an interface and make it testable this way.