Android Test Frameworks

Question

I am new to Android test frameworks ,Would like to know the differences between existing test frameworks : Monkey , CTS ,Instrumentation Framework & Robotium ?

• mparaz: CTS - the Compatibility Test Suite? source.android.com/compatibility/cts-intro.html

Answer 1

Instrumentation is a category of testing, opposite to Unit-testing.

The framework provides hooks for instrumentation testing, but you are going to need an additional third-party framework to really get going.

Robotium is such a framework. It allows you to write "scripts" that run through the user interface, saying "click this", "type that", etc. Well-written it can take you through your usecases and thus provide a good feeling that your app isn't broken. It also allows you to test multiple activities and activities interacting.

Unit-testing in my experience is very hard for Android, especially for the "regular" code dealing with UI, databases, activity state, etc., unless you write your code for testability.

The Android Monkey also uses instrumentation to run through your user interface but it does not follow a script. It does this randomly, with the idea that whatever it does it should not crash your app. By generating 100000's of events it tries to get coverage as high as possible, based on statistics. Other than Robotium, the monkey never leaves your app (that would be dangerous). It's a perfect complement though and it comes nearly for free (the setup is really cheap and there is no maintenance).

CTS is only relevant to the operating system and framework itself.

You'll probably also want to know about mocks?

Answer 2

Observe the testing Pyramid below:

• Manual testing - self explanatory
• Functional testing - testing a feature
• Integration testing - checking the units play nicely
• Unit tests - make sure an individual unit works as expected (See SRP)

It suggests how many tests you should have of each level. Below the pyramid are the Android frameworks that you can use at each level.

Testing Pyramid

In Android, the following frameworks are commonly used for each section:

Functional:

• Monkey runner "kind of" falls under this section, it basically just bashes around the app to see if any combination of interactions crashes it

Integration: Instrumentation falls under this category.

• Espresso (Made by Google, recommended, uses Hamcrest matchers)
• Robotium

Unit:

• JUnit4
• Mockito, Powermock, other mocking libraries
• Matching frameworks like Hamcrest, Fest, AssertJ
• Robolectric (provides Android specific methods)