Applying functional testing is not a new technique during the development of Java software. Writing functional tests gives you a good understanding about the stability of an application or system, during code modifications or if new features are added. But what about using functional testing when developing an Android application, it is also easy to achieve?
“And, is the application still works?” or “Is the application broken?”, these are questions that developers should be able to answer if a code change has been made or an improvement has been added. This is no different when developing an Android application. Also, the use of TDD (Test Driven Development), in which first the test is written, after which the code follows, is good practice. Automated functional testing has become part of software development. Using Selenium and Fitnesse for testing Java application, combined with Jenkins has become avery day pratice to most Java developers. So you would expect that this would differ during the development of Android app. Unfortunately this is not (yet) the case. Because of the relatively rapid growth of Android, these testing methods aren’t yet mature enough.
Lets look at four testing framework that can be used during Android app development:
The Android application that is being tested is a simple calculator. The common test scenario we will be implementing is testing if ‘1’ + ‘3’ gives a result of ‘4’.
The Google test framework UIAutomator makes it possible to write simple Java (and JUnit) functional tests. The framework is not tied to the application that needs to be tested, but also has access to the Android platform itself.
This makes it possible to adjust your Android emulator or physical phone settings while running your test(s). This means that you can test the behavior of the test application if, for example, the internet connection is being switched off.
UiObject appsTab = new UiObject(new UiSelector().text("Apps"));
UiScrollable appViews = new UiScrollable(new UiSelector().scrollable(true));
UiObject calculatorApp = appViews.getChildByText(new UiSelector().className(android.widget.TextView.class.getName()), "Calculator app");
UiObject oneButton = new UiObject(new UiSelector().text("1"));
UiObject plusButton = new UiObject(new UiSelector().text("+"));
UiObject threeButton = new UiObject(new UiSelector().text("3"));
UiObject eqButton = new UiObject(new UiSelector().text("="));
UiObject text = new UiObject(new UiSelector().className("android.widget.EditText").text("4"));
Unfortunately there is little documentation available. Also the framework only supports Android 4.1 or higher.
- Fully control device
- Code written in Java
- Extends JUnit framework
- Easy to read
- Only for Android 4.1 and higher
- Little documentation available
The Java-based testing framework Robolectric is the best of the tested frameworks in terms of performance. This is because it is not running on a device or on the Android emulator, but uses his own visual UI simulator. As a result, the delaying factor of starting an emulator or installing the app on a device has no influence on running the test(s). This makes Robolectric a good choice for TDD.
public class MainActivityTest
private MainActivity _activity;
public void setUp() throws Exception
_activity = Robolectric.buildActivity(MainActivity.class).create().get();
public void testAdding() throws Exception
Button btnOne = (Button) _activity.findViewById(R.id.num_1);
Button btnAdd = (Button) _activity.findViewById(R.id.op_add);
Button btnThree = (Button) _activity.findViewById(R.id.num_3);
Button btnEquals = (Button) _activity.findViewById(R.id.op_equ);
EditText resultText = (EditText) _activity.findViewById(R.id.editText1);
Because Robolectric uses a non-visual UI simulator its not possible to test a problem which only occurs on say a Samsung device.
- Does not require emulator or device
- Code written in Java
- Extends JUnit framework
- Fastest in terms of performance
- Does not run on actual device
The idea behind Robotium is that a functional test should description what the user sees and can do. This framework is a good choice if you are just “black box” testing your application. Besides extensive documentation, there are some well described examples available and the API is easy to use.
private Solo _solo;
protected void setUp() throws Exception
_solo = new Solo(getInstrumentation(), getActivity());
public void testAdding()
_solo.assertCurrentActivity("Is app started", MainActivity.class);
EditText editText = (EditText) getActivity().findViewById(com.elsinga.calculator.R.id.editText1);
- APIs are easy to use
- Tests are text-based
- Good for “black box” testing
Like Robotium, the functional tests described in Calabash Android is written as “What you see you test”. The only difference is that the tests are not written in Java, but in Cucumber (a text-based structure). This makes it possible that tests can be written/used by non developers. A drawback, because test(s) aren’t written in Java, is that “debugging” a failing test is not possible.
I should see "1"
Then I press "1"
And I press "+"
And I press "3"
And I press "="
Then I see "4"
- Well documented
- Test(s) can be writen/used bij non developers
- What you see, you test
- ‘Debuggen’ of tests isn’t possible
Functional testing of Android applications is possible. Which test framework you want to use, dependents on the what you want to do with it. To choice with one fits best, ask yourself who is going to make the tests, what needs to be tested and/or on which device the tests are running on.In any cast, there is enough choice to start writing functional tests.
Source code can be found here