How-To: Automatic testing for Android Applications

After setting up your Android development environment, and Developing an Android Application it’s time to set-up a automatic Android Test environment. The steps to achieve this are:

  • Choose a test technique.
  • Creating a test project.
  • Make the test.
  • Run the test.

I will use the Android application which was developed inside my previous post.

[Update 15-03-2011] Updated to Android 2.3.3.
[Update 15-03-2011] Updated to Robotium 2.2.

Choose a test technique

There are different techniques you can use to test Android applications. The most commonly known is by way of JUnit. JUnit is used to test the technical quality of applications. But what about the testing the functional use of a application, like you would do it with Selenium for web application. For Android applications there is a similar tool for testing called Robotium. It is a UI testing tool which simulates touching, clicks, typing, and other users’ actions relevant for Android applications. In this example I will be using Robotium.

Creating a test project

  1. From within Eclipse select File menu, New and Other.
  2. Choose Android Test Project inside the Android folder.
  3. Provide the information about the Android Test project and click Next.
  4. Press Finish. You should see the following directory structure

Because I will be creating a Robotium test, I must provide the robotium-solo-2.2.jar as library.

Make the test

  1. From within Eclipse select File menu, New and JUnit Test Case.
  2. Provide the information about the Junit Test Case and click Finish.
  3. Change MainTest.java to the following

package com.my;

import java.util.ArrayList;

import com.jayway.android.robotium.solo.Solo;

import android.app.Activity;
import android.test.ActivityInstrumentationTestCase2;
import android.widget.TextView;

public class MainTest extends ActivityInstrumentationTestCase2 {

private Solo solo;
private Activity activity;

public MainTest() {
super("com.my", Main.class);
}

@Override
protected void setUp() throws Exception {
super.setUp();

this.activity = this.getActivity();
this.solo = new Solo(getInstrumentation(), this.activity);
}

@Override
public void tearDown() throws Exception {
try {
this.solo.finalize();
} catch (Throwable e) {
e.printStackTrace();
}
this.activity.finish();
super.tearDown();
}

/**
* @throws Exception Exception
*/

public void testDisplay() throws Exception {
String text = "Congratulations";

//Enter "Congratulations" inside the EditText field.
this.solo.enterText((EditText) this.activity.findViewById(R.id.EditText01), text);

//Click on the button named "Click".
this.solo.clickOnButton("Click");

//Check to see if the given text is displayed.
assertTrue(this.solo.searchText(text));

TextView outputField = (TextView) this.activity.findViewById(R.id.TextView01);
ArrayList currentTextViews = this.solo.getCurrentTextViews(outputField);
assertFalse(currentTextViews.isEmpty());

TextView output = currentTextViews.get(0);
assertEquals(text, output.getText().toString());
}

}

Run the test

Al that is left is running the test.

  1. Right click MyAndroidAppTestProject.
  2. Select Android Junit Test within the Run-As option.
  3. Be patient, the emulator starts up very slow. You should get the following result

Appendix

Download source code Android.zip [16kB] and AndroidTest.zip [16kB]

  1. manohar
    Hii.. I tried all the steps, but when i execute the test case, it will launch the emulator and finally it does not do any testing by itself. I have to go to menu-devtools-instrumentation and i have to select the test runner then it runs the test. Also in eclipse it shows launching instrumentation test runnner. and shows collecting test information, after that it does not indicate in junit explorer with the green health bar and also it does not show ny information. What is the problem??
  2. W.Elsinga
    Something Eclipse was a hard time initializing the Emulator. Perhaps running the Emulator before running the test will help. What does your AndroidManifest.xml look like?, this causes most of the problems associated with running android j-unit test.
  3. Yair O.
    First - thanks a lot for the sample it aided me in starting. The problem is that R is from your actual project and not the test one . So any R.id. should be the full path to the R in the project itself? Can you please put a link to the source so we can understand project structure ?
  4. W.Elsinga
    I made the source code available for download, see Android.zip and AndroidTest.zip
  5. Chris
    All the issues that I had with implementing Instrumentation are resolve. Robotium has been very helpful in my Android project testing. Cheers...
  6. joseph
    I tried to write a test for a simple app which has only a TextView, and I'm getting the following exception : java.lang.RuntimeException: Exception during suite construction at android.test.suitebuilder.TestSuiteBuilder$FailedToCreateTests.testSuiteConstructionFailed(TestSuiteBuilder.java:239) at java.lang.reflect.Method.invokeNative(Native Method) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:430) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447) Caused by: java.lang.NullPointerException: Method name must not be null. at java.lang.ClassCache.findMethodByName(ClassCache.java:297) at java.lang.Class.getMethod(Class.java:1014) at android.test.suitebuilder.TestMethod.getAnnotation(TestMethod.java:60) at android.test.suitebuilder.annotation.HasMethodAnnotation.apply(HasMethodAnnotation.java:39) at android.test.suitebuilder.annotation.HasMethodAnnotation.apply(HasMethodAnnotation.java:30) at com.android.internal.util.Predicates$OrPredicate.apply(Predicates.java:106) at android.test.suitebuilder.annotation.HasAnnotation.apply(HasAnnotation.java:42) at android.test.suitebuilder.annotation.HasAnnotation.apply(HasAnnotation.java:31) at com.android.internal.util.Predicates$NotPredicate.apply(Predicates.java:122) at android.test.suitebuilder.TestSuiteBuilder.satisfiesAllPredicates(TestSuiteBuilder.java:254) at android.test.suitebuilder.TestSuiteBuilder.build(TestSuiteBuilder.java:190) at android.test.InstrumentationTestRunner.onCreate(InstrumentationTestRunner.java:336) at android.app.ActivityThread.handleBindApplication(ActivityThread.java:3982) at android.app.ActivityThread.access$2900(ActivityThread.java:119) at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1901) at android.os.Handler.dispatchMessage(Handler.java:99) at android.os.Looper.loop(Looper.java:123) at android.app.ActivityThread.main(ActivityThread.java:4363) at java.lang.reflect.Method.invokeNative(Native Method) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:860) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:618) at dalvik.system.NativeStart.main(Native Method) My test code is here: public class TestTextView extends ActivityInstrumentationTestCase2 { private Solo solo; private Activity activity; public TestTextView(String name) { super("com.practise.myapp", MyActivity.class); } protected void setUp() throws Exception { super.setUp(); activity = this.getActivity(); solo = new Solo(getInstrumentation(), activity); System.out.println("There are " + this.countTestCases() + "test cases"); } protected void tearDown() throws Exception { try{ solo.finalize(); } catch ( Throwable e ){ e.printStackTrace(); } activity.finish(); super.tearDown(); } public void testTextContent(){ assertEquals("nothing to test","nothing to test"); } } :-( whats happening here?
  7. W.Elsinga
    Do not use public TestTextView(String name) but public TestTextView() as constructor.
  8. Jimmy
    Hi, I was following your steps but somehow my R.java is not created in Step 1. Why is that? I am a complete newbie. Thanks. I've followed your steps from the previous lesson on creating my own project.
  9. W.Elsinga
    How those your .classpath look like?
  10. W.Elsinga
    Make a Class variable called MVN_REPO inside Eclipse and let it point to your (local) maven repository.
  11. Goran
    Hy. I have imported your files: Android.zip and AndroidTest.zip,but then this error appear: Unbound classpath variable: 'MVN_REPO/com/jayway/android/robotium/robotium-solo/1.6.0/robotium-solo-1.6.0.jar' in project 'MyAndroidAppTestProject' Do you know how to fix it ? Thanks :)
  12. Goran
    It's ok,i fixed it.
  13. anurag
    I am writing an andriod testing application which automates testing on device. I am targeting facebook as my base application and writing an app using Robotium framework in order to accomplish my requirement. Till now i could successfully implemented few features and stuck at one point very badly. I want to automate upload picture functionality but as soon as upload button clicked device builtin application gets activated and i couldn't control the default app using robotium. Is there any way by which i can overcome this impediment by writing some code using robotium or writing a layer between OS & Robotim which can generate key stokes. Quick help would be much much appreciated. Thanks
  14. Julian Harty
    A quick update for this useful example. The sample code (available to download from this page) uses an older version of Robotium (1.6.0). The current version of Robotium 2.0.2 apparently has numerous improvements, and seems to fix a bug that caused a bug in this example code to pass with older versions of Robotium (two wrongs cancelled out). The relevant line of code to change is: this.solo.enterText(R.id.EditText01, text); which will fail with the following (trimmed) stacktrace junit.framework.AssertionFailedError: EditText with index 2131034112 is not available! at com.jayway.android.robotium.solo.Solo.enterText(Solo.java:1155) at com.my.MainTest.testDisplay(MainTest.java:53) -- cut -- There are at least 2 ways to fix the bug in this example (both suggested by Renas) EditText editText = (EditText) solo.getView(R.id.EditText01); this.solo.enterText(editText, text); or this.solo.enterText(0, text); See http://groups.google.com/group/robotium-developers/browse_thread/thread/4787d16125372f36 for the gory details. Note: In eclipse you simply edit the Build Path to point to the current version of the robotium JAR and delete the reference to instance in Maven. This seems to be the issue Goran mentioned in his comments here.
  15. dou haitao
    this.solo.enterText(R.id.EditText01, text); this line script will bring a failure. I think if modify to this.solo.enterText((EditText) this.activity.findViewById(R.id.EditText01), text); will get better. =) thanks.
  16. Madhu
    Hi, I have an doubt. If we have two text boxes in our application, Textbox1, textbox2. while entring the text in textbox 2 also entered into textbox 1. public void testValidateLogin() { String text ="madhu"; String text1="devi"; //Enter UserName solo.enterText(0, text); // Enter the Password solo.enterText(1, text1); //Click Login Button solo.clickOnButton("login"); assertFalse(solo.searchText("Incorrect login credentials")); } This is my Testcase. But unfortunately both strings entered into textbox 1. Kindly clarify my doubt. I am using Emulator: Android 2.3.3 Robotium jar: 1.6.0. Thanks, Madhu
  17. W.Elsinga
    As mention above, please use findViewById:
    this.solo.enterText((EditText) this.activity.findViewById(R.id.EditText01), text);
  18. vijay
    Hi Buddy, I have installed all stuffs for Robotium, I have faced that my project has the problem in Instrumentation mainfeast XML.. probs, When I was run my project ,it displaying the error as below... NotesList does not specify a android.test.InstrumentationTestRunner instrumentation or does not declare uses-library android.test.runner in its AndroidManifest.xml Kindly sort it out ASAP.. Thanks in advance, Regards, Vijay
  19. W.Elsinga
    What are the versions you use?
  20. vijay
    Eclipse Android 4.0 Robotium Solo 2.5 Jar While executing one of my application, it was showing the error as --> java.lang.RuntimeException:Unable to resolve activityfor :Iintent{ act= android:intent.action.MAIN flg=0x1000000 cmp=com.calculator/.test.MyFirsrApp } at android.app.Instrumentation.startActivitySync(Instrumentation.java:370) at android.test.InstrumentationTestCase.launchActivityWithIntent(Instrumentation TestCase.java:119) at android.test.InstrumentationTestCase.launchActivity(Instrumentation TestCase.java:97) at android.test.InstrumentationTestCase2.getActivity(ActivityInstrumentationTestCase2.java:098) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169) at android.test.AndroidTestRunner.runTest (AndroidTestRunner.java:154) at android.test.InstrumentationTestRunner.Onstart(InstrumentationTestRunner.java.537) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.javav:1551) When we click in the error message ,it shows that,Class File Editor Source not found --> msg is displayed.. This is my issue... Kindly sort it out ... ASAP Thanks in advance Regards, Vijay
  21. Vijay
    Android 4.0 Robotium Solo Jar 2.5 JDK 1.7 OS --> Vista When I was run the project, it was showing error, that in Class file Editor Error is showing ,Source is not attached error Please Help me.... Kindly sort it out. Thanks in advance. Regards, Vijay
  22. W.Elsinga
    Looks like a classpath problem. Maybe send me the classpath file of the project by mail
  23. W.Elsinga
    Sorry to say, I did not look it too Robotium 2.5 in combination with Android 4.0 yet. Maybe you can ask your question on http://code.google.com/p/robotium/
  24. peng
    What if i want to test Android Apps without writing test cases? That is to say how can I test android apps total automatic?
  25. W.Elsinga
    Their is no tool on the market that currently support this.
  26. Suki
    hi..i try to execute a robotium test for android notepad but it's not working..please send me a sample robotium test project with the step by step instruction..please...!!:)
  27. W.Elsinga
    At the bottom of this article you can find links to download the source code of this project.
  28. peng
    W.Elsinga: Their is no tool on the market that currently support this.
    I am now taking a try with the methods which TEMA uses. In TEMA, it uses the function that HierarchyViewer, Monkey, for example, provided to generate a user event. The cordinates are provided by the hierarchy information. Do you have any idea about that? Could you give me any advice or details about the attempt to test android apps without test cases? Thanks a lot!
  29. DJ
    Hi, i got an error when i try to run Android Junit test for notepad application. my files are * NoteEditor.java * NotePad.java * NotePadProvider.java * NotesList.java * NotesLiveFolder.java * TitleEditor.java when i run this it will show error: NotesList] Failed to launch test what should i do plz help me...
  30. suki
    I try to run robotium from the ubuntu terminal and its works fine, but now i want to pass the arguments (like, wants to run the test 100 times or 200 times) at the time of running from the command line(terminal). How can i do that using ubuntu terminal??
  31. suki
    hi, I want to do cts test, but it's looking very complex to me. Can any one help me out by describing the cts test from the beginning. Means after setting up android sdk and avd plugins then what we have to do for doing CTS test. Please reply soon.. :)
  32. Anna
    I'm new here... I've tried your links but got errors for both Android.zip and AndroidTest.zip Any suggestions? Thank you, -Anna
    W.Elsinga: I made the source code available for download, see Android.zip and AndroidTest.zip
  33. sandesh
    Hi, I am new to Robotium, I am getting following failure trace. Please reply at your convenience! junit.framework.AssertionFailedError: EditText with index 0 is not available! at com.jayway.android.robotium.solo.Waiter.waitForAndGetView(Waiter.java:362) at com.jayway.android.robotium.solo.Solo.enterText(Solo.java:1290) at com.calculator.test.TestMain.testDisplayBlackBox(TestMain.java:28) at java.lang.reflect.Method.invokeNative(Native Method) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194) at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:529) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1448)
  34. Tanuj
    I am getting the error :- ----------------------------- java.lang.NullPointerException at android.test.InstrumentationTestCase.launchActivityWithIntent(InstrumentationTestCase.java:117) at android.test.InstrumentationTestCase.launchActivity(InstrumentationTestCase.java:97) at android.test.ActivityInstrumentationTestCase2.getActivity(ActivityInstrumentationTestCase2.java:104) at com.calculator.test.StartCalculator.setUp(StartCalculator.java:30) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:190) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:175) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1661) I have tried all possible solutions.

Leave a Reply

*

captcha *

This site uses Akismet to reduce spam. Learn how your comment data is processed.