Skip to main content
Article
Understanding the Test Automation Culture of App Developers
ICST 2015 IEEE International Conference on Software Testing, Verification and Validation: 13-17 April 2015, Graz: Proceedings
  • Pavneet Singh KOCHHAR
  • Ferdian. THUNG
  • Nachiappan NAGAPPAN
  • Thomas ZIMMERMANN
  • David LO
Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
4-2015
Abstract

Smartphone applications (apps) have gained popularity recently. Millions of smartphone applications (apps) are available on different app stores which gives users plethora of options to choose from, however, it also raises concern if these apps are adequately tested before they are released for public use. In this study, we want to understand the test automation culture prevalent among app developers. Specifically, we want to examine the current state of testing of apps, the tools that are commonly used by app developers, and the problems faced by them. To get an insight on the test automation culture, we conduct two different studies. In the first study, we analyse over 600 Android apps collected from F- Droid, one of the largest repositories containing information about open-source Android apps. We check for the presence of test cases and calculate code coverage to measure the adequacy of testing in these apps. We also survey developers who have hosted their applications on GitHub to understand the testing practices followed by them. We ask developers about the tools that they use and ''pain points'' that they face while testing Android apps. For the second study, based on the responses from Android developers, we improve our survey questions and resend it to Windows app developers within Microsoft. We conclude that many Android apps are poorly tested - only about 14% of the apps contain test cases and only about 9% of the apps that have executable test cases have coverage above 40%. Also, we find that Android app developers use automated testing tools such as JUnit, Monkeyrunner, Robotium, and Robolectric, however, they often prefer to test their apps manually, whereas Windows app developers prefer to use in-house tools such as Visual Studio and Microsoft Test Manager. Both Android and Windows app developers face many challenges such as time constraints, compatibility issues, lack of exposure, cumbersome tools, etc. We give suggestions to improve the test automation culture in the growing app community.

Keywords
  • Android,
  • App Developers,
  • Microsoft,
  • Test Automation Culture,
  • smartphones
ISBN
9781479971251
Identifier
10.1109/ICST.2015.7102609
Publisher
IEEE
City or Country
Piscataway, NJ
Creative Commons License
Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International
Additional URL
http://dx.doi.org/10.1109/ICST.2015.7102609
Citation Information
Pavneet Singh KOCHHAR, Ferdian. THUNG, Nachiappan NAGAPPAN, Thomas ZIMMERMANN, et al.. "Understanding the Test Automation Culture of App Developers" ICST 2015 IEEE International Conference on Software Testing, Verification and Validation: 13-17 April 2015, Graz: Proceedings (2015) p. 1 - 10
Available at: http://works.bepress.com/david_lo/345/