Skip to main content
Article
The impact of coverage on bug density in a large industrial software project
ESEM 2017: Proceedings of 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement: Toronto, Canada, November 9-10
  • Thomas BACH, Heidelberg University
  • Artur ANDRZEJAK, Heidelberg University
  • Ralf PANNEMANS, SAP, Germany
  • David LO, Singapore Management University
Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
11-2017
Abstract

Measuring quality of test suites is one of the major challenges of software testing. Code coverage identifies tested and untested parts of code and is frequently used to approximate test suite quality. Multiple previous studies have investigated the relationship between coverage ratio and test suite quality, without a clear consent in the results. In this work we study whether covered code contains a smaller number of future bugs than uncovered code (assuming appropriate scaling). If this correlation holds and bug density is lower in covered code, coverage can be regarded as a meaningful metric to estimate the adequacy of testing. To this end we analyse 16000 internal bug reports and bug-fixes of SAP HANA, a large industrial software project. We found that the above-mentioned relationship indeed holds, and is statistically significant. Contrary to most previous works our study uses real bugs and real bug-fixes. Furthermore, our data is derived from a complex and large industrial project.

Keywords
  • software quality,
  • coverage,
  • bug density,
  • large real world project,
  • industry project,
  • empirical research
ISBN
9781509040391
Identifier
10.1109/ESEM.2017.44
Publisher
IEEE
City or Country
Piscataway, NJ
Copyright Owner and License
Authors
Creative Commons License
Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International
Additional URL
https://doi.org/10.1109/ESEM.2017.44
Citation Information
Thomas BACH, Artur ANDRZEJAK, Ralf PANNEMANS and David LO. "The impact of coverage on bug density in a large industrial software project" ESEM 2017: Proceedings of 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement: Toronto, Canada, November 9-10 (2017) p. 307 - 313
Available at: http://works.bepress.com/david_lo/335/