Skip to main content
Article
File-level defect prediction: Unsupervised vs. supervised models
Proceedings of ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2017)
  • Meng YAN
  • Yicheng FANG
  • David LO, Singapore Management University
  • Xin XIA
  • Xiaohong ZHANG
Publication Type
Conference Proceeding Article
Publication Date
11-2017
Abstract

Background: Software defect models can help software quality assurance teams to allocate testing or code review resources. A variety of techniques have been used to build defect prediction models, including supervised and unsupervised methods. Recently, Yang et al. [1] surprisingly find that unsupervised models can perform statistically significantly better than supervised models in effort-aware change-level defect prediction. However, little is known about relative performance of unsupervised and supervised models for effort-aware file-level defect prediction. Goal: Inspired by their work, we aim to investigate whether a similar finding holds in effort-aware file-level defect prediction. Method: We replicate Yang et al.'s study on PROMISE dataset with totally ten projects. We compare the effectiveness of unsupervised and supervised prediction models for effort-aware file-level defect prediction. Results: We find that the conclusion of Yang et al. [1] does not hold under within-project but holds under cross-project setting for file-level defect prediction. In addition, following the recommendations given by the best unsupervised model, developers needs to inspect statistically significantly more files than that of supervised models considering the same inspection effort (i.e., LOC). Conclusions: (a) Unsupervised models do not perform statistically significantly better than state-of-art supervised model under within-project setting, (b) Unsupervised models can perform statistically significantly better than state-ofart supervised model under cross-project setting, (c) We suggest that not only LOC but also number of files needed to be inspected should be considered when evaluating effort-aware filelevel defect prediction models.

Identifier
10.1109/ESEM.2017.48
Publisher
IEEE
City or Country
Toronto, Canada
Creative Commons License
Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International
Additional URL
http://doi.org/10.1109/ESEM.2017.48
Citation Information
Meng YAN, Yicheng FANG, David LO, Xin XIA, et al.. "File-level defect prediction: Unsupervised vs. supervised models" Proceedings of ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2017) (2017)
Available at: http://works.bepress.com/david_lo/257/