Skip to main content
Article
BOAT: An Experimental Platform for Researchers to Comparatively and Reproducibly Evaluate Bug Localization Techniques
ICSE Companion 2014: Proceedings of the 36th International Conference on Software Engineering: May 31-June 7, Hyderabad, India
  • Xinyu WANG, Zhejiang University
  • David LO, Singapore Management University
  • Xin XIA, Zhejiang University
  • Xingen WANG, Zhejiang University
  • Pavneet Singh KOCHHAR, Singapore Management University
  • Yuan TIAN, Singapore Management University
  • Xiaohu YANG, Zhejiang University
  • Shanping LI, Zhejiang University
  • Jianling SUN, Zhejiang University
  • Bo ZHOU, Zhejiang University
Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
6-2014
Abstract

Bug localization refers to the process of identifying source code files that contain defects from descriptions of these defects which are typically contained in bug reports. There have been many bug localization techniques proposed in the literature. However, often it is hard to compare these techniques since different evaluation datasets are used. At times the datasets are not made publicly available and thus it is difficult to reproduce reported results. Furthermore, some techniques are only evaluated on small datasets and thus it is not clear whether the results are generalizable. Thus, there is a need for a platform that allows various techniques to be compared with one another on a common pool containing a large number of bug reports with known defective source code files. In this paper, we address this need by proposing our Bug lOcalization experimental plATform (BOAT). BOAT is an extensible web application that contains thousands of bug reports with known defective source code files. Researchers can create accounts in BOAT, upload executables of their bug localization techniques, and see how these techniques perform in comparison with techniques uploaded by other researchers, with respect to some standard evaluation measures. BOAT is already preloaded with several bug localization techniques and thus researchers can directly compare their newly proposed techniques against these existing techniques. BOAT has been made available online since October 2013, and researchers could access the platform at: http://www.vlis.zju.edu.cn/blp.

Keywords
  • Benchmark,
  • BOAT,
  • Bug localization,
  • Experimental platform
ISBN
9781450327688
Identifier
10.1145/2591062.2591066
Publisher
ACM
City or Country
New York
Copyright Owner and License
Publisher
Creative Commons License
Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International
Comments

BOAT available at http://www.vlis.zju.edu.cn/blp.

Additional URL
https://doi.org/10.1145/2591062.2591066
Citation Information
Xinyu WANG, David LO, Xin XIA, Xingen WANG, et al.. "BOAT: An Experimental Platform for Researchers to Comparatively and Reproducibly Evaluate Bug Localization Techniques" ICSE Companion 2014: Proceedings of the 36th International Conference on Software Engineering: May 31-June 7, Hyderabad, India (2014) p. 572 - 575
Available at: http://works.bepress.com/david_lo/111/