Skip to main content
Article
Will fault localization work for these failures? An automated approach to predict effectiveness of fault localization tools
2013 29th IEEE International Conference on Software Maintenance: Eindhoven, September 22-28: Proceedings
  • Tien-Duy B. LE, Singapore Management University
  • David LO, Singapore Management University
Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
9-2013
Abstract

Debugging is a crucial yet expensive activity to improve the reliability of software systems. To reduce debugging cost, various fault localization tools have been proposed. A spectrum-based fault localization tool often outputs an ordered list of program elements sorted based on their likelihood to be the root cause of a set of failures (i.e., their suspiciousness scores). Despite the many studies on fault localization, unfortunately, however, for many bugs, the root causes are often low in the ordered list. This potentially causes developers to distrust fault localization tools. Recently, Parnin and Orso highlight in their user study that many debuggers do not find fault localization useful if they do not find the root cause early in the list. To alleviate the above issue, we build an oracle that could predict whether the output of a fault localization tool can be trusted or not. If the output is not likely to be trusted, developers do not need to spend time going through the list of most suspicious program elements one by one. Rather, other conventional means of debugging could be performed. To construct the oracle, we extract the values of a number of features that are potentially related to the effectiveness of fault localization. Building upon advances in machine learning, we process these feature values to learn a discriminative model that is able to predict the effectiveness of a fault localization tool output. In this preliminary work, we consider an output of a fault localization tool to be effective if the root cause appears in the top 10 most suspicious program elements. We have experimented our proposed oracle on 200 faulty programs from Space, NanoXML, XML-Security, and the 7 programs in Siemens test suite. Our experiments demonstrate that we could predict the effectiveness of fault localization tool with a precision, recall, and F-measure (harmonic mean of precision and recall) of 54.36%, 95.29%, and 69.23%. The numbers indicate that many ineffective fault localization instances are identified correctly, while only very few effective ones are identified wrongly.

Keywords
  • Classification,
  • Fault Localization,
  • Effectiveness Prediction
ISBN
9780769549811
Identifier
10.1109/ICSM.2013.42
Publisher
IEEE Computer Society
City or Country
Los Alamitos, CA
Copyright Owner and License
Publisher
Creative Commons License
Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International
Additional URL
https://doi.org/10.1109/ICSM.2013.42
Citation Information
Tien-Duy B. LE and David LO. "Will fault localization work for these failures? An automated approach to predict effectiveness of fault localization tools" 2013 29th IEEE International Conference on Software Maintenance: Eindhoven, September 22-28: Proceedings (2013) p. 310 - 319
Available at: http://works.bepress.com/david_lo/135/