Skip to main content
SpringerLink
Log in

Iterative User-Driven Fault Localization

  • Conference paper
  • First Online:
Hardware and Software: Verification and Testing (HVC 2016)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10028))

Included in the following conference series:

Abstract

Because debugging is a notoriously expensive activity, numerous automated debugging techniques have been proposed in the literature. In the last ten years, statistical fault localization emerged as the most popular approach to automated debugging. One problem with statistical fault localization techniques is that they tend to make strong assumptions on how developers behave during debugging. These assumptions are often unrealistic, which considerably limits the practical applicability and effectiveness of these techniques. To mitigate this issue, we propose Swift, an iterative user-driven technique designed to support developers during debugging. Swift (1) leverages statistical fault localization to identify suspicious methods, (2) generates high-level queries to the developer about the correctness of specific executions of the most suspicious methods, (3) uses the feedback from the developer to improve the localization results, and (4) repeats this cycle until the fault has been localized. Our empirical evaluation of Swift, performed on 26 faults in 5 programs, produced promising results; on average, Swift required less than 10 user queries to identify the fault. Most importantly, these queries were only about input/output relationships for specific executions of the methods, which developers should be able to answer quickly and without having to look at the code. We believe that Swift is a first important step towards defining fault localization techniques that account for the presence of humans in the loop and are practically applicable.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. SAEG - Software Analysis and Experimentation Group (at Universidade de São Paulo (USP), Brazil). https://github.com/saeg/experiments/tree/master/jaguar-2015

  2. SIR Repository. http://sir.unl.edu/portal/index.php

  3. Abreu, R., Zoeteweij, P., van Gemund, A.J.C.: An evaluation of similarity coefficients for software fault localization. In: Proceedings of the 12th Pacific Rim International Symposium on Dependable Computing, PRDC 2006, pp. 39–46. IEEE Computer Society, Washington (2006)

    Google Scholar 

  4. Ball, T., Naik, M., Rajamani, S.K.: From symptom to cause: localizing errors in counterexample traces. In: POPL, pp. 97–105 (2003)

    Google Scholar 

  5. Bandyopadhyay, A., Ghosh, S.: Tester feedback driven fault localization. In: Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, ICST 2012, pp. 41–50. IEEE Computer Society, Washington (2012)

    Google Scholar 

  6. Chandra, S., Torlak, E., Barman, S., Bodik, R.: Angelic debugging. In: ICSE, pp. 121–130 (2011)

    Google Scholar 

  7. Cleve, H., Zeller, A.: Locating causes of program failures. In: ICSE, pp. 342–351 (2005)

    Google Scholar 

  8. Demsky, B., Ernst, M.D., Guo, P.J., McCamant, S., Perkins, J.H., Rinard, M.: Inference and enforcement of data structure consistency specifications. In: ISSTA, pp. 233–244 (2006)

    Google Scholar 

  9. Dillig, I., Dillig, T., Aiken, A.: Automated error diagnosis using abductive inference. In: PLDI, pp. 181–192 (2012)

    Google Scholar 

  10. Gong, L., Lo, D., Jiang, L., Zhang, H.: Interactive fault localization leveraging simple user feedback. In: 2012 28th IEEE International Conference on Software Maintenance (ICSM), pp. 67–76, September 2012

    Google Scholar 

  11. Hao, D., Zhang, L., Xie, T., Mei, H., Sun, J.-S.: Interactive fault localization using test information. J. Comput. Sci. Technol. 24(5), 962–974 (2009)

    Article  Google Scholar 

  12. Hao, D., Zhang, L., Zhang, L., Sun, J., Mei, H.: Vida: Visual interactive debugging. In: 2009 IEEE 31st International Conference on Software Engineering, pp. 583–586, May 2009

    Google Scholar 

  13. Jones, J.A., Bowring, J.F., Harrold, M.J.: Debugging in parallel. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis, ISSTA 2007, pp. 16–26. ACM, New York (2007)

    Google Scholar 

  14. Jones, J.A., Harrold, M.J., Stasko, J.: Visualization of test information to assist fault localization. In: ICSE, pp. 467–477 (2002)

    Google Scholar 

  15. Jose, M., Majumdar, R.: Cause clue clauses: error localization using maximum satisfiability. In: PLDI, pp. 437–446 (2011)

    Google Scholar 

  16. Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for Java programs. In: Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), San Jose, CA, USA, pp. 437–440, July 23–25 (2014)

    Google Scholar 

  17. Ko, A.J., Myers, B.A.: Designing the whyline: a debugging interface for asking questions about program behavior. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2004, pp. 151–158. ACM, New York (2004)

    Google Scholar 

  18. Liblit, B., Naik, M., Zheng, A.X., Aiken, A., Jordan, M.I.: Scalable statistical bug isolation. In: PLDI, pp. 15–26 (2005)

    Google Scholar 

  19. Mangal, R., Zhang, X., Nori, A.V., Naik, M.: A user-guided approach to program analysis. In: ESEC/FSE, pp. 462–473 (2015)

    Google Scholar 

  20. Naish, L., Lee, H.J., Ramamohanarao, K.: A model for spectra-based software diagnosis. ACM Trans. Softw. Eng. Methodol. 11:20(3), 1–11:32 (2011)

    Article  Google Scholar 

  21. Parnin, C., Orso, A.: Are automated debugging techniques actually helping programmers? In: ISSTA, pp. 199–209 (2011)

    Google Scholar 

  22. Renieris, M., Reiss, S.P.: Fault localization with nearest neighbor queries. In: ASE, pp. 30–39 (2003)

    Google Scholar 

  23. Shapiro, E.Y.: Algorithmic Program DeBugging. MIT Press, Cambridge (1983)

    MATH  Google Scholar 

  24. Stotts, P.D., Lindsey, M., Antley, A.: An informal formal method for systematic junit test case generation. In: Proceedings of the Second XP Universe and First Agile Universe Conference on Extreme Programming and Agile Methods - XP/Agile Universe 2002, pp. 131–143 (2002)

    Google Scholar 

  25. Vessey, I.: Expertise in debugging computer programs: An analysis of the content of verbal protocols. IEEE Trans. Syst. Man Cybern. B Cybern. 16(5), 621–637 (1986)

    Article  Google Scholar 

  26. Wang, X., Cheung, S.C., Chan, W.K., Zhang, Z.: Taming coincidental correctness: coverage refinement with context patterns to improve fault localization. In: Proceedings of the 31st International Conference on Software Engineering, ICSE 2009, pp. 45–55. IEEE Computer Society , Washington (2009)

    Google Scholar 

  27. Zhang, X., Gupta, N., Gupta, R.: Locating faults through automated predicate switching. In: ICSE, pp. 272–281 (2006)

    Google Scholar 

  28. Zheng, A.X., Jordan, M.I., Liblit, B., Naik, M., Aiken, A.: Statistical debugging: simultaneous identification of multiple bugs. In: ICML, pp. 1105–1112 (2006)

    Google Scholar 

Download references

Acknowledgments

Mayur Naik was engaged in early discussions about this work. Higor Amario de Souza shared the code. This work was partially supported by CNPq grants 457756/2014-4 and 203981/2014-6, by NSF grants CCF1320783 and CCF1161821, and by funding from Google, IBM Research, and Microsoft Research.

Author information

Authors and Affiliations

  1. Georgia Institute of Technology, Atlanta, USA

    Xiangyu Li & Alessandro Orso

  2. Federal University of Pernambuco, Recife, Brazil

    Marcelo d’Amorim

Authors
  1. Xiangyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcelo d’Amorim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alessandro Orso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangyu Li .

Editor information

Editors and Affiliations

  1. IAIK, Graz University of Technology, Graz, Austria

    Roderick Bloem

  2. IBM Research Labs, Haifa, Israel

    Eli Arbel

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Li, X., d’Amorim, M., Orso, A. (2016). Iterative User-Driven Fault Localization. In: Bloem, R., Arbel, E. (eds) Hardware and Software: Verification and Testing. HVC 2016. Lecture Notes in Computer Science(), vol 10028. Springer, Cham. https://doi.org/10.1007/978-3-319-49052-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49052-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49051-9

  • Online ISBN: 978-3-319-49052-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation