Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Lightweight fault localization combined with fault context to improve fault absolute rank

Abstract

Lightweight fault localization (LFL), which outputs a list of suspicious program entities in descending order based on their likelihood to be a root fault, is a popular method used by programmers to assist them in debugging. However, owing to the nature of program structures, it is impossible for LFL to always rank the root faulty program entity at the top of a ranking list. Recently, Xia et al. noted in their study that programmers inspect the first top-K-ranked program entities outputted by an LFL tool in sequence. Therefore, it is of practical significance to further improve the absolute rank of those buggy programs by using LFL if the root fault is ranked higher. To solve this issue, we propose a new LFL combined with a fault context to improve the fault absolute rank. We conduct experiments in which we apply our proposed approach to seven Siemens benchmark programs. The results show that by combining the suspiciousness scores of program entities with their fault-context suspiciousness scores that are based on an LFL called Dstar, our approach can improve the fault absolute rank with an effectiveness rate of 35.7% for 129 faulty versions from the seven benchmark programs. It should be noted that our approach can obtain an average improvement of 65.18% for those improved programs to which LFL can be effectively applied, and that there were improvements to seven top-ranked root faults of buggy programs.

This is a preview of subscription content, log in to check access.

References

  1. 1

    Wong W E, Gao R, Li Y, et al. A survey on software fault localization. IEEE Trans Softw Eng, 2016, 42: 707–740

  2. 2

    Chen M Y, Kiciman E, Fratkin E, et al. Pinpoint: problem determination in large, dynamic internet services. In: Proceedings of International Conference on Dependable Systems and Networks, Bethesda, 2002. 595–604

  3. 3

    Jones J A, Harrold M J, Stasko J. Visualization of test information to assist fault localization. In: Proceedings of the 24th International Conference on Software Engineering, Orlando, 2002. 467–477

  4. 4

    Abreu R, Zoeteweij P, Golsteijn R, et al. A practical evaluation of spectrum-based fault localization. J Syst Softw, 2009, 82: 1780–1792

  5. 5

    Jones J A, Harrold M J. Empirical evaluation of the tarantula automatic fault-localization technique. In: Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering. Long Beach: ACM, 2005. 273–282

  6. 6

    Parnin C, Orso A. Are automated debugging techniques actually helping programmers? In: Proceedings of the 2011 International Symposium on Software Testing and Analysis, Toronto, 2011. 199–209

  7. 7

    Kochhar P S, Xia X, Lo D, et al. Practitioners’ expectations on automated fault localization. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, Saarbrcken, 2016. 165–176

  8. 8

    Xie X, Liu Z, Song S, et al. Revisit of automatic debugging via human focus-tracking analysis. In: Proceedings of the 38th International Conference on Software Engineering, Austin, 2016. 808–819

  9. 9

    Xia X, Bao L, Lo D, et al. “Automated debugging considered harmful” considered harmful: a user study revisiting the usefulness of spectra-based fault localization techniques with professionals using real bugs from large systems. In: Proceedings of 2016 IEEE International Conference on Software Maintenance and Evolution. Raleigh: IEEE, 2016. 267–278

  10. 10

    Thung F, Lo D, Jiang L. Are faults localizable? In: Proceedings of the 9th IEEE Working Conference on Mining Software Repositories, Zurich, 2012. 74–77

  11. 11

    Le T D B, Lo D, Thung F. Should I follow this fault localization tool’s output? Automated prediction of fault localization effectiveness. Empir Softw Eng, 2015, 20: 1237–1274

  12. 12

    Do H, Elbaum S, Rothermel G. Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir Softw Eng, 2005, 10: 405–435

  13. 13

    Wong W E, Debroy V, Gao R Z, et al. The Dstar method for effective software fault localization. IEEE Trans Reliab, 2014, 63: 290–308

  14. 14

    Harrold M J, Rothermel G, Sayre K, et al. An empirical investigation of the relationship between spectra differences and regression faults. Software Test Verif Reliab, 2000, 10: 171–194

  15. 15

    Choi S S, Cha S H, Tappert C C. A survey of binary similarity and distance measures. J Syst Cybern Inf, 2010, 8: 43–48

  16. 16

    Renieres M, Reiss S P. Fault localization with nearest neighbor queries. In: Proceedings of the 18th IEEE International Conference on Automated Software Engineering, Montreal, 2003. 30–39

  17. 17

    Liblit B, Naik M, Zheng A X, et al. Scalable statistical bug isolation. ACM SIGPLAN Notices, 2005, 40: 15–26

  18. 18

    Liu C, Yan X, Fei L, et al. SOBER: statistical model-based bug localization. ACM SIGSOFT Software Eng Notes, 2005, 30: 286–295

  19. 19

    Wong W E, Debroy V, Golden R, et al. Effective software fault localization using an RBF neural network. IEEE Trans Reliab, 2012, 61: 149–169

  20. 20

    Wong W E, Debroy V, Xu D X. Towards better fault localization: a crosstab-based statistical approach. IEEE Trans Syst Man Cybern Part C-Appl Rev, 2012, 42: 378–396

  21. 21

    Yu Y, Jones J A, Harrold M J. An empirical study of the effects of test-suite reduction on fault localization. In: Proceedings of the 30th International Conference on Software Engineering. Leipzig: ACM, 2008. 201–210

  22. 22

    Xia X, Gong L, Le T D B, et al. Diversity maximization speedup for localizing faults in single-fault and multi-fault programs. Autom Softw Eng, 2016, 23: 43–75

  23. 23

    Xie X, Wong W E, Chen T Y, et al. Spectrum-based fault localization: testing oracles are no longer mandatory. In: Proceedings of the 11th International Conference on Quality Software, Madrid, 2010. 1–10

  24. 24

    Gong D D, Wang T T, Su X H, et al. A test-suite reduction approach to improving fault-localization effectiveness. Comput Lang Syst Struct, 2013, 39: 95–108

  25. 25

    Lo D, Xia X. Fusion fault localizers. In: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering, Vasteras, 2014. 127–138

  26. 26

    Santelices R, Jones J, Yu Y, et al. Lightweight fault-localization using multiple coverage types. In: Proceedings of IEEE 31st International Conference on Software Engineering, Vancouver, 2009. 56–66

  27. 27

    Zhang Z, Chan W K, Tse T, et al. Capturing propagation of infected program states. In: Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, Amsterdam, 2009. 43–52

  28. 28

    Naish L, Lee H J, Ramamohanarao K. Spectral debugging with weights and incremental ranking. In: Proceedings of the 16th Asia-Pacific Software Engineering Conference. Batu Ferringhi: IEEE, 2009. 168–175

  29. 29

    Xu S, Xu J, Yang H, et al. An improvement to fault localization technique based on branch-coverage spectra. In: Proceedings of the 39th IEEE Annual Computer Software and Applications Conference. Taichung: IEEE, 2015. 2: 282–287

  30. 30

    Baah G K, Podgurski A, Harrold M J. Causal inference for statistical fault localization. In: Proceedings of the 19th International Symposium on Software Testing and Analysis, Trento, 2010. 73–84

Download references

Acknowledgements

This work was supported by National High Technology Research and Development Program of China (863) (Grant No. 2015AA015303), National Natural Science Foundation of China (Grant Nos. 61272083, 61562087, 71371012, 61300170, 61572033), Key Support Program Projects for Outstanding Young Talents of Anhui Province (Grant No. gxyqZD2016124), Advanced Research of National Natural Science Foundation (Grant No. 2016yyzr10), and Anhui Natural Science Foundation (Grant Nos. KJ2016A252, 1608085MF147).

Author information

Correspondence to Zhiqiu Huang.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Huang, Z., Li, Y. et al. Lightweight fault localization combined with fault context to improve fault absolute rank. Sci. China Inf. Sci. 60, 092113 (2017). https://doi.org/10.1007/s11432-017-9112-2

Download citation

Keywords

  • fault context
  • fault localization
  • program spectrum
  • absolute rank
  • debugging