Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Cost-effective testing based fault localization with distance based test-suite reduction

Abstract

The aim of testing based fault localization (TBFL) involves improving the efficiency of program debugging by providing developers with a guide of ranked list of suspicious statements. However, collection of testing information of the whole original test-suite is excessively expensive or even infeasible for developers to conduct TBFL. Traditional test-suite reduction (TSR) techniques are utilized to reduce the size of test-suite. However, they entail a time-consuming process of whole testing information collection. In this study, the distance based test-suite reduction (DTSR) technique is proposed. As opposed to the whole testing information, the distances among the test cases are used to guide the process of test-suite reduction in DTSR. Hence, it is only necessary to collect the testing information for a portion of the test cases for TSR and TBFL. The investigation on the Siemens and SIR benchmarks reveals that DTSR can effectively reduce the size of the given test-suite as well as the time cost of TBFL. Additionally, the fault locating effectiveness of DTSR results is close to that when the whole test-suite is used.

This is a preview of subscription content, log in to check access.

References

  1. 1

    Wong W E, Gao R Z, Li Y H, et al. A survey on software fault localization. IEEE Trans Softw Eng, 2016, 42: 707–740

  2. 2

    Xie X Y, Chen T Y, Kuo F C, et al. A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization. ACM Trans Softw Eng Meth, 2013, 22: 402–418

  3. 3

    Ledru Y, Petrenko A, Boroday S, et al. Prioritizing test cases with string distances. Automat Softw Eng, 2012, 19: 65–95

  4. 4

    Hao D, Xie T, Zhang L, et al. Test input reduction for result inspection to facilitate fault localization. Automat Softw Eng, 2010, 17: 5–31

  5. 5

    Gong L, Lo D, Jiang L X, et al. Diversity maximization speedup for fault localization. In: Proceedings of the 27th International Conference on Automated Software Engineering, Essen, 2012. 30–39

  6. 6

    Yu Y B, Jones J A, Harrold M J. An empirical study of the effects of test-suite reduction on fault localization. In: Proceedings of the 30th International Conference on Software Engineering, Leipzig, 2008. 201–210

  7. 7

    Shi A, Yung T, Gyori A, et al. Comparing and combining test-suite reduction and regression test selection. In: Proceedings of the 10th Joint Meeting on Foundations of Software Engineering, Bergamo, 2015. 237–247

  8. 8

    Jones J A, Harrold M J, Stasko J. Visualization of test information to assist fault localization. In: Proceedings of the 24th International Conference on Software Engineering, Orlando, 2002. 467–477

  9. 9

    Ju X L, Jiang S J, Chen X, et al. HSFal: effective fault localization using hybrid spectrum of full slices and execution slices. J Syst Softw, 2014, 90: 3–17

  10. 10

    Baudry B. Improving test suites for efficient fault localization. In: Proceedings of the 28th International Conference on Software Engineering, Shanghai, 2006. 82–91

  11. 11

    Rothermel G, Harrold M J. Empirical studies of a safe regression test selection technique. IEEE Trans Softw Eng, 1998, 24: 401–419

  12. 12

    Wang R C, Qu B B, Lu Y S. Empirical study of the effects of different profiles on regression test case reduction. IET Softw, 2015, 9: 29–38

  13. 13

    Jiang B, Zhai K, Chan W K, et al. On the adoption of MC/DC and control-flow adequacy for a tight integration of program testing and statistical fault localization. Inform Softw Tech, 2013, 55: 897–917

  14. 14

    Hamming R W. Error detecting and error correcting codes. BELL Labs Tech J, 1950, 29: 147–160

  15. 15

    Levenshtein V I. Binary codes capable of correcting deletions, insertions and reversals. Dokl Akad Nauk Sssr, 1965, 10: 707–710

  16. 16

    Chen T Y, Poon P L, Tang S F, et al. On the identification of categories and choices for specification-based test case generation. Inform Softw Tech, 2004, 46: 887–898

  17. 17

    Zhang X Y, Towey D, Chen T Y, et al. Using partition information to prioritize test cases for fault localization. In: Proceedings of the 39th Annual Computer Software and Applications Conference, Taichung, 2002. 121–126

  18. 18

    Ostrand T J, Balcer M J. The category-partition method for specifying and generating fuctional tests. Commun ACM, 1988, 31: 676–686

  19. 19

    Santelices R, Jones J A, Yu Y B, et al. Lightweight fault-localization using multiple coverage types. In: Proceedings of the 31st International Conference on Software Engineering, Vancouver, 2009. 56–66

  20. 20

    Do H, Elbaum S, Rothermel G. Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir Softw Eng, 2005, 10: 405–435

  21. 21

    Masri W, Assi R A. Prevalence of coincidental correctness and mitigation of its impact on fault localization. ACM Trans Softw Eng Meth, 2014, 23: 294–304

  22. 22

    Zhang Z Y, Chen Z Y, Gao R Z, et al. An empirical study on constraint optimization techniques for test generation. Sci China Inf Sci, 2017, 60: 012105

  23. 23

    Fang C R, Chen Z Y, Xu B W. Comparing logic coverage criteria on test case prioritization. Sci China Inf Sci, 2012, 55: 2826–2840

  24. 24

    Jiang B, Zhang Z Y, Chan W K, et al. Adaptive random test case prioritization. In: Proceedings of the 24th International Conference on Automated Software Engineering, Auckland, 2009. 233–244

  25. 25

    Zhang X F, Xie X Y, Chen T Y. Test case prioritization using adaptive random sequence with category-partition-based distance. In: Proceedings of the 2016 International Conference on Software Quality, Reliability and Security, Vienna, 2016. 374–385

  26. 26

    Chen L, Wang Z Y, Xu L, et al. Test case prioritization for web service regression testing. In: Proceedings of the 5th International Symposium on Service-Oriented System Engineering, Nanjing, 2010. 173–178

  27. 27

    Rothermel G, Harrold M J. Analyzing regression test selection techniques. IEEE Trans Softw Eng, 1996, 22: 529–551

  28. 28

    Wong WE, Horgan J R, London S, et al. Effect of test set minimization on fault detection effectiveness. In: Proceedings of the 17th International Conference on Software Engineering, Washington, 1995. 41–50

  29. 29

    Rothermel G, Harrold M J, Ronne J V. Empirical studies of test-suite reduction. Softw Test Verif Rel, 2002, 12: 219–249

  30. 30

    Wang X Y, Jiang S J, Ju X L, et al. Mitigating the dependence confounding effect for effective predicate-based statistical fault localization. In: Proceedings of the 39th Annual Computer Software and Applications Conference, Taichung, 2015. 105–114

  31. 31

    Vidacs L, Beszedes A, Tengeri D, et al. Test suite reduction for fault detection and localization: a combined approach. In: Proceedings of the 2014 IEEE Conference on Software Maintenance, Reengineering and Reverse Engineering, Antwerp, 2014. 204–213

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. 61673384, 61502497, 61562015), Guangxi Key Laboratory of Trusted Software (Grant Nos. kx201609, kx201532), Scientific Research Innovation Project for Graduate Students of Jiangsu Province (Grant No. KYLX 1390), Science and Technology Program of Xuzhou (Grant No. KC15SM051), and China Postdoctoral Science Foundation (Grant No. 2015M581887).

Author information

Correspondence to Shujuan Jiang.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Jiang, S., Gao, P. et al. Cost-effective testing based fault localization with distance based test-suite reduction. Sci. China Inf. Sci. 60, 092112 (2017). https://doi.org/10.1007/s11432-016-9057-8

Download citation

Keywords

  • program debugging
  • fault localization
  • test-suite reduction
  • distance estimation
  • category partition