Search Based Test Suite Minimization for Fault Detection and Localization: A Co-driven Method

Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9962)

Abstract

Fault detection and fault localization are two independent but important processes in software testing, in which test cases and associated information are usually used. As there are different goals of the two processes, i.e., detect faults early and locate faults accurately, different information of test cases are required and thus different subsets of test cases are selected to achieve the goals. In general, fault localization is adjacent to fault detection. However, independence of these two processes will restrict the automatic process of software testing. This paper proposes an automatic approach to combining fault detection and fault localization, where a multi-objective optimization for test suite minimization is presented to balance and achieve the both goals. Empirical studies show the proposed method can give consideration to both fault detection and fault localization with a high test suite reduction ratio.

Keywords

Multi-objective Test Suite Minimization Fault Localization Fault detection Combination 
This is a preview of subscription content, log in to check access

Notes

Acknowledgment

The work described in this paper is supported by the National Natural Science Foundation of China under Grant No.61170082 and 61472025 and the Program for New Century Excellent Talents in University (NCET-12-0757).

References

  1. 1.
    Branke, J., Deb, K., Dierolf, H., Osswald, M.: Finding knees in multi-objective optimization. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 722–731. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Dandan, G., Tiantian, W., Xiaohong, S., Peijun, M.: A test-suite reduction approach to improving fault-localization effectiveness. Comput. Lang. Syst. Struct. 39(3), 95–108 (2013)Google Scholar
  3. 3.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)CrossRefGoogle Scholar
  4. 4.
    Do, H., Elbaum, S.G., Rothermel, G.: Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empirical Softw. Eng. Int. J. 10(4), 405–435 (2005)CrossRefGoogle Scholar
  5. 5.
    Gonzalez-Sanchez, A., Abreu, R., Gross, H.G., van Gemund, A.J.: Prioritizing tests for fault localization through ambiguity group reduction. In: 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 83–92. IEEE (2011)Google Scholar
  6. 6.
    Gonzalez-Sanchez, A., Piel, É., Abreu, R., Gross, H.-G., van Gemund, A.J.C.: Prioritizing tests for fault localization. In: van de Laar, P., Tretmans, J., Borth, M. (eds.) Situation Awareness with Systems of Systems, pp. 247–257. Springer, New York (2013)CrossRefGoogle Scholar
  7. 7.
    Hao, D., Xie, T., Zhang, L., Wang, X., Sun, J., Mei, H.: Test input reduction for result inspection to facilitate fault localization. Autom. Softw. Eng. 17(1), 5–31 (2010)CrossRefGoogle Scholar
  8. 8.
    Harman, M., Jia, Y., Zhang, Y.: Achievements, open problems and challenges for search based software testing. In: IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pp. 1–12 (2015)Google Scholar
  9. 9.
    Harman, M., Jones, B.F.: Search-based software engineering. Inf. Softw. Technol. 43(14), 833–839 (2001)CrossRefGoogle Scholar
  10. 10.
    Harrold, M.J., Rothermel, G., Wu, R., Yi, L.: An empirical investigation of program spectra. In: ACM SIGPLAN Notices, vol. 33, pp. 83–90. ACM (1998)Google Scholar
  11. 11.
    Jones, J.A., Harrold, M.J.: Empirical evaluation of the tarantula automatic fault-localization technique. In: Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering, pp. 273–282. ACM (2005)Google Scholar
  12. 12.
    Li, Z., Harman, M., Hierons, R.: Search algorithms for regression test case prioritization. IEEE Trans. Softw. Eng. 33(4), 225–237 (2007)CrossRefGoogle Scholar
  13. 13.
    Masri, W., Assi, R.: Cleansing test suites from coincidental correctness to enhance fault-localization. In: Third International Conference on Software Testing, Verification and Validation (ICST), pp. 165–174 (2010)Google Scholar
  14. 14.
    Naish, L., Lee, H.J., Ramamohanarao, K.: A model for spectra-based software diagnosis. ACM Trans. Softw. Eng. Methodol. (TOSEM) 20(3), 11 (2011)CrossRefGoogle Scholar
  15. 15.
    Praditwong, K., Harman, M., Yao, X.: Software module clustering as a multi-objective search problem. IEEE Trans. Softw. Eng. 37(2), 264–282 (2011)CrossRefGoogle Scholar
  16. 16.
    Vidács, L., Beszédes, Á., Tengeri, D., Siket, I., Gyimóthy, T.: Test suite reduction for fault detection and localization: a combined approach. In: 2014 Software Evolution Week-IEEE Conference on Software Maintenance, Reengineering and Reverse Engineering (CSMR-WCRE), pp. 204–213. IEEE (2014)Google Scholar
  17. 17.
    Wang, Q., Parnin, C., Orso, A.: Evaluating the usefulness of IR-based fault localization techniques. In: Proceedings of the 2015 International Symposium on Software Testing and Analysis, pp. 1–11. ACM (2015)Google Scholar
  18. 18.
    Wong, W.E., Debroy, V.: A survey of software fault localization. Department of Computer Science, University of Texas at Dallas, Technical report UTDCS-45-09 (2009)Google Scholar
  19. 19.
    Wong, W.E., Qi, Y., Zhao, L., Cai, K.Y.: Effective fault localization using code coverage. In: 31st Annual International Computer Software and Applications Conference, COMPSAC 2007, vol. 1, pp. 449–456. IEEE (2007)Google Scholar
  20. 20.
    Xie, X., Chen, T.Y., Kuo, F.C., Xu, B.: A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization. ACM Trans. Softw. Eng. Methodol. 22(4), 31:1–31:40 (2013)CrossRefGoogle Scholar
  21. 21.
    Xie, X., Kuo, F.-C., Chen, T.Y., Yoo, S., Harman, M.: Provably optimal and human-competitive results in SBSE for spectrum based fault localisation. In: Ruhe, G., Zhang, Y. (eds.) Search Based Software Engineering. LNCS, vol. 8084, pp. 224–238. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  22. 22.
    Yoo, S., Harman, M.: Pareto efficient multi-objective test case selection. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis, ISSTA 2007, pp. 140–150. ACM, New York (2007)Google Scholar
  23. 23.
    Yoo, S., Harman, M.: Pareto efficient multi-objective test case selection. In: Proceedings of the 2007 International Symposium on Software Testing and Analysis, pp. 140–150. ACM (2007)Google Scholar
  24. 24.
    Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw. Test. Verification Reliab. 22(2), 67–120 (2012)CrossRefGoogle Scholar
  25. 25.
    Yoo, S., Harman, M., Clark, D.: Fault localization prioritization: comparing information-theoretic and coverage-based approaches. ACM Trans. Softw. Eng. Methodol. 22(3), 19:1–19:29 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Jingyao Geng
    • 1
  • Zheng Li
    • 1
  • Ruilian Zhao
    • 1
  • Junxia Guo
    • 1
  1. 1.College of Information Science and TechnologyBeijing University of Chemical TechnologyBeijingPeople’s Republic of China

Personalised recommendations