Advertisement

FTScMES: A New Mutation Execution Strategy Based on Failed Tests’ Mutation Score for Fault Localization

  • André Assis Lôbo de Oliveira
  • Celso G. CamiloJr
  • Eduardo Noronha de Andrade Freitas
  • Auri Marcelo Rizzo Vincenzi
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 935)

Abstract

Fault localization has been one of the most expensive activity in the whole debugging process. The spectrum-based fault localization (SBFL) is the most studied and evaluated technique. Other approach is the mutation-based fault localization technique (MBFL) that needs to execute the test suite on a large amount of mutants increasing its cost. Efforts from research community have been performed in order to achieve solutions reducing such cost and keeping a minimum quality. Current mutation execution strategies are evaluated considering artificial faults. However, recent researches show that some MBFL techniques exhibit low efficacy fault localization when evaluated on real faults. In this paper, we propose a new mutation execution strategy based on failed tests’ mutation score, called (FTScMES), aiming to increase the efficacy on fault localization reducing the cost of mutants execution. FTScMES uses only the set of failed test cases to execute mutants and bases on mutation score concept to compute the suspiciousness statements. The experiments were conducted considering 221 real faults, comparing the efficacy of localization of FTScMES against 5 baselines from the literature. We found that FTScMES outperforms the baselines reducing the cost in 90% on average with a high efficacy of ranking defective code.

Keywords

Fault localization Mutation analysis Mutation-based fault localization Debugging Mutation execution strategy 

Notes

Acknowledgments

This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001, Universidade Federal de Mato Grosso do Sul (UFMS) and, the Instituo Federal de Educação, Ciência e Tecnologia de Mato Grosso (IFMT) for their financial support.

References

  1. 1.
    National institute of standards and technology. http://www.abeacha.com/nist_press_release_bugs_cost.htm
  2. 2.
    Abreu, R., Zoeteweij, P., c. Van Gemund, A.J.: An evaluation of similarity coefficients for software fault localization. In: 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC 2006), pp. 39–46, December 2006Google Scholar
  3. 3.
    Abreu, R., Zoeteweij, P., van Gemund, A.J.C.: On the accuracy of spectrum-based fault localization. In: Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION, pp. 89–98. TAICPART-MUTATION 2007. IEEE Computer Society, Washington, DC, USA (2007)Google Scholar
  4. 4.
    Campos, J., Riboira, A., Perez, A., Abreu, R.: Gzoltar: an eclipse plug-in for testing and debugging. In: 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, pp. 378–381, September 2012.  https://doi.org/10.1145/2351676.2351752
  5. 5.
    Gong, P., Zhao, R., Li, Z.: Faster mutation-based fault localization with a novel mutation execution strategy. In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–10, April 2015Google Scholar
  6. 6.
    Jones, J.A., Harrold, M.J.: Empirical evaluation of the tarantula automatic fault-localization technique. In: Proceedings of the IEEE/ACM International Conference on Automated Software Engineering, pp. 273–282 (2005)Google Scholar
  7. 7.
    Just, R.: The major mutation framework: efficient and scalable mutation analysis for java. In: Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014, pp. 433–436. ACM, New York (2014).  https://doi.org/10.1145/2610384.2628053
  8. 8.
    Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for Java programs. In: Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014, pp. 437–440. San Jose, CA, USA, July 2014. (tool demo)Google Scholar
  9. 9.
    Liu, Y., Li, Z., Wang, L., Hu, Z., Zhao, R.: Statement-oriented mutant reduction strategy for mutation based fault localization. In: 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS), pp. 126–137, July 2017Google Scholar
  10. 10.
    Moon, S., Kim, Y., Kim, M., Yoo, S.: Ask the mutants: mutating faulty programs for fault localization. In: Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation, ICST 2014, pp. 153–162. IEEE Computer Society, Washington, DC, USA (2014)Google Scholar
  11. 11.
    Naish, L., Lee, H.J., Ramamohanarao, K.: A model for spectra-based software diagnosis. ACM Trans. Softw. Eng. Methodol. 20(3), 11:1–11:32 (2011).  https://doi.org/10.1145/2000791.2000795CrossRefGoogle Scholar
  12. 12.
    Papadakis, M., Traon, Y.L.: Using mutants to locate “unknown” faults. In: 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation (ICST), pp. 691–700, April 2012Google Scholar
  13. 13.
    Papadakis, M., Le Traon, Y.: Effective fault localization via mutation analysis: a selective mutation approach. In: Proceedings of the 29th Annual ACM Symposium on Applied Computing, SAC 2014, pp. 1293–1300. ACM, New York (2014)Google Scholar
  14. 14.
    Pearson, S., et al.: Evaluating and improving fault localization. In: Proceedings of the 39th International Conference on Software Engineering, pp. 609–620. IEEE Press, Piscataway (2017).  https://doi.org/10.1109/ICSE.2017.62
  15. 15.
    Sohn, J., Yoo, S.: FLUCCS: Using code and change metrics to improve fault localization. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2017, pp. 273–283. ACM, New York (2017).  https://doi.org/10.1145/3092703.3092717
  16. 16.
    Wong, E., Wei, T., Qi, Y., Zhao, L.: A crosstab-based statistical method for effective fault localization. In: 2008 1st International Conference on Software Testing, Verification, and Validation, pp. 42–51, April 2008.  https://doi.org/10.1109/ICST.2008.65
  17. 17.
    Wong, W.E., Debroy, V., Gao, R., Li, Y.: The dstar method for effective software fault localization. IEEE Trans. Reliability 63(1), 290–308 (2014).  https://doi.org/10.1109/TR.2013.2285319CrossRefGoogle Scholar
  18. 18.
    Wong, W.E., Gao, R., Li, Y., Abreu, R., Wotawa, F.: A survey on software fault localization. IEEE Trans. Softw. Eng. 42(8), 707–740 (2016)CrossRefGoogle Scholar
  19. 19.
    Yoo, S., Harman, M.: Using hybrid algorithm for pareto efficient multi-objective test suite minimisation. J. Syst. Softw. 83(4), 689–701 (2010).  https://doi.org/10.1016/j.jss.2009.11.706CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Instituto de Informática (UFG)GoiâniaBrazil
  2. 2.Instituto Federal de Mato Grosso (IFMT)SorrisoBrazil
  3. 3.Instituto Federal de Goiás (IFG)GoiâniaBrazil
  4. 4.Universidade Federal de São Carlos - UFSCARSão CarlosBrazil

Personalised recommendations