Abstract
Fault localization is one of the most important and difficult tasks in the software debugging process. Therefore, several methods have been proposed to automate and improve this process. Mutation-based fault localization is one of the states of the art techniques that try to locate faults by executing different mutants of the faulty program. In addition to favorable results, it is along with a massive increase in mutation execution cost. In this paper, we propose a new mutation-based fault localization approach called SMBFL, that aim to reduce the execution cost by reducing the number of statements to be mutated. As fewer mutants execute with SMBFL, the whole process will become faster and the cost will decrease. SMBFL only examines the statements in the dynamic slice of the program under test. The statements that present in the dynamic slice have a direct effect on the execution of the program with the specified test case. In the SMBFL method, the suspiciousness score of program statements is measured based on the entropy of their mutants. The proposed formula, MuEn, determines the suspiciousness score based on the result of executing mutants of each statement of the program. SMBFL is evaluated during a series of tests. The results show a relative increase in the accuracy of fault localization, by an average of 14.2%, and a decrease in the execution time of the fault localization process, by an average of 24.3%. Finally, the MuEn formula applies the least execution overhead to the fault localization process.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abreu R, Zoeteweij P, Golsteijn R, van Gemund AJC (2009) A practical evaluation of spectrum-based fault localization. J Syst Softw 82:1780–1792. https://doi.org/10.1016/j.jss.2009.06.035
Bellamy J, Liebowitz J (1991) Elements of information theory expert system applications to telecommunications. Wiley-Interscience, New York
Campos J, Arcuri A, Fraser G, Abreu R (2014) Continuous test generation: enhancing continuous integration with automated test generation. Proc - 29th ACM/IEEE Int Conf autom Softw Eng, ASE, Vasteras, pp 55–66. doi: https://doi.org/10.1145/2642937.2643002
Cao H, Jiang S, Ju X et al (2014) Applying association analysis to dynamic slicing based fault localization. IEICE Trans Inf Syst 97:2057–2066. https://doi.org/10.1587/transinf.E97.D.2057
Cazorla FJ, Vardanega T, Quiñones E, Abella J (2013) Upper-bounding program execution time with extreme value theory. OpenAccess Ser Informatics 30:64–76. https://doi.org/10.4230/OASIcs.WCET.2013.64
Danglot B, Vera-Pérez OL, Baudry B, Monperrus M (2019) Automatic test improvement with DSpot: a study with ten mature open-source projects. Empir Softw Eng 24:2603–2635. https://doi.org/10.1007/s10664-019-09692-y
Deng L, Offutt J, Li N (2013) Empirical evaluation of the statement deletion mutation operator. Proc - IEEE 6th Int Conf Softw testing, Verif validation, ICST, Luxembourg, pp 84–93. doi: https://doi.org/10.1109/ICST.2013.20
DiGiuseppe N, Jones JA (2015) Fault density, fault types, and spectra-based fault localization. Empir Softw Eng 20:928–967. https://doi.org/10.1007/s10664-014-9304-1
Do H, Elbaum S, Rothermel G (2005) Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empir Softw Eng 10:405–435. https://doi.org/10.1007/s10664-005-3861-2
Feyzi F, Parsa S (2018) A program slicing-based method for effective detection of coincidentally correct test cases. Computing 100:927–969. https://doi.org/10.1007/s00607-018-0591-z
Feyzi F, Parsa S (2019) Inforence: effective fault localization based on information-theoretic analysis and statistical causal inference. Front Comput Sci 13:735–759. https://doi.org/10.1007/s11704-017-6512-z
Gong P, Zhao R, Li Z (2015) Faster mutation-based fault localization with a novel mutation execution strategy. Proc - IEEE 8th Int Conf Softw testing, Verif valid work, ICSTW, Graz, pp 1–10. doi: https://doi.org/10.1109/ICSTW.2015.7107448
Gyimóthy T, Beszédes Á, Forgács I (1999) An efficient relevant slicing method for debugging. Proc - 7th Eur Softw Eng Conf held jointly with 7th ACM SIGSOFT Int Symp found Softw Eng, ESEC/FSE, Toulouse, pp 303–321
Halabi D, Shaout A (2016) Mutation testing tools for Java programs – a survey. IJCSE Int J Comput Sci Eng 5:11–22
Hammacher C, Streit K, Hack S, Zeller A (2009) Profiling java programs for parallelism. Proc - ICSE Work Multicore Softw Eng, Vancouver, pp 49–55. https://doi.org/10.1109/IWMSE.2009.5071383
Just R, Jalali D, Ernst MD (2014) Defects4J: a database of existing faults to enable controlled testing studies for Java programs. Proc - Int Symp Softw Test Anal, ISSTA, San Jose, pp 437–440
Kapoor K, Bowen JP (2005) Ordering mutants to minimise test effort in mutation testing. Lect Notes Comput Sci 3395:195–209
Khatiwada S, Tushev M, Mahmoud A (2018) Just enough semantics: An information theoretic approach for IR-based software bug localization. Inf Softw Technol 93:45–57. https://doi.org/10.1016/j.infsof.2017.08.012
Kim J, An G, Feldt R, Yoo S (2019) Amortising the cost of mutation based fault localisation using statistical inference, CoRR abs/1902.09729
Korel B, Laski J (1988) Dynamic program slicing. Inf Process Lett 29:155–163
Le TDB, Thung F, Lo D (2017) Will this localization tool be effective for this bug? Mitigating the impact of unreliability of information retrieval based bug localization tools. Empir Softw Eng 22:2237–2279. https://doi.org/10.1007/s10664-016-9484-y
Lei Y, Mao X, Chen TY (2013) Backward-slice-based statistical fault localization without test oracles. Proc - IEEE Int Conf Qual Softw, Najing, pp 212–221. https://doi.org/10.1109/QSIC.2013.45
Lei Y, Mao X, Dai Z, Wang C (2012) Effective statistical fault localization using program slices. Proc - Int Comput Softw Appl Conf, Izmir, pp 1–10. https://doi.org/10.1109/COMPSAC.2012.9
Li X, Wong WE, Gao R et al (2018a) Genetic algorithm-based test generation for software product line with the integration of fault localization techniques. Empir Softw Eng 23:1–51. https://doi.org/10.1007/s10664-016-9494-9
Li Z, Yan L, Liu Y, et al (2018b) MURE: making use of MUtations to REfine Spectrum-based fault localization. Proc - IEEE 18th Int Conf Softw Qual Reliab Secur companion, QRS-C, Lisbon, pp 56–63.doi: https://doi.org/10.1109/QRS-C.2018.00024
Liu B, Nejati S, Lucia BLC (2019) Effective fault localization of automotive Simulink models: achieving the trade-off between test oracle effort and fault localization accuracy. Empir Softw Eng 24:444–490. https://doi.org/10.1007/s10664-018-9611-z
Liu Y, Li Z, Wang L et al (2017) Statement-oriented mutant reduction strategy for mutation based fault localization. Proc - IEEE Int Conf Softw Qual Reliab Secur, QRS, Prague, pp 126–137. https://doi.org/10.1109/QRS.2017.23
Liu Y, Li Z, Zhao R, Gong P (2018) An optimal mutation execution strategy for cost reduction of mutation-based fault localization. Inf Sci 422:572–596. https://doi.org/10.1016/j.ins.2017.09.006
Lobo de Oliveira AA, Goncalves Camilo-Junior C, Noronha de Andrade Freitas E, Rizzo Vincenzi AM (2018) FTMES: A Failed-Test-Oriented Mutant Execution Strategy for Mutation-Based Fault Localization. Proc - IEEE 29th Int Symp Softw Reliab Eng, ISSRE, Memphis, pp 155–165
Lu HL, Gao R, Huang SK, Wong WE (2017) Spectrum-Base fault localization by exploiting the failure path. Proc - Int Comput Symp, ICS, Chiayi, pp 252–257
Mao X, Lei Y, Dai Z et al (2014) Slice-based statistical fault localization. J Syst Softw 89:51–62. https://doi.org/10.1016/j.jss.2013.08.031
Mateo PR, Usaola MP (2014) Reducing mutation costs through uncovered mutants. Softw Testing, Verif Reliab 25:464–489. https://doi.org/10.1002/stvr.1534
Moon S, Kim Y, Kim M, Yoo S (2014) Ask the mutants: mutating faulty programs for fault localization. Proc - IEEE 7th Int Conf Softw testing, Verif validation, ICST, Cleveland, pp 153–162. doi: https://doi.org/10.1109/ICST.2014.28
Mvnrepository (2007) Maven Repository: jaxen » jaxen » 1.1.1. In: Mvnrepository.com. https://mvnrepository.com/artifact/jaxen/jaxen/1.1.1. Accessed 5 May 2007
Namin AS, Andrews JH, Murdoch DJ (2008) Sufficient mutation operators for measuring test effectiveness. Proc - Int Conf Softw Eng, Leipzig, pp 351–360. https://doi.org/10.1145/1368088.1368136
Papadakis M, Kintis M, Zhang J et al (2019) Mutation testing advances: An analysis and survey. Adv Comput 112:275–378. https://doi.org/10.1016/bs.adcom.2018.03.015
Papadakis M, Le Traon Y (2014) Effective fault localization via mutation analysis: a seletive mutation approach. Proc - 29th Annu ACM Symp Appl Comput, SAC, Gyeongju, pp 1293–1300. doi: https://doi.org/10.1145/2554850.2554978
Papadakis M, Taron Y (2015) Metallaxis-FL: mutation-based fault localization Mike. Softw Test Verif Reliab 25:605–628. https://doi.org/10.1002/stvr
Reis S, Abreu R, D’Amorim M (2019) Demystifying the combination of dynamic slicing and spectrum-based fault localization. Proc - Int Jt Conf Artif Intell, IJCAI, Macao, pp 4760–4766. Doi: https://doi.org/10.24963/ijcai.2019/661
Silva J (2012) A vocabulary of program slicing-based techniques. ACM Comput Surv 44:1–12. https://doi.org/10.1145/2187671.2187674
Sohn J, Yoo S (2017) FLUCCS: using code and change metrics to improve fault localization. Proc - 26th ACM SIGSOFT Int Symp Softw test anal, ISSTA, Santa Barbara, pp 273–283. doi: https://doi.org/10.1145/3092703.3092717
Steimann F, Frenkel M, Abreu R (2013) Threats to the validity and value of empirical assessments of the accuracy of coverage-based fault locators. Proc - Int Symp Softw Test Anal, ISSTA, Lugano, pp 314–324. https://doi.org/10.1145/2483760.2483767
Tahchiev P, Leme F, Massol V, Gregory G (2010) JUnit in action, 2nd edn. Manning, Greenwich
Weiser M (1981) Program slicing. Proc - 5th Int Conf Softw Eng, Piscataway, pp 439-449
Xie X, Wong WE, Chen TY, Xu B (2013) Metamorphic slice: An application in spectrum-based fault localization. Inf Softw Technol 55:866–879. https://doi.org/10.1016/j.infsof.2012.08.008
Yoo S, Harman M, Clark D (2013) Fault localization prioritization : comparing information theoretic and coverage based approaches. ACM Trans Softw Eng Methodol 22:1–29. https://doi.org/10.1145/2491509.2491513
Zakari A, Lee SP, Abreu R et al (2020) Multiple fault localization of software programs: a systematic literature review. Inf Softw Technol 124:106312. https://doi.org/10.1016/j.infsof.2020.106312
Zhang M, Li X, Zhang L, Khurshid S (2017) Boosting spectrum-based fault localization using PageRank. Proc - 26th ACM SIGSOFT Int Symp Softw test anal, ISSTA, Santa Barbara, pp 261–272. doi: https://doi.org/10.1145/3092703.3092731
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Paolo Tonella
Publisher’s Note
Springer Nature remains neutral with regard tojurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Bayati Chaleshtari, N., Parsa, S. SMBFL: slice-based cost reduction of mutation-based fault localization. Empir Software Eng 25, 4282–4314 (2020). https://doi.org/10.1007/s10664-020-09845-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-020-09845-4