Using Semantic Metrics to Predict Mutation Equivalence

  • Amani Ayad
  • Imen Marsit
  • Nazih Mohamed Omri
  • JiMeng Loh
  • Ali MiliEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1077)


Equivalent mutants are a major nuisance in mutation testing because they introduce a significant amount of bias. But weeding them out is difficult because it requires a detailed analysis of the source code of the base program and the mutant. In this paper we argue that for most applications, it is not necessary to identify equivalent mutants individually; rather it suffices to estimate their number. Also, we explore how we can estimate their number by a cursory/automatable analysis of the base program and the mutant generation policy.


Redundancy Mutant equivalence Mutant survival ratio Software metrics 



This research is partially supported by a grant to the last author from the (US) National Science Foundation, number DGE 1565478.


  1. 1.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011)CrossRefGoogle Scholar
  2. 2.
    Just, R., Jalali, D., Inozemtseva, L., Ernst, M.D., Holmes, R., Fraser, G.: Are mutants a valid substitute for real faults in software testing. In: Foundations of Software Engineering, Hong Kong, China (2014)Google Scholar
  3. 3.
    Andrews, J.H., Briand, L.C., Labiche, I.: Is mutation an appropriate tool for testing experiments. In: International Conference on Software Testing, St. Louis, MO, USA (2005)Google Scholar
  4. 4.
    Namin, A.S., Kakarla, S.: The Use of mutation in testing experiments and its sensitivity to external threats. In: ISSTA 2011, Toronto, Ontario, Canada (2011)Google Scholar
  5. 5.
    Yao, X., Harman, M., Jia, Y.: A study of equivalent and stubborn mutation operators using human analysis of equivalence. In: Proceedings, International Conference on Software Engineering, Hyderabad, India (2014)Google Scholar
  6. 6.
    Schuler, D., Zeller, A.: Covering and uncovering equivalent mutants. J. Softw. Test. Verification Reliab. 23(5), 353–374 (2012)CrossRefGoogle Scholar
  7. 7.
    Gruen, B.J., Schuler, D., Zeller, A.: The impact of equivalent mutants. In: MUTATION 2009. Denver CO., USA (2009)Google Scholar
  8. 8.
    Just, R., Ernst, M.D., Fraser, G.: Using state infection conditions to detect equivalent mutants and speed up mutation analysis. In: Dagstuhl Seminar 13021: Symbolic Methods in Testing, Wadern, Germany (2013)Google Scholar
  9. 9.
    Just, R., Ernst, M.D., Fraser, G.: Efficient mutation analysis by propagating and partitioning infected execution states. In: ISSTA 2014, San Jose, CA, USA (2014)Google Scholar
  10. 10.
    Wang, B., Xiong, Y., Shi, Y., Zhang, L., Hao, D.: Faster mutation analysis via equivalence modulo states. In: ISSTA 2017, Santa Barbara, CA, USA (2017)Google Scholar
  11. 11.
    Papadakis, M., Delamaro, M., Le Traon, Y.: Mitigating the effects of equivalent mutants with mutant classification strategies. Sci. Comput. Program. 95(12), 298–319 (2014)CrossRefGoogle Scholar
  12. 12.
    Papadakis, M., Kintis, M., Zhang, J., Jia, Y., Le Traon, Y., Harman, M.: Mutation testing advances: an analysis and survey. In: Advances in Computers (2019)Google Scholar
  13. 13.
    Budd, T.A., Angluin, D.: Two notions of correctness and their relation to testing. Acta Informatica 18(1), 31–45 (1982)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Offutt, J.A., Pan, J.: Automatically detecting equivalent mutants and infeasible paths. J. Softw. Test. Verification Reliab. 7(3), 164–192 (1997)Google Scholar
  15. 15.
    Voas, J., McGraw, G.: Software Fault Injection: Inoculating Programs Against Errors. Wiley, New York (1997)Google Scholar
  16. 16.
    Harman, M., Hierons, R., Danicic, S.: The relationship between program dependence and mutation analysis. In: MUTATION 2000, San Jose, CA, USA (2000)Google Scholar
  17. 17.
    Hierons, R.M., Harman, M., Danicic, S.: Using program slicing to assist in the detection of equivalent mutants. J. Softw. Test. Verification Reliab. 9(4), 233–262 (1999)CrossRefGoogle Scholar
  18. 18.
    Ellims, M., Ince, D.C., Petre, M.: The Csaw C mutation tool: initial results. In: MUTATION 2007, Windsor, UK (2007)Google Scholar
  19. 19.
    Howden, W.E.: Weak mutation testing and completeness of test sets. IEEE Trans. Softw. Eng. 8(4), 371–379 (1982)CrossRefGoogle Scholar
  20. 20.
    Schuler, D., Dallmaier, V., Zeller, A.: Efficient mutation testing by checking invariant violations. In: ISSTA 2009, Chicago, IL, USA (2009)Google Scholar
  21. 21.
    Ernst, M.D., Cockrell, J., Griswold, W.G., Notkin, D.: Dynamically discovering likely program invariants to support program evolution. IEEE Trans. Softw. Eng. 27(2), 99–123 (2001)CrossRefGoogle Scholar
  22. 22.
    Nica, S., Wotawa, F.: Using constraints for equivalent mutant detection. In: Workshop on Formal Methods in the Development of Software (2012)Google Scholar
  23. 23.
    Carvalho, L., Guimaraes, M.A., Fernandes, L., Al Hajjaji, M., Gheyi, R., Thuem, T.: Equivalent mutants in configurable systems: an empirical study. In: VAMOS 2018, Madrid, Spain (2018)Google Scholar
  24. 24.
    Kintis, M., Papadakis, M., Jia, Y., Malevris, N., Le Traon, Y., Harman, M.: Detecting trivial mutant equivalences via compiler optimizations. IEEE Trans. Softw. Eng. 44(4), 308–333 (2018)CrossRefGoogle Scholar
  25. 25.
    Marsit, I., Omri, M.N., Loh, J.M., Mili, A.: Impact of mutation operators on mutant equivalence. In: ICSOFT, Madrid, Spain (2018)Google Scholar
  26. 26.
    Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Laurent, T., Papadakis, M., Kintis, M., Henard, C., Le Traon, Y., Ventresque, A.: Assessing and improving the mutation testing practice of PIT. In: ICST, Vasteras, Sweden (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.New Jersey Institute of TechnologyNewarkUSA
  2. 2.University of MonastirMonastirTunisia

Personalised recommendations