Skip to main content
SpringerLink
Log in

A Source-code Aware Method for Software Mutation Testing Using Artificial Bee Colony Algorithm

  • Published:
Journal of Electronic Testing Aims and scope Submit manuscript

Abstract

The effectiveness of software test data relates to the number of found faults by the test data. Software mutation test is used to evaluate the effectiveness of the software test methods and is one of the challenging fields of software engineering. In order to evaluate the capability of test data in finding the program faults, some syntactical changes are made in the program source code to cause faulty program; then, the generated mutants (faulty programs) and original program are executing with the corresponding test data. One of the main drawbacks of mutation testing is its computational cost. Indeed, high execution time of mutation testing is a challenging research problem. Reducing the time and cost of mutation test is the main objective of this paper. In the traditional mutation methods and tools the mutants are injected randomly in each instructions of a program. Meanwhile, in the real-world program, the probability of fault occurrences in the simple locations (instructions and data) of a program is negligible. With respect to the 80–20 rule, 80% of the faults are found in 20% of the fault-prone code of a program. In the first stage of the proposed method, Artificial Bee Colony optimization algorithm is used to identifying the most fault prone paths of a program; in the next stage, the mutation operators (faults) are injected only on the identified fault-prone instructions and data. Regarding the results of conducted experiments on the standard benchmark programs, Compared to existing methods, the proposed method reduces 28.10% of the generated mutants. Reducing the number of generated mutants will reduce the cost of mutation testing. The traditional mutation testing tools (Mujava, Muclipse, Jester, Jumble) can perform the mutation testing with a lower cost using the method presented in this study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The datasets generated during and the implemented code during the current study is available in the google.drive can be freely accessed by the following link: https://drive.google.com/drive/folders/1XFId09ZM88thDHCRWNoTkJWNBiokk-S_?usp=sharing.

References

  1. Acree A, Budd T, DeMillo R, Lipton R, Sayward F (1980) Mutation Analysis. School of Information and Computer Science, Georgia Institute of Technology

    Google Scholar 

  2. Aghdam ZK, Arasteh B (2017) An efficient method to generate test data for software structural testing using artificial bee colony optimization algorithm. Int J Softw Eng Knowl Eng 27(6):2017

    Article  Google Scholar 

  3. Arasteh B (2019) ReDup: A software-based method for detecting soft-error using data analysis. Comput Electr Eng 78:89–107

    Article  Google Scholar 

  4. Arasteh B, Hosseini SMJ (2022) Traxtor: An Automatic Software Test Suit Generation Method Inspired by Imperialist Competitive Optimization Algorithms. J Electron Test. https://doi.org/10.1007/s10836-022-05999-9

    Article  Google Scholar 

  5. Barbosa EF, Maldonado JC, Vincenzi AMR (2001) Toward the determination of sufficient mutant operators for C. Softw Test Verification Reliab 11(2):113–136

    Article  Google Scholar 

  6. Budd TA (1980) Mutation Analysis of Program Test Data. Yale University

    Google Scholar 

  7. Cutigi F, Viola Pizzoleto A, Offutt J (2018) A Systematic Review of Cost Reduction Techniques for Mutation Testing: Preliminary Results. In: Proc. IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp 1–10. https://doi.org/10.1109/ICSTW.2018.00021

  8. Delgado-Pérez P, Medina-Bulo I (2018) Search-based mutant selection for efficient test suite improvement: Evaluation and results. Inf Softw Technol 104(2018):130–143

    Article  Google Scholar 

  9. Deng L, Offutt J, Ammann P, Mirzaei N (2017) Mutation operators for testing Android apps. Inf Softw Technol 81(2017):154–168

    Article  Google Scholar 

  10. Dominguez-Jimenez JJ, Estero-Botaro A, Garcia-Dominguez A, Medina-Bulo I (2011) Evolutionary mutation testing. Inf Softw Technol 53(10):1108–1123

    Article  Google Scholar 

  11. Fenton NE, Ohlsson N (2000) Quantitative analysis of faults and failures in a complex software system. IEEE Trans Softw Eng 26(8):797–814

    Article  Google Scholar 

  12. Ghaemi A, Arasteh B (2020) SFLA-based heuristic method to generate software structural test data. J Softw Evol Proc 32:e2228. https://doi.org/10.1002/smr.2228

    Article  Google Scholar 

  13. Gharehpasha S, Masdari M, Jafarian A (2021) Power efficient virtual machine placement in cloud data centers with a discrete and chaotic hybrid optimization algorithm. Cluster Comput 24:1293–1315. https://doi.org/10.1007/s10586-020-03187-y

    Article  Google Scholar 

  14. Gheyi R, Ribeiro M, Souza B, Guimarães M, Fernandes L, d’Amorim M, Alves V, Teixeira L, Fonseca B (2021) (2021), Identifying method-level mutation subsumption relations using Z3. Inf Softw Technol 132:106496

    Article  Google Scholar 

  15. Hosseini S, Arasteh B, Isazadeh A, Mohsenzadeh M, Mirzarezaee M (2021) An error-propagation aware method to reduce the software mutation cost using genetic algorithm. Data Technol Appl 55(1):118–148. https://doi.org/10.1108/DTA-03-2020-0073

    Article  Google Scholar 

  16. Howden WE (1982) Weak mutation testing and completeness of test sets. IEEE Trans Softw Eng 8(4):371–379

    Article  Google Scholar 

  17. Jafarian T, Masdari M, Ghaffari A et al (2021) A survey and classification of the security anomaly detection mechanisms in software defined networks. Cluster Comput 24:1235–1253. https://doi.org/10.1007/s10586-020-03184-1

    Article  Google Scholar 

  18. Karaboga D, Basturk B (2007) Artificial bee colony (ABC) optimization algorithm for solving constrained optimization problems. Advances in Soft Computing: Foundations of Fuzzy Logic and Soft Computing, vol 4529. Springer, Berlin, pp 789–798

    Chapter  Google Scholar 

  19. King KN, Offutt AJ (1991) A Fortran language system for mutation-based software testing. Softw Pract Exper 21(7):685–718

    Article  Google Scholar 

  20. Kintis M, Papadakis M, Malevris N (2010) Evaluating mutation testing alternatives: a collateral experiment. In: Proceedings of the 17th Asia-Pacific Software Engineering Conference (APSEC)

  21. Kurtz B, Ammann P, Delamaro M, Offutt J, Deng L (2014) Mutant subsumption graphs. In: 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

  22. Kurtz B, Ammann P, Offutt J (2015) Static analysis of mutant subsumption. In: IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

  23. Ma YS, Offutt J, Kwon YR (2006) MuJava: A Mutation System for Java. In: 28th International Conference on Software Engineering (ICSE ’06)

  24. Malevris N, Yates D (2006) The collateral coverage of data flow criteria when branch testing. Inf Softw Technol 48(8):676–686

    Article  Google Scholar 

  25. Masdari M, Khezri H (2020) Efficient VM migrations using forecasting techniques in cloud computing: a comprehensive review. Cluster Comput 23:2629–2658. https://doi.org/10.1007/s10586-019-03032-x

    Article  Google Scholar 

  26. Offutt AJ, Lee A, Rothermel G, Untch RH, Zapf C (1996) An experimental determination of sufficient mutant operators. ACM Trans Softw Eng Methodol 5(2):99–118

    Article  Google Scholar 

  27. Offutt AJ, Rothermel G, Zapf C (1993) An experimental evaluation of selective mutation. In: Proceedings of the 15th International Conference on Software Engineering, ICSE ’93. IEEE Computer Society Press, Los Alamitos, CA

  28. Papadakis M, Malevris N (2010) An empirical evaluation of the first and second order mutation testing strategies. In: 2010 Third International Conference on Software Testing, Verification, and Validation Workshops (ICSTW)

  29. Wei C, Yao X, Gong D, Liu H (2021) Spectral clustering based mutant reduction for mutation testing. Inf Softw Technol 132:106502

    Article  Google Scholar 

  30. Wong WE (1993) On mutation and data flow. Purdue University (Ph.D. dissertation)

    Google Scholar 

  31. Woodward M, Halewood K (1998) From weak to strong, dead or alive? An analysis of some mutation testing issues. In: Proceedings of the Second Workshop on Software Testing, Verification, and Analysis

  32. Zhang L, Gligoric M, Marinov D, Khurshid S (2013) Operator-based and random mutant selection: better together. In: Proc. IEEE/ACM 28th International Conference on Automated Software Engineering (ASE)

  33. Zhang L, Hou S-S, Hu J-J, Xie T, Mei H (2010) Is operator-based mutant selection superior to random mutant selection? In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering

Download references

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Department of Software Engineering, Faculty of Engineering and Natural Science, Istinye University, Istanbul, Turkey

    Bahman Arasteh & Keyvan Arasteh

  2. Department of Computer Engineering, Tabriz Branch, Islamic Azad University, Tabriz, Iran

    Parisa Imanzadeh

  3. Department of Computer Engineering, Urmia Branch, Islamic Azad University, Urmia, Iran

    Farhad Soleimanian Gharehchopogh

  4. Department of Computer Engineering, Shabestar Branch, Islamic Azad University, Shabestar, Iran

    Bagher Zarei

Authors
  1. Bahman Arasteh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Parisa Imanzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Keyvan Arasteh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Farhad Soleimanian Gharehchopogh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bagher Zarei
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Thesis statement, data collection and analysis were performed by Bahman Arasteh. Experiments have been performed by Bahman Arasteh, Parisa Imanzadeh, Keyvan Arasteh, Farhad Soleimanian Gharehchopogh and Bagher Zarei and also, the first draft of the manuscript was written by Parisa Imanzadeh. All authors commented on previous versions of the manuscript. Bahman Arasteh read and approved the final manuscript.

Corresponding author

Correspondence to Bahman Arasteh.

Ethics declarations

Conflicts of Interests

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Responsible Editor: V. D. Agrawal

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arasteh, B., Imanzadeh, P., Arasteh, K. et al. A Source-code Aware Method for Software Mutation Testing Using Artificial Bee Colony Algorithm. J Electron Test 38, 289–302 (2022). https://doi.org/10.1007/s10836-022-06008-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10836-022-06008-9

Keywords

Search

Navigation