Abstract
The research focus in the field of combinatorial testing has shifted from unconstrained covering array generation (CAG) to constrained covering array generation (CCAG). Most heuristic search algorithms exclude all invalid solutions from the search space so that all intermediate and final solutions satisfy the constraints. However, some values of intermediate solutions that are sometimes invalid may help in finding the optimal solution. Therefore, such solutions may need to be "tolerated" in the generation process. This paper proposes a tolerance-based memetic algorithm named QSMA that employs quantum particle swarm optimization (QPSO) as a global searching operator and improved simulated annealing as a local searching operator to balance exploration and exploitation synergistically. Meanwhile, QSMA incorporates a stochastic ranking strategy to address the challenge of setting the penalty coefficient. Besides, a specific penalty function is proposed to deal with constraints, and design guidelines for penalty functions are suggested. In the experiment, the impacts of parameter settings on the performance of QSMA are investigated. Extensive experimental results show that the QSMA algorithm outperforms other methods and tools for both CAG and CCAG problems.
Similar content being viewed by others
Data availability
Data are available on request to the authors.
Notes
References
Sheng Y (2013) Development of testing platform of embedded software based on combinatorial testing. Ph.D. thesis, Harbin Institute of Technology
Kuhn DR, Reilly MJ (2003) An investigation of the applicability of design of experiments to software testing. In: 27th Annual NASA Goddard/IEEE software engineering workshop, 2002. Proceedings
Nie C, Leung H (2011) A survey of combinatorial testing. ACM Comput Surv 43(2):1–29
Wu H, Nie C, Petke J, Jia Y, Harman M (2019) A survey of constrained combinatorial testing. arXiv preprint arXiv:1908.02480
Ahmed BS, Zamli KZ, Afzal W, Bures M (2017) Constrained interaction testing: a systematic literature study. IEEE Access 5:25706–25730
Wu H, Nie C, Petke J, Jia Y, Harman M (2019) Comparative analysis of constraint handling techniques for constrained combinatorial testing. IEEE Trans Softw Eng 47(11):2549–2562
Guo X, Song X, Zhou J (2022) A synergic quantum particle swarm optimisation for constrained combinatorial test generation. IET Softw 16(3):279–300
Adam SP, Alexandropoulos S-AN, Pardalos PM, Vrahatis MN (2019) No free lunch theorem: a review. Approx Optim 57–82
Kimbrough SO, Koehler GJ, Lu M, Wood DH (2008) On a feasible-infeasible two-population (fi-2pop) genetic algorithm for constrained optimization: distance tracing and no free lunch. Eur J Oper Res 190(2):310–327
Galinier P, Kpodjedo S, Antoniol G (2017) A penalty-based tabu search for constrained covering arrays. In: Proceedings of the genetic and evolutionary computation conference, pp 1288–1294
Wang Y, Wu H, Niu X, Nie C, Xu J (2022) An adaptive penalty based parallel tabu search for constrained covering array generation. Inf Softw Technol 143:106768
Runarsson TP, Yao X (2000) Stochastic ranking for constrained evolutionary optimization. IEEE Trans Evol Comput 4(3):284–294
Avila-George H, Torres-Jimenez J, Gonzalez-Hernandez L, Hernández V (2013) Metaheuristic approach for constructing functional test-suites. IET Softw 7(2):104–117
Jia Y, Cohen MB, Harman M, Petke J (2015) Learning combinatorial interaction test generation strategies using hyperheuristic search. In: 2015 IEEE/ACM 37th IEEE international conference on software engineering, vol. 1, pp 540–550. IEEE
Garvin BJ, Cohen MB, Dwyer MB (2009) An improved meta-heuristic search for constrained interaction testing. In: 2009 1st International symposium on search based software engineering, pp 13–22. IEEE
Esfandyari S, Rafe V (2018) A tuned version of genetic algorithm for efficient test suite generation in interactive t-way testing strategy. Inf Softw Technol 94:165–185
Alazzawi AK, Rais HM, Basri S, Alsariera YA, Capretz LF, Balogun AO, Imam AA (2021) Habcsm: a hamming based t-way strategy based on hybrid artificial bee colony for variable strength test sets generation. arXiv preprint arXiv:2110.03728
Lin J, Luo C, Cai S, Su K, Hao D, Zhang L (2015) TCA: an efficient two-mode meta-heuristic algorithm for combinatorial test generation (t). In: 2015 30th IEEE/ACM international conference on automated software engineering (ASE), pp 494–505. IEEE
Lin J, Cai S, Luo C, Lin Q, Zhang H (2019) Towards more efficient meta-heuristic algorithms for combinatorial test generation. In: Proceedings of the 2019 27th ACM joint meeting on European Software engineering conference and symposium on the foundations of software engineering, pp 212–222
Luo C, Lin J, Cai S, Chen X, He B, Qiao B, Zhao P, Lin Q, Zhang H, Wu W et al (2021) Autoccag: an automated approach to constrained covering array generation. In: 2021 IEEE/ACM 43rd international conference on software engineering (ICSE), pp 201–212. IEEE
Fu Y, Lei Z, Cai S, Lin J, Wang H (2020) WCA: a weighting local search for constrained combinatorial test optimization. Inf Softw Technol 122:106288
Wu H, Nie C, Kuo F-C, Leung H, Colbourn CJ (2014) A discrete particle swarm optimization for covering array generation. IEEE Trans Evol Comput 19(4):575–591
Sheng Y, Wei C, Jiang S (2017) Constraint test cases generation based on particle swarm optimization. Int J Reliab Qual Saf Eng 24(05):1750021
Wagner M, Kampel L, Simos DE (2019) Quantum-inspired evolutionary algorithms for covering arrays of arbitrary strength. In: International symposium on experimental algorithms, pp 300–316. Springer
Cai X, Zhao H, Shang S, Zhou Y, Deng W, Chen H, Deng W (2021) An improved quantum-inspired cooperative co-evolution algorithm with muli-strategy and its application. Expert Syst Appl 171:114629
Coello CAC (2002) Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art. Comput Methods Appl Mech Eng 191(11–12):1245–1287
Ahmed BS, Gambardella LM, Afzal W, Zamli KZ (2017) Handling constraints in combinatorial interaction testing in the presence of multi objective particle swarm and multithreading. Inf Softw Technol 86:20–36
Li Y, Sun Z-A, Fang J-Y (2016) Generating an automated test suite by variable strength combinatorial testing for web services. J Comput Inf Technol 24(3):271–282
Huang S, Cohen MB, Memon AM (2010) Repairing gui test suites using a genetic algorithm. In: 2010 Third international conference on software testing, verification and validation, pp 245–254. IEEE
Guo X, Song X, Zhou J-t (2018) Effective discrete memetic algorithms for covering array generation. In: 2018 IEEE 42nd annual computer software and applications conference (COMPSAC), vol. 1, pp 298–303. IEEE
Sun J, Fang W, Wu X, Palade V, Xu W (2012) Quantum-behaved particle swarm optimization: analysis of individual particle behavior and parameter selection. Evol Comput 20(3):349–393
Sun J, Xu W, Liu J (2005) Parameter selection of quantum-behaved particle swarm optimization. In: International conference on natural computation, pp 543–552. Springer
Chen X, Gu Q, Wang Z-Y, Chen D-X (2011) Framework of particle swarm optimization based pairwise testing. J Softw 22(12):2879–2893
Garvin BJ, Cohen MB, Dwyer MB (2011) Evaluating improvements to a meta-heuristic search for constrained interaction testing. Empir Softw Eng 16(1):61-102
Alazzawi AK, Rais HM, Basri S (2019) Abcvs: An artificial bee colony for generating variable t-way test sets. Int J Adv Comput Sci Appl 10(4)
Ahmed BS, Zamli KZ, Lim CP (2012) Application of particle swarm optimization to uniform and variable strength covering array construction. Appl Soft Comput 12(4):1330–1347
Alsewari ARA, Zamli KZ (2012) Design and implementation of a harmony-search-based variable-strength t-way testing strategy with constraints support. Inf Softw Technol 54(6):553–568
Wagner M, Kleine K, Simos DE, Kuhn R, Kacker R (2020) Cagen: a fast combinatorial test generation tool with support for constraints and higher-index arrays. In: 2020 IEEE international conference on software testing, verification and validation workshops (ICSTW), pp 191–200. IEEE
Acknowledgements
This work is partially supported by the National Natural Science Foundation of China under Grant No. 62162046, the Inner Mongolia Science and Technology Project under Grant No. 2021GG0155, the Natural Science Foundation of Major Research Plan of Inner Mongolia under Grant No. 2019ZD15, and the Inner Mongolia Natural Science Foundation under Grant No. 2019GG372.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Guo, X., Song, X., Zhou, Jt. et al. A tolerance-based memetic algorithm for constrained covering array generation. Memetic Comp. 15, 319–340 (2023). https://doi.org/10.1007/s12293-023-00392-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12293-023-00392-1