Skip to main content

On the Use of Meta-Heuristic Algorithms for Automated Test Suite Generation in Software Testing

Part of the Studies in Systems, Decision and Control book series (SSDC,volume 344)

Abstract

There exists a dire need to automate the process of test suite generation to get the most optimal results as testing accounts for more than 40% of total cost. A solution consists of using meta-heuristic algorithms which iteratively improve the test data to reach the most optimized test suites. The goal of the study is to find the best suited algorithm to narrow down future research in the field of test automation and also provide issues on the design of new proposals. We focus on the performance evaluation of different major Meta-Heuristic Algorithms namely: Hill Climbing Algorithm (HCA), Particle Swarm Optimization (PSO), Firefly Algorithm (FA), Cuckoo Search Algorithm (CA), Bat Algorithm (BA) and Artificial Bee Colony Algorithm (ABC). Each algorithm is implemented to automatically generate test suites based on the program under test. Then, we develop a performance evaluation of each algorithm for five programs written in Java. The algorithms are compared using several process metrics (average time, best time, worst time) and also product metrics (path coverage & objective function values of the generated test suites). Results indicate ABC as the best suited algorithm as it gave the most optimal Test Suites in reasonable time. BA is the fastest one but produced less optimal results. FA is the slowest algorithm while CA, PSO and HCA perform in between. Some issues and strategies to create hybrid algorithms are discusses and pointed out.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-67163-1_6
  • Chapter length: 49 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   189.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-67163-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   249.99
Price excludes VAT (USA)
Hardcover Book
USD   249.99
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

References

  1. Banker, R.D., Datar, S.M., Kemerer, C.F., Zweig, D.: Software complexity and maintenance costs. Commun. ACM 36(11), 81–95 (1993)

    CrossRef  Google Scholar 

  2. Li, A., Zhang, Y.: Automatic generating all-path test data of a program based on pso. In: Software Engineering, 2009. WCSE’09. WRI World Congress on, vol. 4, pp. 189–193. IEEE (2009, May)

    Google Scholar 

  3. Skalak, D.B.: Prototype and feature selection by sampling and random mutation hill climbing algorithms. In: Machine Learning Proceedings 1994, pp. 293–301

    Google Scholar 

  4. Yang, X.-S., Deb, S.: Cuckoo search via Lévy flights. In: Nature & Biologically Inspired Computing, 2009. NaBIC 2009. World Congress on, pp. 210–214. IEEE (2009)

    Google Scholar 

  5. Yang, X.-S.: Firefly algorithm. In: Engineering Optimization, pp. 221–230 (2010)

    Google Scholar 

  6. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, Perth, Australia, 1942–1948. IEEE Service Center, Piscataway, NJ (1995)

    Google Scholar 

  7. Karaboga, D., Basturk, B.: A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J. Global Optim. 39(3), 459–471 (2007)

    MathSciNet  CrossRef  Google Scholar 

  8. Yang, X.-S.: A new metaheuristic bat-inspired algorithm. In: Nature inspired cooperative strategies for optimization (NICSO 2010), pp. 65–74. Springer, Berlin Heidelberg (2010)

    Google Scholar 

  9. Nguyen, D.C., Perini, A., Tonella, P.: A goal-oriented software testing methodology. Lect. Notes Comput. Sci. 4951, 58–72 (2008)

    CrossRef  Google Scholar 

  10. Ngo, M.N., Tan, H.B.K.: Heuristics-based infeasible path detection for dynamic test data generation. Inf. Softw. Technol. 50(7), 641–655 (2008)

    CrossRef  Google Scholar 

  11. Kanmani, S., Maragathavalli, P.: Search-based software test data generation using evolutionary testing techniques. Int. J. Softw. Eng. (IJSE) 1(5), 10–22 (2010)

    Google Scholar 

  12. Yang, X.S.: A new metaheuristic bat-inspired algorithm. In: Nature Inspired Cooperative Strategies for Optimization (NICSO 2010), pp. 65–74 (2010)

    Google Scholar 

  13. Mansouri, P., Asady, B., Gupta, N.: The combination of bisection method and artificial bee colony algorithm for solving hard fix point problems. Soft Comput. Models Ind. Environ. Appl., 33–41 (2013)

    Google Scholar 

  14. Sharma, C., Sabharwal, S., Sibal, R.: A survey on software testing techniques using genetic algorithm (2014). arXiv preprint arXiv:1411.1154

  15. Varshney, S., Mehrotra, M.: (2016). Search-based test data generator for data-flow dependencies using dominance concepts, branch distance and elitism. Arab. J. Sci. Eng. (Springer Science & Business Media BV), 41(3)

    Google Scholar 

  16. Panichella, A., Kifetew, F., Tonella, P.: Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets. IEEE Trans. Softw. Eng. (2017)

    Google Scholar 

  17. Kumar, S., Yadav, D.K., Khan, D.A.: A novel approach to automate test data generation for data flow testing based on hybrid adaptive PSO-GA algorithm. Int. J. Adv. Intell. Paradigms 9(2–3), 278–312 (2017)

    CrossRef  Google Scholar 

  18. Mann, M., Tomar, P., Sangwan, O.P.: Bio-inspired metaheuristics: evolving and prioritizing software test data. Appl. Intell. 1–16 (2017)

    Google Scholar 

  19. Khari, M., Kumar, P., Burgos, D., Crespo, R.G.: Optimized test suites for automated testing using different optimization techniques. Soft Comput., 1–12 (2017)

    Google Scholar 

  20. Malhotra, R., Khari, M.: Test suite optimization using mutated artificial bee colony. In: Proceedings of the International Conference on Advances in Communication, Network, and Computing, CNC, pp. 45–54. Elsevier (2014)

    Google Scholar 

  21. Khari, M., Kumar, P.: An extensive evaluation of search-based software testing: a review. Soft Comput., 1–14 (2017)

    Google Scholar 

  22. Lourenço, H.R., Martin, O.C., Stützle, T.: Iterated local search. In: Handbook of Metaheuristics, pp. 320–353. Springer, Boston, MA (2003)

    Google Scholar 

  23. Wang, H., Wang, D., Yang, S.: A memetic algorithm with adaptive hill climbing strategy for dynamic optimization problems. Soft. Comput. 13(8–9), 763–780 (2009)

    CrossRef  Google Scholar 

  24. Tsamardinos, I., Brown, L.E., Aliferis, C.F.: The max-min hill-climbing Bayesian network structure learning algorithm. Mach. Learn. 65(1), 31–78 (2006)

    CrossRef  Google Scholar 

  25. Lotem, A., Nakamura, H., Zahavi, A.: Rejection of cuckoo eggs in relation to host age: a possible evolutionary equilibrium. Behav. Ecol. 3(2), 128–132 (1992)

    CrossRef  Google Scholar 

  26. Rajabioun, R.: Cuckoo optimization algorithm. Applied soft computing 11(8), 5508–5518 (2011)

    Google Scholar 

  27. Basu, M., Chowdhury, A.: Cuckoo search algorithm for economic dispatch. Energy 60, 99–108 (2013)

    CrossRef  Google Scholar 

  28. Civicioglu, P., Besdok, E.: A conceptual comparison of the Cuckoo-search, particle swarm optimization, differential evolution and artificial bee colony algorithms. Artif. Intell. Rev. 39(4), 315–346 (2013)

    CrossRef  Google Scholar 

  29. Karaboga, D., Gorkemli, B., Ozturk, C., Karaboga, N.: A comprehensive survey: artificial bee colony (ABC) algorithm and applications. Artif. Intell. Rev. 42(1), 21–57 (2014)

    CrossRef  Google Scholar 

  30. Cheriguene, S., Azizi, N., Zemmal, N., Dey, N., Djellali, H., Farah, N.: Optimized tumor breast cancer classification using combining random subspace and static classifiers selection paradigms. In Applications of Intelligent Optimization in Biology and Medicine, pp. 289–307. Springer International Publishing (2016)

    Google Scholar 

  31. Yıldız, A.R.: An effective hybrid immune-hill climbing optimization approach for solving design and manufacturing optimization problems in industry. J. Mater. Process. Technol. 209(6), 2773–2780 (2009)

    CrossRef  Google Scholar 

  32. Coelho, L.S., Guerra, F., Batistela, N.J., Leite, J.V.: Multiobjective cuckoo search algorithm based on Duffing’s oscillator applied to Jiles-Atherton vector hysteresis parameters estimation. IEEE Trans. Magn. 49(5), 1745–1748 (2013)

    CrossRef  Google Scholar 

  33. Moravej, Z., Akhlaghi, A.: A novel approach based on cuckoo search for DG allocation in distribution network. Int. J. Electr. Power Energy Syst. 44(1), 672–679 (2013)

    CrossRef  Google Scholar 

  34. Sur, C., Shukla, A.: Discrete cuckoo search optimization algorithm for combinatorial optimization of vehicle route in graph based road network. In: Proceedings of the Third International Conference on Soft Computing for Problem Solving, pp. 307–320. Springer, New Delhi (2014)

    Google Scholar 

  35. Durgun, İ., Yildiz, A.R.: Structural design optimization of vehicle components using cuckoo search algorithm. Materials Testing 54(3), 185–188 (2012)

    CrossRef  Google Scholar 

  36. Gandomi, A.H., Yang, X.S., Alavi, A.H.: Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng. Comput. 29(1), 17–35 (2013)

    CrossRef  Google Scholar 

  37. Bulatović, R.R., Đorđević, S.R., Đorđević, V.S.: Cuckoo search algorithm: a metaheuristic approach to solving the problem of optimum synthesis of a six-bar double dwell linkage. Mech. Mach. Theory 61, 1–13 (2013)

    CrossRef  Google Scholar 

  38. Senthilnath, J., Das, V., Omkar, S. N., Mani, V.: Clustering using levy flight cuckoo search. In: Proceedings of Seventh International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA 2012), pp. 65–75. Springer, India (2013)

    Google Scholar 

  39. Kumar, R., Rajan, A., Talukdar, F.A., Dey, N., Santhi, V., Balas, V.E. Optimization of 5.5-GHz CMOS LNA parameters using firefly algorithm. Neural Comput. Appl. 28(12), 3765–3779 (2017)

    Google Scholar 

  40. Vimalarani, C., Subramanian, R., Sivanandam, S.N.: An enhanced PSO-based clustering energy optimization algorithm for wireless sensor network. Sci. World J. (2016)

    Google Scholar 

  41. Sousa, T., Silva, A., Neves, A.: Particle swarm based data mining algorithms for classification tasks. Parallel Comput. 30(5–6), 767–783 (2004)

    CrossRef  Google Scholar 

  42. Satapathy, S.C., Naik, A., Parvathi, K. A teaching learning based optimization based on orthogonal design for solvin (2013)

    Google Scholar 

  43. Singh, A.: An artificial bee colony algorithm for the leaf-constrained minimum spanning tree problem. Appl. Soft Comput. 9(2), 625–631 (2009)

    CrossRef  Google Scholar 

  44. Karaboga, N.: A new design method based on artificial bee colony algorithm for digital IIR filters. J. Franklin Inst. 346(4), 328–348 (2009)

    MathSciNet  CrossRef  Google Scholar 

  45. Sharma, H., Bansal, J.C., Arya, K.V., Yang, X.S.: Lévy flight artificial bee colony algorithm. Int. J. Syst. Sci. 47(11), 2652–2670 (2016)

    CrossRef  Google Scholar 

  46. Elbeltagi, E., Hegazy, T., Grierson, D.: Comparison among five evolutionary-based optimization algorithms. Adv. Eng. Inform. 19(1), 43–53 (2005)

    CrossRef  Google Scholar 

  47. Nidhra, S., Dondeti, J.: Black box and white box testing techniques-a literature review. Int. J. Embed. Syst. Appl. (IJESA) 2(2), 29–50 (2012)

    Google Scholar 

  48. Watson, A.H., Wallace, D.R., McCabe, T.J.: Structured Testing: A Testing Methodology Using the Cyclomatic Complexity Metric (Vol. 500, No. 235). US Department of Commerce, Technology Administration, National Institute of Standards and Technology

    Google Scholar 

  49. Yan, J., Zhang, J.: An efficient method to generate feasible paths for basis path testing. Inf. Proces. Letters 107(3–4), 87–92 (2008)

    MathSciNet  CrossRef  Google Scholar 

  50. Zhonglin, Z., Lingxia, M. (2010, August). An improved method of acquiring basis path for software testing. In: Computer Science and Education (ICCSE), 2010 5th International Conference on, pp. 1891–1894. IEEE

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rubén González Crespo .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Khari, M., Sinha, A., Herrerra-Viedma, E., Crespo, R.G. (2021). On the Use of Meta-Heuristic Algorithms for Automated Test Suite Generation in Software Testing. In: Kahraman, C., Bolturk, E. (eds) Toward Humanoid Robots: The Role of Fuzzy Sets. Studies in Systems, Decision and Control, vol 344. Springer, Cham. https://doi.org/10.1007/978-3-030-67163-1_6

Download citation