Abstract
Test Case Prioritization reduces the cost of software testing by executing earlier the subset of test cases showing higher priorities. The methodology consists of ranking test cases so that, in case of a limited budget, only the top-ranked tests are exercised. One possible direction for prioritizing test cases relies on considering the usage frequency of a software sub-system. To this end, a promising direction is to identify the likelihood of events occurring in software systems, and this can be achieved by adopting Markov chains. This paper presents a novel approach that analyzes the system scenarios modeled as a Markov chain and ranks the generated test sequences to prioritize test cases. To assess the proposed approach, we developed an algorithm and conducted a preliminary and experimental study that investigates the feasibility of using Markov chains as an appropriate means to prioritize test cases. We demonstrate the strength of the novel strategy by evaluating two heuristics, namely H1 (based on the transition probabilities) and H2 (based on the steady-state probabilities), with established metrics. Results show (i) coverage of 100% for both H1 and H2, and (ii) efficiency equal to 98.4% for H1 and 99.4% for H2, on average.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
SATM: Simple Automatic Teller Machine. https://slideplayer.com/slide/3835819/. Accessed 4 Oct 2022
Barbosa, G., de Souza, É.F., dos Santos, L.B.R., da Silva, M., Balera, J.M., Vijaykumar, N.L.: A systematic literature review on prioritizing software test cases using Markov chains. Inf. Softw. Technol. 147, 106902 (2022)
Bertolino, A., Miranda, B., Pietrantuono, R., Russo, S.: Adaptive coverage and operational profile-based testing for reliability improvement. In: International Conference on Software Engineering (ICSE), pp. 541–551 (2017)
Bohme, M., Pham, V.T., Roychoudhury, A.: Coverage-based Greybox fuzzing as Markov chain. IEEE Trans. Software Eng. 45(5), 489–506 (2019)
Bolch, G., Greiner, S., De Meer, H., Trivedi, K.S.: Queueing Networks and Markov Chains: Modeling and Performance Evaluation with Computer Science Applications. John Wiley & Sons (2006)
Brémaud, P.: Markov Chains: Gibbs fields, Monte Carlo Simulation, and Queues, vol. 31. Springer, New York (2013). https://doi.org/10.1007/978-3-030-45982-6
Cai, K.Y.: Optimal software testing and adaptive software testing in the context of software cybernetics. Inf. Softw. Technol. 44(14), 841–855 (2002)
Camilli, M., Gargantini, A., Scandurra, P., Trubiani, C.: Uncertainty-aware exploration in model-based testing. In: IEEE Conference on Software Testing, Verification and Validation (ICST), pp. 71–81 (2021)
Cruciani, E., Miranda, B., Verdecchia, R., Bertolino, A.: Scalable approaches for test suite reduction. In: Proceedings of the International Conference on Software Engineering (ICSE), pp. 419–429 (2019)
Devroey, X., et al.: Statistical prioritization for software product line testing: an experience report. Softw. Syst. Model. 16(1), 153–171 (2015)
Elbaum, S., Malishevsky, A.G., Rothermel, G.: Test case prioritization: a family of empirical studies. IEEE Trans. Software Eng. 28(2), 159–182 (2002)
Everett, G.D., McLeod Jr., R.: Software Testing. Testing Across the Entire (2007)
Ferreira, A.R.: Análise e Melhoria de Processos, p. 59 (2013)
Gagniuc, P.A.: Markov Chains: From Theory to Implementation and Experimentation. Wiley (2017)
Jorgensen, P.C.: Software Testing: A Craftsman’s Approach. Auerbach Publications (2013)
Juntao, W., Mishima, N.: Development of resource efficiency index for electrical and electronic equipment. Procedia CIRP 61, 275–280 (2017)
Kashyap, A., Holzer, T., Sarkani, S., Eveleigh, T.: Model based testing for software systems: an application of Markov modulated Markov process. Int. J. Comput. Appl. 46(14), 13–20 (2012)
Kaur, A., Goyal, S.: A genetic algorithm for regression test case prioritization using code coverage. Int. J. Comput. Sci. Eng. 3(5), 1839–1847 (2011)
Konsaard, P., Ramingwong, L.: Total coverage based regression test case prioritization using genetic algorithm. In: 2015 12th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), pp. 1–6. IEEE (2015)
Lee, D., Yannakakis, M.: Principles and methods of testing finite state machines - a survey. Proc. IEEE 84(8), 1090–1123 (1996)
Miranda, B., Cruciani, E., Verdecchia, R., Bertolino, A.: FAST approaches to scalable similarity-based test case prioritization. In: Proceedings of the International Conference on Software Engineering (ICSE), pp. 222–232 (2018)
Morozov, A., Ding, K., Chen, T., Janschek, K.: Test suite prioritization for efficient regression testing of model-based automotive software. In: 2017 International Conference on Software Analysis, Testing and Evolution (SATE), pp. 20–29 (2017)
Muthyala, K., Naidu, R.: A novel approach to test suite reduction using data mining approach. Indian J. Comput. Sci. Eng. 2(3), 500–505 (2011)
Ouriques, J.F.S., Cartaxo, E.G., Machado, P.D.: Test case prioritization techniques for model-based testing: a replicated study. Software Qual. J. 26(4), 1451–1482 (2018)
Ozawa, M., Dohi, T., Okamura, H.: How do software metrics affect test case prioritization? In: Annual Computer Software and Applications Conference (COMPSAC), vol. 01, pp. 245–250 (2018)
Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley (2014)
Raiyani, A.G., Pandya, S.S.: Proritization technique for minimizing number of test cases. Int. J. Softw. Eng. Res. Pract. 1, 3–9 (2011)
Saad, Y.: Numerical Methods for Large Eigenvalue Problems. Manchester University Press, Manchester (1992)
Sayyari, F., Emadi, S.: Automated generation of software testing path based on ant colony. In: 2015 International Congress on Technology, Communication and Knowledge (ICTCK), pp. 435–440. IEEE, November 2015
Singh, L.K., Tripathi, A.K., Vinod, G.: Software reliability early prediction in architectural design phase: overview and limitations. J. Softw. Eng. Appl. 4, 181–186 (2011)
Tijms, H.C., Tijms, H.C.: Stochastic Models: An Algorithmic Approach, vol. 303. Wiley, New York (1994)
Walton, G., Poore, J.: Measuring complexity and coverage of software specifications. Inf. Softw. Technol. 42(12), 859–872 (2000)
Whittaker, J.A., Thomason, M.G.: A Markov chain model for statistical software testing. IEEE Trans. Software Eng. 20(10), 812–824 (1994)
Xuan, J., Jiang, H., Ren, Z., Hu, Y., Luo, Z.: A random walk based algorithm for structural test case generation. In: The 2nd International Conference on Software Engineering and Data Mining, pp. 583–588 (2010)
Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw. Test. Verification Reliab. 22(2), 67–120 (2012)
Zhou, B., Okamura, H., Dohi, T.: Application of Markov chain Monte Carlo random testing to test case prioritization in regression testing. IEICE Trans. Inf. Syst. E95.D(9), 2219–2226 (2012)
Acknowledgements
This work has been partially funded by MUR PRIN 2017TWRCNB SEDUCE, and the PNRR MUR VITALITY (ECS00000041) Spoke 2 ASTRA - Advanced Space Technologies and Research Alliance. The authors acknowledge the support of the MUR (Italy) Department of Excellence 2023–2027 for GSSI.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 IFIP International Federation for Information Processing
About this paper
Cite this paper
Rebelo, L. et al. (2023). Prioritizing Test Cases with Markov Chains: A Preliminary Investigation. In: Bonfanti, S., Gargantini, A., Salvaneschi, P. (eds) Testing Software and Systems. ICTSS 2023. Lecture Notes in Computer Science, vol 14131. Springer, Cham. https://doi.org/10.1007/978-3-031-43240-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-43240-8_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43239-2
Online ISBN: 978-3-031-43240-8
eBook Packages: Computer ScienceComputer Science (R0)