Set-Based Algorithms for Combinatorial Test Set Generation
Testing is an important and expensive part of software and hardware development. Over the recent years, the construction of combinatorial interaction tests rose to play an important role towards making the cost of testing more efficient. Covering arrays are the key element of combinatorial interaction testing and a means to provide abstract test sets. In this paper, we present a family of set-based algorithms for generating covering arrays and thus combinatorial test sets. Our algorithms build upon an existing mathematical method for constructing independent families of sets, which we extend sufficiently in terms of algorithmic design in this paper. We compare our algorithms against commonly used greedy methods for producing 3-way combinatorial test sets, and these initial evaluation results favor our approach in terms of generating smaller test sets.
KeywordsCombinatorial testing Independent families of sets Set-based algorithms
This work has been funded by the Austrian Research Promotion Agency (FFG) under grant 851205 and the Austrian COMET Program (FFG).
- 1.The economic impacts of inadequate infrastructure for software testing. U.S. Department of Commerce, National Institute of Standards and Technology (2002)Google Scholar
- 3.Colbourn, C.J.: Table for CAN(3, k, 2) for k up to 10000. http://www.public.asu.edu/~ccolbou/src/tabby/3-2-ca.html. Accessed 25 Apr 2016
- 11.Kuhn, D., Kacker, R., Lei, Y.: Practical combinatorial testing. In: NIST Special Publication pp. 800–142 (2010)Google Scholar
- 15.Lei, Y., Tai, K.C.: In-parameter-order: a test generation strategy for pairwise testing. In: 1998 3rd IEEE International Proceedings of High-Assurance Systems Engineering Symposium, pp. 254–261. IEEE (1998)Google Scholar
- 16.NIST: Table for CA(3, k, 2). National Institute of Standards and Technology. http://math.nist.gov/coveringarrays/ipof/tables/table.3.2.html. Accessed 25 Apr 2016
- 18.Torres-Jimenez, J., Izquierdo-Marquez, I.: Survey of covering arrays. In: 2013 15th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), pp. 20–27. IEEE (2013)Google Scholar
- 19.Yu, L., Lei, Y., Kacker, R.N., Kuhn, D.R.: Acts: a combinatorial test generation tool. In: 2013 IEEE 6th International Conference on Software Testing, Verification and Validation (ICST), pp. 370–375. IEEE (2013)Google Scholar