Efficient Combinatorial Test Generation Based on Multivalued Decision Diagrams
Combinatorial interaction testing (CIT) is an emerging testing technique that has proved to be effective in finding faults due to the interaction among inputs. Efficient test generation for CIT is still an open problem especially when applied to real models having meaningful size and containing many constraints among inputs. In this paper we present a novel technique for the automatic generation of compact test suites starting from models containing constraints given in general form. It is based on the use of Multivalued Decision Diagrams (MDDs) which prove to be suitable to efficiently support CIT. We devise and experiment several optimizations including a novel variation of the classical greedy policy normally used in similar algorithms. The results of a thorough comparison with other similar techniques are presented and show that our approach can provide several advantages in terms of applicability, test suite size, generation time, and cost.
Unable to display preview. Download preview PDF.
- 1.Advanced Combinatorial Testing System (ACTS), http://csrc.nist.gov/groups/SNS/acts/
- 2.Arcaini, P., Gargantini, A., Vavassori, P.: Validation of models and tests for constrained combinatorial interaction testing. In: The 3rd International Workshop on Combinatorial Testing (IWCT 2014) In conjunction with International Conference on Software Testing ICSTW, pp. 98–107. IEEE (2014)Google Scholar
- 3.Babar, J., Miner, A.: Meddly: Multi-terminal and edge-valued decision diagram library. In: 7th International Conference on the Quantitative Evaluation of Systems. IEEE (2010)Google Scholar
- 6.Bryce, R.C., Colbourn, C.J.: One-test-at-a-time heuristic search for interaction test suites. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO 2007, pp. 1082–1089. ACM, New York (2007)Google Scholar
- 7.Bryce, R.C., Colbourn, C.J., Cohen, M.B.: A framework of greedy methods for constructing interaction test suites. In: ICSE 2005: Proc. of the 27th Int. Conf. on Software Engineering, pp. 146–155. ACM, New York (2005)Google Scholar
- 10.Calvagna, A., Gargantini, A., Vavassori, P.: Combinatorial interaction testing with CitLab. In: Sixth IEEE International Conference on Software Testing, Verification and Validation - Testing Tool Track (2013)Google Scholar
- 13.Covering Arrays by Simulated Annealing, http://cse.unl.edu/citportal/tools/casa/
- 17.Hadzic, T., Hansen, E.R.: On automata, MDDs and BDDs in constraint satisfaction. In: Proceedings of the ECAI 2008 Workshop on Inference Methods based on Graphical Structures of Knowledge (2008)Google Scholar
- 18.Kuhn, D.R., Reilly, M.J.: An investigation of the applicability of design of experiments to software testing. In: Society, I. (ed.) 27th NASA/IEEE Software Engineering Workshop, pp. 91–95 (2002)Google Scholar
- 22.Nagayama, S., Sasao, T.: Compact representations of logic functions using heterogeneous MDDs. In: Proceedings of 33rd International Symposium on Multiple-Valued Logic, pp. 247–252 (2003)Google Scholar
- 24.Pairwise web site, http://www.pairwise.org/
- 25.Salecker, E., Reicherdt, R., Glesner, S.: Calculating prioritized interaction test sets with constraints using binary decision diagrams. In: Proceedings of IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, pp. 278–285. IEEE Computer Society (2011)Google Scholar
- 26.Segall, I., Tzoref-Brill, R., Farchi, E.: Using binary decision diagrams for combinatorial test design. In: Proceedings of the 2011 International Symposium on Software Testing and Analysis, ISSTA 2011, pp. 254–264. ACM, New York (2011)Google Scholar