The comKorat Tool: Unified Combinatorial and Constraint-Based Generation of Structurally Complex Tests
This tool paper presents comKorat, which unifies constraint-based generation of structurally complex tests with combinatorial testing. Constraint-based test generation is an effective approach for generating structurally complex inputs for systematic testing. While this approach can typically generate large numbers of tests, it has limited scalability – tests generated are usually only up to a small bound on input size. Combinatorial test generation, e.g., pair-wise testing, is a more scalable approach but is challenging to apply on commercial software systems that require complex input structures that cannot be formed by using arbitrary combinations. The comKorat tool integrates Korat and ACTS test generators to generate test suites for large scale commercial systems. This paper presents a case-study of applying comKorat on a software application developed at Yahoo!. The experimental results show that comKorat outperforms existing solution in execution time and finds a total of 23 previously unknown bugs in the application.
Part of this work was completed when the first author worked at Yahoo!. Lingming Zhang was funded in part by UT Dallas start-up fund and Google Faculty Research Award. Sarfraz Khurshid was funded in part by National Science Foundation (Grant No. CCF-0845628).
- 2.Boyapati, C., Khurshid, S., Marinov, D.: Korat: automated testing based on java predicates. In: ISSTA, pp. 123–133 (2002)Google Scholar
- 3.Cohen, D., Dalal, S., Fredman, M.L., Patton, G.: The AETG system: an approach to testing based on combinatorial design. TSE 23(7), 437–444 (1997)Google Scholar
- 5.Richard Kuhn, D., Okum, V.: Pseudo-exhaustive testing for software. In: SEW (2006)Google Scholar
- 6.Zhong, H.: Pairwise-Korat: Automated Testing Using Korat in an Industrial Setting. Master’s Report, The University of Texas at Austin, USA (2015)Google Scholar