Using Coverage Criteria on RepOK to Reduce Bounded-Exhaustive Test Suites

  • Valeria Bengolea
  • Nazareno Aguirre
  • Darko Marinov
  • Marcelo F. Frias
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7305)


Bounded-exhaustive exploration of test case candidates is a commonly employed approach for test generation in some contexts. Even when small bounds are used for test generation, executing the obtained tests may become prohibitive, despite the time for test generation not being prohibitive. In this paper, we propose a technique for reducing the size of bounded-exhaustive test suites. This technique is based on the application of coverage criteria on the representation invariant of the structure for which the suite was produced. More precisely, the representation invariant (which is often implemented as a repOK routine) is executed to determine how its code is exercised by (valid) test inputs. Different valid test inputs are deemed equivalent if they exercise the repOK code in a similar way according to a white-box testing criterion. These equivalences between test cases are exploited for reducing test suites by removing from the suite those tests that are equivalent to some test already present in the suite.

We present case studies that evaluate the effectiveness of our technique. The results show that by reducing the size of bounded-exhaustive test suites up to two orders of magnitude, we obtain test suites whose efficacy measured as their mutant-killing ability is comparable to that of bounded-exhaustive test suites.


Equivalence Class Test Suite Coverage Criterion Path Coverage Decision Coverage 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aguirre, N., Bengolea, V., Frias, M., Galeotti, J.: Incorporating Coverage Criteria in Bounded Exhaustive Black Box Test Generation of Structural Inputs. In: Gogolla, M., Wolff, B. (eds.) TAP 2011. LNCS, vol. 6706, pp. 15–32. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    Boyapati, C., Khurshid, S., Marinov, D.: Korat: Automated Testing based on Java Predicates. In: Proc. of Intl. Symposium on Software Testing and Analysis ISSTA 2002. ACM Press (2002)Google Scholar
  3. 3.
    Chalin, P., Kiniry, J.R., Leavens, G.T., Poll, E.: Beyond Assertions: Advanced Specification and Verification with JML and ESC/Java2. In: de Boer, F.S., Bonsangue, M.M., Graf, S., de Roever, W.-P. (eds.) FMCO 2005. LNCS, vol. 4111, pp. 342–363. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  4. 4.
  5. 5.
    Daniel, B., Dig, D., García, K., Marinov, D.: Automated Testing of Refactoring Engines. In: Proc. of European Software Engineering Conference and Intl. Symposium on Foundations of Software Engineering ESEC/FSE 2007. ACM Press (2007)Google Scholar
  6. 6.
    Myers, G.J.: The Art of Sofware Testing, 2nd edn. John Wiley & Sons, Inc. (2004)Google Scholar
  7. 7.
    Gligoric, M., Gvero, T., Jagannath, V., Khurshid, S., Kuncak, V., Marinov, D.: Test Generation through Programming in UDITA. In: Proc. of Intl. Conference on Software Engineering ICSE 2010. ACM Press (2010)Google Scholar
  8. 8.
    Jackson, D.: Software Abstractions: Logic, Language, and Analysis. The MIT Press (2006)Google Scholar
  9. 9.
    Jagannath, V., Lee, Y.Y., Daniel, B., Marinov, D.: Reducing the Costs of Bounded-Exhaustive Testing. In: Chechik, M., Wirsing, M. (eds.) FASE 2009. LNCS, vol. 5503, pp. 171–185. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  10. 10.
    Khurshid, S., Marinov, D.: TestEra: Specification-Based Testing of Java Programs Using SAT. Automated Software Engineering 11(4) (2004)Google Scholar
  11. 11.
    Liskov, B., Guttag, J.: Program Development in Java: Abstraction, Specification and Object-Oriented Design. Addison-Wesley (2000)Google Scholar
  12. 12.
    Ma, Y.-S., Offutt, J., Kwon, Y.-R.: MuJava: An Automated Class Mutation System. Journal of Software Testing, Verification and Reliability 15(2) (2005)Google Scholar
  13. 13.
    Milicevic, A., Misailovic, S., Marinov, D., Khurshid, S.: Korat: A Tool for Generating Structurally Complex Test Inputs. In: Proc. of Intl. Conference on Software Engineering ICSE 2007. IEEE Press (2007)Google Scholar
  14. 14.
  15. 15.
  16. 16.
    Rothermel, G., Elbaum, S., Malishevsky, A., Kallakuri, P., Davia, B.: The Impact of Test Suite Granularity on the Cost-Effectiveness of Regression Testing. In: Proc. of Intl. Conference on Software Engineering ICSE 2002. ACM Press (2002)Google Scholar
  17. 17.
    Sullivan, K., Yang, J., Coppit, D., Khurshud, S., Jackson, D.: Software Assurance by Bounded Exhaustive Testing. In: Proc. of Intl. Symposium on Software Testing and Analysis ISSTA 2004. ACM Press (2004)Google Scholar
  18. 18.
    Yu, Y., Jones, J., Harrold, M.: An Empirical Study of the Effects of Test-Suite Reduction on Fault Localization. In: Proc. of Intl. Conference on Software Engineering ICSE 2008. ACM Press (2008)Google Scholar
  19. 19.
    Zhu, H., Hall, P., May, J.: Software Unit Test Coverage and Adequacy. ACM Computing Surveys 29(4) (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Valeria Bengolea
    • 1
    • 4
  • Nazareno Aguirre
    • 1
    • 4
  • Darko Marinov
    • 2
  • Marcelo F. Frias
    • 3
    • 4
  1. 1.Department of Computer Science, FCEFQyNUniversidad Nacional de Río CuartoArgentina
  2. 2.Department of Computer ScienceUniversity of Illinois at Urbana-ChampaignUSA
  3. 3.Department of Software EngineeringBuenos Aires Institute of TechnologyArgentina
  4. 4.Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)Argentina

Personalised recommendations