Incorporating Coverage Criteria in Bounded Exhaustive Black Box Test Generation of Structural Inputs

  • Nazareno M. Aguirre
  • Valeria S. Bengolea
  • Marcelo F. Frias
  • Juan P. Galeotti
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6706)


The automated generation of test cases for heap allocated, complex, structures is particularly difficult. Various state of the art tools tackle this problem by bounded exhaustive exploration of potential test cases, using constraint solving mechanisms based on techniques such as search, model checking, symbolic execution and combinations of these.

In this article we present a technique for improving the bounded exhaustive constraint based test case generation of structurally complex inputs, for “filtering” approaches. The technique works by guiding the search considering a given black box test criterion. Such a test criterion is incorporated in the constraint based mechanism so that the exploration of potential test cases can be pruned without missing coverable classes of inputs, corresponding to the test criterion.

We present the technique, together with some case studies illustrating its performance for some black box testing criteria. The experimental results associated with these case studies are shown in the context of Korat, a state of the art tool for constraint based test case generation, but the approach is applicable in other contexts using a filtering approach to test generation.


Equivalence Class Test Criterion Test Input Coverage Criterion Symbolic Execution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barnett, M., Grieskamp, W., Nachmanson, L., Schulte, W., Tillmann, N., Veanes, M.: Model-Based Testing with AsmL.NET. In: Proceedings of the 1st European Conference on Model-Driven Software Engineering (2003)Google Scholar
  2. 2.
    Boyapati, C., Khurshid, S., Marinov, D.: Korat: Automated Testing based on Java Predicates. In: Proceedings of International Symposium on Software Testing and Analysis ISSTA 2002. ACM Press, New York (2002)Google Scholar
  3. 3.
    Galeotti, J.P., Rosner, N., López Pombo, C., Frias, M.: Analysis of invariants for efficient bounded verification. In: Proceedings of the 19th International Symposium on Software Testing and Analysis ISSTA 2010. ACM Press, Trento (2010)Google Scholar
  4. 4.
    Gligoric, M., Gvero, T., Jagannath, V., Khurshid, S., Kuncak, V., Marinov, D.: Test generation through programming in UDITA. In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering ICSE 2010. ACM Press, Cape Town (2010)Google Scholar
  5. 5.
    Jackson, D.: Software Abstractions: Logic, Language, and Analysis. The MIT Press, Cambridge (2006)Google Scholar
  6. 6.
    Kaner, C., Bach, J., Pettichord, B.: Lessons Learned in Software Testing. Wiley, Chichester (2001)Google Scholar
  7. 7.
    Khurshid, S., Marinov, D.: TestEra: Specification-Based Testing of Java Programs Using SAT. Automated Software Engineering 11(4) (2004)Google Scholar
  8. 8.
    Liskov, B., Guttag, J.: Program Development in Java: Abstraction, Specification and Object-Oriented Design. Addison-Wesley, Reading (2000)zbMATHGoogle Scholar
  9. 9.
    Ma, Y.-S., Offutt, J., Kwon, Y.-R.: MuJava: An Automated Class Mutation System. Journal of Software Testing, Verification and Reliability 15(2) (2005)Google Scholar
  10. 10.
    Milicevic, A., Misailovic, S., Marinov, D., Khurshid, S.: Korat: A Tool for Generating Structurally Complex Test Inputs. In: Proceedings of International Conference on Software Engineering ICSE 2007. IEEE Press, Los Alamitos (2007)Google Scholar
  11. 11.
  12. 12.
    Sen, K., Marinov, D., Agha, G.: CUTE: A Concolic Unit Testing Engine for C. In: Proceedings of the 5th Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering ESEC/FSE 2005. ACM Press, New York (2005)Google Scholar
  13. 13.
    Siddiqui, J., Khurshid, S.: An Empirical Study of Structural Constraint Solving Techniques. In: Breitman, K., Cavalcanti, A. (eds.) ICFEM 2009. LNCS, vol. 5885, pp. 88–106. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. 14.
    Siddiqui, J., Khurshid, S.: PKorat: Parallel Generation of Structurally Complex Test Inputs. In: Proceedings of the 2nd International Conference on Software Testing Verification and Validation ICST 2009. IEEE Computer Society, Los Alamitos (2009)Google Scholar
  15. 15.
    Visser, W., Pasareanu, C., Khurshid, S.: Test Input Generation with Java PathFinder. In: Proceedings of International Symposium on Software Testing and Analysis ISSTA 2004. ACM Press, New York (2004)Google Scholar
  16. 16.
    Xie, T., Marinov, D., Notkin, D.: Rostra: A Framework for Detecting Redundant Object-Oriented Unit Tests. In: Proceedings of the 19th IEEE International Conference on Automated Software Engineering ASE 2004. IEEE Computer Society, Linz (2004)Google Scholar
  17. 17.
    Zhu, H., Hall, P., May, J.: Software Unit Test Coverage and Adequacy. ACM Computing Surveys 29(4) (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Nazareno M. Aguirre
    • 1
  • Valeria S. Bengolea
    • 1
  • Marcelo F. Frias
    • 2
  • Juan P. Galeotti
    • 3
  1. 1.Departamento de Computación, FCEFQyNUniversidad Nacional de Río Cuarto and CONICETRío Cuarto, CórdobaArgentina
  2. 2.Departamento de Ingeniería InformáticaInstituto Tecnológico Buenos Aires and CONICETBuenos AiresArgentina
  3. 3.Departamento de Computación, FCEyNUniversidad de Buenos Aires and CONICETBuenos AiresArgentina

Personalised recommendations