Generation of Test Data Structures Using Constraint Logic Programming

  • Valerio Senni
  • Fabio Fioravanti
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7305)


The goal of Bounded-Exhaustive Testing (BET) is the automatic generation of all the test cases satisfying a given invariant, within a given bound. When the input has a complex structure, the development of correct and efficient generators becomes a very challenging task. In this paper we use Constraint Logic Programming (CLP) to systematically develop generators of structurally complex test data.

Similarly to filtering-based test generation, we follow a declarative approach which allows us to separate the issue of (i) defining the test structure and invariant, from that of (ii) generating admissible test input instances. This separation helps improve the correctness of the developed test case generators. However, in contrast with filtering approaches, we rely on a symbolic representation and we take advantage of efficient search strategies provided by CLP systems for generating test instances.

Through some experiments on examples taken from the literature on BET, we show that CLP, by combining the use of constraints and recursion, allows one to write intuitive and easily understandable test generators. We also show that these generators can be much more efficient than those built using ad-hoc filtering-based test generation tools like Korat.


Logic Programming Test Input Coverage Criterion Symbolic Execution Program Transformation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    The MAP transformation system (1995-2012),
  2. 2.
    ICST 2009, Second International Conference on Software Testing Verification and Validation, April 1-4. IEEE Computer Society, Denver (2009)Google Scholar
  3. 3.
    Aguirre, N., Bengolea, V.S., Frias, M.F., Galeotti, J.P.: Incorporating Coverage Criteria in Bounded Exhaustive Black Box Test Generation of Structural Inputs. In: Gogolla, M., Wolff, B. (eds.) TAP 2011. LNCS, vol. 6706, pp. 15–32. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  4. 4.
    Bartolini, C., Bertolino, A., Marchetti, E., Polini, A.: Ws-taxi: A wsdl-based testing tool for web services. In: ICST [2], pp. 326–335 (2009)Google Scholar
  5. 5.
    Boyapati, C., Khurshid, S., Marinov, D.: Korat: automated testing based on java predicates. In: ISSTA, pp. 123–133 (2002)Google Scholar
  6. 6.
    Charreteur, F., Botella, B., Gotlieb, A.: Modelling dynamic memory management in constraint-based testing. Journal of Systems and Software 82(11), 1755–1766 (2009)CrossRefGoogle Scholar
  7. 7.
    Coppit, D., Yang, J., Khurshid, S., Le, W., Sullivan, K.J.: Software assurance by bounded exhaustive testing. IEEE Trans. Software Eng. 31(4), 328–339 (2005)CrossRefGoogle Scholar
  8. 8.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 3rd edn. MIT Press (2009)Google Scholar
  9. 9.
    Degrave, F., Schrijvers, T., Vanhoof, W.: Towards a Framework for Constraint-Based Test Case Generation. In: De Schreye, D. (ed.) LOPSTR 2009. LNCS, vol. 6037, pp. 128–142. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    DeMillo, R.A., Offutt, A.J.: Constraint-based automatic test data generation. IEEE Trans. Software Eng. 17(9), 900–910 (1991)CrossRefGoogle Scholar
  11. 11.
    Dovier, A., Formisano, A., Pontelli, E.: An empirical study of constraint logic programming and answer set programming solutions of combinatorial problems. J. Exp. Theor. Artif. Intell. 21(2), 79–121 (2009)zbMATHCrossRefGoogle Scholar
  12. 12.
    Fioravanti, F., Pettorossi, A., Proietti, M.: Transformation Rules for Locally Stratified Constraint Logic Programs. In: Bruynooghe, M., Lau, K.-K. (eds.) Program Development in CL. LNCS, vol. 3049, pp. 291–339. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  13. 13.
    Fioravanti, F., Pettorossi, A., Proietti, M., Senni, V.: Program transformation for development, verification, and synthesis of programs. Intelligenza Artificiale 5(1), 119–125 (2011)Google Scholar
  14. 14.
    Gligoric, M., Gvero, T., Jagannath, V., Khurshid, S., Kuncak, V., Marinov, D.: Test generation through programming in udita. In: Kramer, J., Bishop, J., Devanbu, P.T., Uchitel, S. (eds.) ICSE (1), pp. 225–234. ACM (2010)Google Scholar
  15. 15.
    Gligoric, M., Gvero, T., Lauterburg, S., Marinov, D., Khurshid, S.: Optimizing generation of object graphs in java pathfinder. In: ICST [2], pp. 51–60Google Scholar
  16. 16.
    Gómez-Zamalloa, M., Albert, E., Puebla, G.: Test case generation for object-oriented imperative languages in clp. TPLP 10(4-6), 659–674 (2010)zbMATHGoogle Scholar
  17. 17.
    Gotlieb, A., Botella, B., Rueher, M.: A CLP Framework for Computing Structural Test Data. In: Palamidessi, C., Moniz Pereira, L., Lloyd, J.W., Dahl, V., Furbach, U., Kerber, M., Lau, K.-K., Sagiv, Y., Stuckey, P.J. (eds.) CL 2000. LNCS (LNAI), vol. 1861, pp. 399–413. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  18. 18.
    Wang, D., Chang, H.-Y., Ly-Gagnon, M., Hoffman, D.: Grammar based testing of html injection vulnerabilities in rss feeds. In: ICST [2], pp. 105–110 (2009)Google Scholar
  19. 19.
    Howe, J.M., King, A.: Efficient groundness analysis in prolog. Theory Pract. Log. Program. 3, 95–124 (2003)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Jaffar, J., Maher, M.J.: Constraint logic programming: A survey. J. Log. Program. 19/20, 503–581 (1994)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Jones, N.D., Gomard, C.K., Sestoft, P.: Partial evaluation and automatic program generation. Prentice-Hall, Inc., Upper Saddle River (1993)zbMATHGoogle Scholar
  22. 22.
    Khalek, S.A., Yang, G., Zhang, L., Marinov, D., Khurshid, S.: Testera: A tool for testing java programs using alloy specifications. In: Alexander, P., Pasareanu, C.S., Hosking, J.G. (eds.) ASE, pp. 608–611. IEEE (2011)Google Scholar
  23. 23.
    Kriener, J., King, A.: RedAlert: Determinacy Inference for Prolog. Theory and Practice of Logic Programming 11(4-5), 537–553 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Lloyd, J.W.: Foundations of Logic Programming, 2nd edn. Springer (1987)Google Scholar
  25. 25.
    Marinov, D.: Automatic Testing of Software with Structurally Complex Inputs. PhD thesis. MIT (2005)Google Scholar
  26. 26.
    Marinov, D., Andoni, A., Daniliuc, D., Khurshid, S., Rinard, M.: An evaluation of exhaustive testing for data structures. Technical report, MIT Computer Science and Artificial Intelligence Laboratory Report MIT -LCS-TR-921 (2003)Google Scholar
  27. 27.
    Marriott, K., Stuckey, P.J.: Programming with constraints: an introduction. MIT Press, Cambridge (1998)zbMATHGoogle Scholar
  28. 28.
    Meudec, C.: Atgen: automatic test data generation using constraint logic programming and symbolic execution. Software Testing, Verification and Reliability 11(2), 81–96 (2001)CrossRefGoogle Scholar
  29. 29.
    Milicevic, A., Misailovic, S., Marinov, D., Khurshid, S.: Korat: A tool for generating structurally complex test inputs. In: ICSE, pp. 771–774. IEEE Computer Society (2007)Google Scholar
  30. 30.
    Proietti, M., Pettorossi, A.: Unfolding - definition - folding, in this order, for avoiding unnecessary variables in logic programs. Theor. Comput. Sci. 142(1), 89–124 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Robinson, R.W.: Counting unlabeled acyclic digraphs. In: Little, C. (ed.) Combinatorial Mathematics V. Lecture Notes in Mathematics, vol. 622, pp. 28–43. Springer, Heidelberg (1977), 10.1007/BFb0069178 Google Scholar
  32. 32.
    Senni, V., Fioravanti, F.: Generation of test data structures using constraint logic programming. Technical Report 12-04, IASI-CNR, Roma, Italy (2012)Google Scholar
  33. 33.
    Visser, W., Pasareanu, C.S., Khurshid, S.: Test input generation with java pathfinder. In: Avrunin, G.S., Rothermel, G. (eds.) ISSTA, pp. 97–107. ACM (2004)Google Scholar
  34. 34.
    Visvanathan, S., Gupta, N.: Generating test data for functions with pointer inputs. In: ASE, p. 149. IEEE Computer Society (2002)Google Scholar
  35. 35.
    Zhao, R., Li, Q.: Automatic test generation for dynamic data structures. In: SERA, pp. 545–549. IEEE Computer Society (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Valerio Senni
    • 1
  • Fabio Fioravanti
    • 2
  1. 1.DISPUniversity of Rome Tor VergataRomeItaly
  2. 2.Dipartimento di ScienzeUniversity ‘G. D’Annunzio’PescaraItaly

Personalised recommendations