We present a novel counterexample generator for the interactive theorem prover Isabelle based on a compiler that synthesizes test data generators for functional programming languages (e.g. ML, Haskell) from specifications in Isabelle. In contrast to naive type-based test data generators, the smart generators take the preconditions into account and only generate tests that fulfill the preconditions.

The smart generators are constructed by a compiler that reformulates the preconditions as logic programs and analyzes them with an enriched mode inference. From this inference, the compiler can construct the desired generators in the functional programming language.

Applying these test data generators reduces the number of tests significantly and enables us to find errors in specifications where naive random and exhaustive testing fail.


Mode Analysis Horn Clause Symbolic Execution Test Data Generator Exhaustive Testing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Antoy, S., Echahed, R., Hanus, M.: A needed narrowing strategy. J. ACM 47, 776–822 (2000)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Berghofer, S., Bulwahn, L., Haftmann, F.: Turning Inductive into Equational Specifications. In: Berghofer, S., Nipkow, T., Urban, C., Wenzel, M. (eds.) TPHOLs 2009. LNCS, vol. 5674, pp. 131–146. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Berghofer, S., Nipkow, T.: Random Testing in Isabelle/HOL. In: SEFM 2004, pp. 230–239. IEEE Computer Society (2004)Google Scholar
  4. 4.
    Blanchette, J.C., Nipkow, T.: Nitpick: A Counterexample Generator for Higher-Order Logic Based on a Relational Model Finder. In: Kaufmann, M., Paulson, L.C. (eds.) ITP 2010. LNCS, vol. 6172, pp. 131–146. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  5. 5.
    Bulwahn, L.: Smart test data generators via logic programming. In: ICLP 2011 (Technical Communications). Leibniz Int. Proc. in Informatics, vol. 11, pp. 139–150. Schloss Dagstuhl, Leibniz-Zentrum für Informatik ( (2011)Google Scholar
  6. 6.
    Carlier, M., Dubois, C., Gotlieb, A.: Constraint Reasoning in FocalTest. In: ICSOFT 2010 (2010)Google Scholar
  7. 7.
    Christiansen, J., Fischer, S.: EasyCheck — Test Data for Free. In: Garrigue, J., Hermenegildo, M.V. (eds.) FLOPS 2008. LNCS, vol. 4989, pp. 322–336. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Claessen, K., Hughes, J.: QuickCheck: A lightweight tool for random testing of Haskell programs. In: ICFP 2000, pp. 268–279. ACM SIGPLAN (2000)Google Scholar
  9. 9.
    Fischer, S., Kuchen, H.: Systematic generation of glass-box test cases for functional logic programs. In: PPDP 2007, pp. 63–74. ACM (2007)Google Scholar
  10. 10.
    Haftmann, F., Nipkow, T.: Code Generation via Higher-Order Rewrite Systems. In: Blume, M., Kobayashi, N., Vidal, G. (eds.) FLOPS 2010. LNCS, vol. 6009, pp. 103–117. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Hanus, M.: Multi-paradigm Declarative Languages. In: Dahl, V., Niemelä, I. (eds.) ICLP 2007. LNCS, vol. 4670, pp. 45–75. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  12. 12.
    Jackson, D.: Software Abstractions: Logic, Language, and Analysis. The MIT Press (2006)Google Scholar
  13. 13.
    Lindblad, F.: Property directed generation of first-order test data. In: The Eigth Symposium on Trends in Functional Programming (2007)Google Scholar
  14. 14.
    Mellish, C.S.: The automatic generation of mode declarations for Prolog programs. Technical Report 163, Department of Artificial Intelligence (1981)Google Scholar
  15. 15.
    Naish, L.: Adding Equations to NU-Prolog. In: Małuszyński, J., Wirsing, M. (eds.) PLILP 1991. LNCS, vol. 528, pp. 15–26. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  16. 16.
    Nipkow, T.: Verifying a Hotel Key Card System. In: Barkaoui, K., Cavalcanti, A., Cerone, A. (eds.) ICTAC 2006. LNCS, vol. 4281, pp. 1–14. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Overton, D., Somogyi, Z., Stuckey, P.J.: Constraint-based mode analysis of mercury. In: PPDP 2002, pp. 109–120. ACM (2002)Google Scholar
  18. 18.
    Rouveirol, C.: Flattening and Saturation: Two Representation Changes for Generalization. Mach. Learn. 14(2), 219–232 (1994)zbMATHCrossRefGoogle Scholar
  19. 19.
    Runciman, C., Naylor, M., Lindblad, F.: SmallCheck and Lazy SmallCheck: automatic exhaustive testing for small values. In: Haskell 2008, pp. 37–48. ACM (2008)Google Scholar
  20. 20.
    Smaus, J.G., Hill, P.M., King, A.: Mode Analysis Domains for Typed Logic Programs. In: Bossi, A. (ed.) LOPSTR 1999. LNCS, vol. 1817, pp. 82–101. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  21. 21.
    Weber, T.: Bounded model generation for Isabelle/HOL. In: PDPAR 2004. Electronic Notes in Theoretical Computer Science, vol. 125(3), pp. 103–116. Elsevier (2005)Google Scholar
  22. 22.
    Wenzel, M., Paulson, L.C., Nipkow, T.: The Isabelle Framework. In: Mohamed, O.A., Muñoz, C., Tahar, S. (eds.) TPHOLs 2008. LNCS, vol. 5170, pp. 33–38. Springer, Heidelberg (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Lukas Bulwahn
    • 1
  1. 1.Technische Universität MünchenGermany

Personalised recommendations