Lazy Generation of Canonical Test Programs

  • Jason S. Reich
  • Matthew Naylor
  • Colin Runciman
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7257)


Property-based testing can be a highly effective form of lightweight verification, but it relies critically on the method used to generate test cases. If we wish to test properties of compilers and related tools we need a generator for source programs as test cases.

We describe experiments generating functional programs in a core first-order language with algebraic data types. Candidate programs are generated freely over a syntactic representation with positional names. Static conditions for program validity and canonical representatives of large equivalence classes are defined separately. The technique is used to investigate the correctness properties of a program optimisation and two language implementations.


automated testing Small Check lightweight verification compiler correctness search-based software engineering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abel, A.: Foetus — termination checker for simple functional programs (1998),
  2. 2.
    Boujarwah, A.S., Saleh, K.: Compiler test case generation methods: a survey and assessment. Information & Software Technology 39, 617–625 (1997)CrossRefGoogle Scholar
  3. 3.
    de Bruijn, N.G.: Lambda calculus notation with nameless dummies, a tool for automatic formula manipulation, with application to the Church-Rosser theorem. Indagationes Mathematicae 75(5), 381–392 (1972)zbMATHCrossRefGoogle Scholar
  4. 4.
    Claessen, K., Hughes, J.: QuickCheck: a lightweight tool for random testing of haskell programs. In: Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming, ICFP 2000, pp. 268–279. ACM (2000)Google Scholar
  5. 5.
    Dietz, P.F.: The GCL ANSI Common Lisp test suite (2008),
  6. 6.
    Jackson, D.: Software Abstractions: Logic, Language and Analysis, Revised edn. MIT Press (2012)Google Scholar
  7. 7.
    Katayama, S.: Systematic search for lambda expressions. In: Trends in Functional Programming, TFP 2005, vol. 6, pp. 111–126. Intellect Books (2007)Google Scholar
  8. 8.
    Naylor, M., Runciman, C.: The Reduceron reconfigured. In: Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming, ICFP 2010, pp. 75–86. ACM (2010)Google Scholar
  9. 9.
    O’Sullivan, B.: The criterion package, v0.5.1.1 (2011),
  10. 10.
    Palka, M.H., Claessen, K., Russo, A., Hughes, J.: Testing an optimising compiler by generating random lambda terms. In: Proceedings of the Sixth IEEE/ACM Workshop on Automation of Software Test, AST 2011, pp. 91–97 (2011)Google Scholar
  11. 11.
    Partain, W.: The nofib benchmark suite of Haskell programs. In: Functional Programming, Workshops in Computing, Glasgow 1992, pp. 195–202. Springer (1992)Google Scholar
  12. 12.
    Runciman, C., Naylor, M., Lindblad, F.: SmallCheck and Lazy SmallCheck: automatic exhaustive testing for small values. In: Proceedings of the First ACM SIGPLAN Symposium on Haskell, Haskell 2008, pp. 37–48. ACM (2008)Google Scholar
  13. 13.
    Sestoft, P.: Deriving a lazy abstract machine. Journal of Functional Programming 7, 231–264 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Tolmac, A., Chevalier, T.: The GHC Team: An external representation for the GHC Core Language (for GHC 6.10) (2009),

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jason S. Reich
    • 1
  • Matthew Naylor
    • 1
  • Colin Runciman
    • 1
  1. 1.Department of Computer ScienceUniversity of YorkUK

Personalised recommendations