Validating the Meta-Theory of Programming Languages (Short Paper)
We report on work in progress in building an environment for the validation of the meta-theory of programming languages artifacts, for example the correctness of compiler translations; the basic idea is to couple property-based testing with binders-aware functional programming as the meta-language for specification and testing. Treating binding signatures and related notions, such as new names generation, \(\alpha \)-equivalence and capture-avoiding substitution correctly and effectively is crucial in the verification and validation of programming language (meta)theory. We use Haskell as our meta-language, since it offers various libraries for both random and exhaustive generation of tests, as well as for binders. We validate our approach on benchmarks of mutations presented in the literature and some examples of code “in the wild”. In the former case, not only did we very quickly (re)discover all the planted bugs, but we achieved that with very little configuration effort with comparison to the competition. In the second case we located several simple bugs that had survived for years in publicly available (academic) code. We believe that our approach adds to the increasing evidence of the usefulness of property-based testing for semantic engineering of programming languages, in alternative or prior to full verification.
- 1.Amin, N., Tate, R.: Java and Scala’s type systems are unsound: the existential crisis of null pointers. In: OOPSLA 2016, pp. 838–848 (2016)Google Scholar
- 6.Claessen, K., Hughes, J.: QuickCheck: a lightweight tool for random testing of Haskell programs. In: ICFP 2000, pp. 268–279. ACM (2000)Google Scholar
- 7.Duregård, J., Jansson, P., Wang, M.: Feat: functional enumeration of algebraic types. In: Voigtländer, J. (ed.) Haskell Workshop, pp. 61–72. ACM (2012)Google Scholar
- 9.Runciman, C., Naylor, M., Lindblad, F.: Smallcheck and lazy SmallCheck: automatic exhaustive testing for small values. In: Haskell Workshop, pp. 37–48 (2008)Google Scholar
- 11.Visser, E., et al.: A language designer’s workbench: a one-stop-shop for implementation and verification of language designs. In: Onward! 2014, SPLASH 2014, pp. 95–111 (2014)Google Scholar
- 12.Weirich, S., Yorgey, B.A., Sheard, T.: Binders unbound. In: Chakravarty, M.M.T., Hu, Z., Danvy, O. (eds.) ICFP 2011, pp. 333–345. ACM (2011)Google Scholar