Skip to main content
SpringerLink
Log in

Mechanized Metatheory for the Masses: The PoplMark Challenge

  • Conference paper
Theorem Proving in Higher Order Logics (TPHOLs 2005)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 3603))

Included in the following conference series:

Abstract

How close are we to a world where every paper on programming languages is accompanied by an electronic appendix with machine-checked proofs?

We propose an initial set of benchmarks for measuring progress in this area. Based on the metatheory of System F< :, a typed lambda-calculus with second-order polymorphism, subtyping, and records, these benchmarks embody many aspects of programming languages that are challenging to formalize: variable binding at both the term and type levels, syntactic forms with variable numbers of components (including binders), and proofs demanding complex induction principles. We hope that these benchmarks will help clarify the current state of the art, provide a basis for comparing competing technologies, and motivate further research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Appel, A.W.: Foundational proof-carrying code. In: IEEE Symposium on Logic in Computer Science (LICS), Boston, Massachusetts, June 2001, pp. 247–258 (2001)

    Google Scholar 

  2. Cardelli, L.: Extensible records in a pure calculus of subtyping. Research report 81, DEC/Compaq Systems Research Center (January 1992) Also In: Gunter, C.A., Mitchell, J.C. (eds.) Theoretical Aspects of Object-Oriented Programming: Types, Semantics, and Language Design. MIT Press, Cambridge (1994)

    Google Scholar 

  3. Cardelli, L., Martini, S., Mitchell, J.C., Scedrov, A.: An extension of System F with subtyping. Information and Computation 109(1–2), 4–56 (1994); Summary in TACS 1991 (Sendai, Japan, pp. 750–770) (1991)

    Article  MATH  MathSciNet  Google Scholar 

  4. Crary, K.: Toward a foundational typed assembly language. In: ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), New Orleans, Louisiana, January 2003, pp. 198–212 (2003)

    Google Scholar 

  5. Dennis, L.A.: Inductive challenge problems (2000), http://www.cs.nott.ac.uk/~lad/research/challenges

  6. Dubois, C., Menissier-Morain, V.: Certification of a type inference tool for ML: Damas-Milner within Coq. Journal of Automated Reasoning 23(3-4), 319–346 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  7. Floyd, R.W.: Assigning meanings to programs. In: Schwartz, J.T. (ed.) Proceedings of Symposia in Applied Mathematics. Mathematical Aspects of Computer Science, vol. 19, pp. 19–32. American Mathematical Society, Providence (1967)

    Google Scholar 

  8. Gabbay, M., Pitts, A.: A new approach to abstract syntax involving binders. In: 14th Symposium on Logic in Computer Science, pp. 214–224 (1999)

    Google Scholar 

  9. Gent, I.P., Walsh, T.: CSPLib: a benchmark library for constraints. Technical Report APES-09-1999, APES. A shorter version appears in the Proceedings of the 5th International Conference on Principles and Practices of Constraint Programming, CP-99 (1999), Available from http://www.dcs.st-and.ac.uk/~apes/apesreports.html

  10. Ghelli, G.: Proof Theoretic Studies about a Minimal Type System Integrating Inclusion and Parametric Polymorphism. PhD thesis, Università di Pisa, Technical report TD–6/90, Dipartimento di Informatica, Università di Pisa (1990)

    Google Scholar 

  11. Gordon, A.D., Melham, T.: Five axioms of alpha-conversion. In: von Wright, J., Harrison, J., Grundy, J. (eds.) TPHOLs 1996. LNCS, vol. 1125, pp. 173–190. Springer, Heidelberg (1996)

    Google Scholar 

  12. Green, I.: The dream corpus of inductive conjectures (1999), http://dream.dai.ed.ac.uk/dc/lib.html

  13. Gunter, E., Maharaj, S.: Studying the ML module system in HOL. The Computer Journal: Special Issue on Theorem Proving in Higher Order Logics 38(2), 142–151 (1995)

    Google Scholar 

  14. Hoare, T.: The verifying compiler: A grand challenge for computing research. J. ACM 50(1), 63–69 (2003)

    Article  Google Scholar 

  15. Hoos, H., Stuetzle, T.: SATLIB., http://www.intellektik.informatik.tu-darmstadt.de/SATLIB/

  16. Joyce, J.J., Seger, C.-J.H. (eds.): HUG 1993. LNCS, vol. 780. Springer, Heidelberg (1994)

    MATH  Google Scholar 

  17. Klein, G., Nipkow, T.: A machine-checked model for a Java-like language, virtual machine and compiler. Technical Report 0400001T.1, National ICT Australia, Sydney (March 2004)

    Google Scholar 

  18. McKinna, J., Pollack, R.: James McKinna and Robert Pollack. Some lambda calculus and type theory formalized. Journal of Automated Reasoning 23(3–4) (November 1999)

    Google Scholar 

  19. Milner, R., Tofte, M., Harper, R., MacQueen, D.: The Definition of Standard ML, Revised edition. MIT Press, Cambridge (1997)

    Google Scholar 

  20. Moore, J.S.: A grand challenge proposal for formal methods: A verified stack. In: Aichernig, B.K., Maibaum, T. (eds.) Formal Methods at the Crossroads. From Panacea to Foundational Support. LNCS, vol. 2757, pp. 161–172. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  21. Moore, J.S., Porter, G.: The apprentice challenge. ACM Trans. Program. Lang. Syst. 24(3), 193–216 (2002)

    Article  Google Scholar 

  22. Naraschewski, W., Nipkow, T.: Type inference verified: Algorithm W in Isabelle/HOL. Journal of Automated Reasoning 23, 299–318 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  23. Nipkow, T., von Oheimb, D., Pusch, C.: μJava: Embedding a programming language in a theorem prover. In: Bauer, F.L., Steinbrüggen, R. (eds.) Foundations of Secure Computation. Proc. Int. Summer School Marktoberdorf 1999, pp. 117–144. IOS Press, Amsterdam (2000)

    Google Scholar 

  24. Nipkow, T., von Oheimb, D.: Javalight is type-safe—definitely. In: POPL 1998: Proceedings of the 25th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 161–170. ACM Press, New York (1998)

    Chapter  Google Scholar 

  25. Norrish, M.: Formalising C in HOL. PhD thesis, Computer Laboratory, University of Cambridge (1998)

    Google Scholar 

  26. Norrish, M.: Mechanising Hankin and Barendregt using the Gordon-Melham axioms. In: MERLIN 2003: Proceedings Of The 2003 Workshop On Mechanized Reasoning About Languages With Variable Binding, pp. 1–7. ACM Press, New York (2003)

    Chapter  Google Scholar 

  27. Norrish, M.: Recursive function definition for types with binders. In: Slind, K., Bunker, A., Gopalakrishnan, G.C. (eds.) TPHOLs 2004. LNCS, vol. 3223, pp. 241–256. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  28. Pfenning, F., Elliot, C.: Higher-order abstract syntax. In: PLDI 1988: Proceedings of the ACM SIGPLAN 1988 Conference on Programming Language Design and Implementation, pp. 199–208. ACM Press, New York (1988)

    Chapter  Google Scholar 

  29. Pierce, B.C.: Types and Programming Languages. MIT Press, Cambridge (2002)

    Google Scholar 

  30. Pitts, A.M.: Nominal logic, a first order theory of names and binding. Inf. Comput. 186(2), 165–193 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  31. Sutcliffe, G., Suttner, C.: The TPTP problem library. Journal of Automated Reasoning 21(2), 177–203 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  32. Syme, D.: Reasoning with the formal definition of Standard ML in HOL. In: Joyce and Seger [16], pp. 43–60

    Google Scholar 

  33. Urban, C., Tasson, C.: Nominal techniques in Isabelle/HOL. Accepted at CADE-20 in Tallinn. See http://www.mathematik.uni-muenchen.de/~urban/nominal/

  34. VanInwegen, M., Gunter, E.: HOL-ML. In: Joyce and Seger [16], pp. 61–74

    Google Scholar 

  35. Vestergaard, R., Brotherston, J.: The mechanisation of Barendregt-style equational proofs (the residual perspective). In: Mechanized Reasoning about Languages with Variable Binding (MERLIN). Electronic Notes in Theoretical Computer Science, vol. 58. Elsevier, Amsterdam (2001)

    Google Scholar 

  36. Vestergaard, R., Brotherston, J.: A formalised first-order confluence proof for the λ-calculus using one-sorted variable names. RTA 2001 183(2), 212–244 (2003); Special edition with selected papers from RTA 2001

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Pennsylvania,  

    Brian E. Aydemir, Aaron Bohannon, J. Nathan Foster, Benjamin C. Pierce, Dimitrios Vytiniotis, Geoffrey Washburn, Stephanie Weirich & Steve Zdancewic

  2. Computer Laboratory, University of Cambridge,  

    Matthew Fairbairn & Peter Sewell

Authors
  1. Brian E. Aydemir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aaron Bohannon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew Fairbairn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. Nathan Foster
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Benjamin C. Pierce
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Peter Sewell
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dimitrios Vytiniotis
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Geoffrey Washburn
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Stephanie Weirich
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Steve Zdancewic
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computing Laboratory, Oxford University,  

    Joe Hurd

  2. Computing Laboratory, Oxford University, Wolfson Building, Parks Road Oxford, OX1 3QD, UK

    Tom Melham

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Aydemir, B.E. et al. (2005). Mechanized Metatheory for the Masses: The PoplMark Challenge. In: Hurd, J., Melham, T. (eds) Theorem Proving in Higher Order Logics. TPHOLs 2005. Lecture Notes in Computer Science, vol 3603. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11541868_4

Download citation

  • DOI: https://doi.org/10.1007/11541868_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-28372-0

  • Online ISBN: 978-3-540-31820-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation