Abstract
Logical relations are one among the most powerful techniques in the theory of programming languages, and have been used extensively for proving properties of a variety of higher-order calculi. However, there are properties that cannot be immediately proved by means of logical relations, for instance program continuity and differentiability in higher-order languages extended with real-valued functions. Informally, the problem stems from the fact that these properties are naturally expressed on terms of non-ground type (or, equivalently, on open terms of base type), and there is no apparent good definition for a base case (i.e. for closed terms of ground types). To overcome this issue, we study a generalization of the concept of a logical relation, called open logical relation, and prove that it can be fruitfully applied in several contexts in which the property of interest is about expressions of first-order type. Our setting is a simply-typed \(\lambda \)-calculus enriched with real numbers and real-valued first-order functions from a given set, such as the one of continuous or differentiable functions. We first prove a containment theorem stating that for any collection of real-valued first-order functions including projection functions and closed under function composition, any well-typed term of first-order type denotes a function belonging to that collection. Then, we show by way of open logical relations the correctness of the core of a recently published algorithm for forward automatic differentiation. Finally, we define a refinement-based type system for local continuity in an extension of our calculus with conditionals, and prove the soundness of the type system using open logical relations.
The Second and Fourth Authors are supported by the ANR project 16CE250011 REPAS, the ERC Consolidator Grant DIAPASoN – DLV-818616, and the MIUR PRIN 201784YSZ5 ASPRA.
Chapter PDF
Similar content being viewed by others
References
Abadi, M., Plotkin, G.D.: A simple differentiable programming language. PACMPL 4(POPL), 38:1–38:28 (2020)
Ahmed, A.J.: Step-indexed syntactic logical relations for recursive and quantified types. In: Proc. of ESOP 2006. pp. 69–83 (2006)
Appel, A.W., McAllester, D.A.: An indexed model of recursive types for foundational proof-carrying code. ACM Trans. Program. Lang. Syst. 23(5), 657–683 (2001)
Appel, A.W., Mellies, P.A., Richards, C.D., Vouillon, J.: A very modal model of a modern, major, general type system. In: ACM SIGPLAN Notices. vol. 42, pp. 109–122. ACM (2007)
Baillot, P., Dal Lago, U.: Higher-order interpretations and program complexity. In: Proc. of CSL 2012. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2012)
Barendregt, H.P.: The lambda calculus: its syntax and semantics. North-Holland (1984)
Barthe, G., Crubillé, R., Dal Lago, U., Gavazzo, F.: On the versatility of open logical relations: Continuity, automatic differentiation, and a containment theorem (long version) (2019), available at https://arxiv.org/abs/2002.08489
Bartholomew-Biggs, M., Brown, S., Christianson, B., Dixon, L.: Automatic differentiation of algorithms. Journal of Computational and Applied Mathematics 124(1), 171–190 (2000), numerical Analysis 2000. Vol. IV: Optimization and Nonlinear Equations
Baydin, A.G., Pearlmutter, B.A., Radul, A.A., Siskind, J.M.: Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research 18, 153:1–153:43 (2017)
Benton, N., Hofmann, M., Nigam, V.: Abstract effects and proof-relevant logical relations. In: Proc. of POPL 2014. pp. 619–632 (2014)
Biernacki, D., Piróg, M., Polesiuk, P., Sieczkowski, F.: Handle with care: relational interpretation of algebraic effects and handlers. PACMPL 2(POPL), 8:1–8:30 (2018)
Birkedal, L., Jaber, G., Sieczkowski, F., Thamsborg, J.: A kripke logical relation for effect-based program transformations. Inf. Comput. 249, 160–189 (2016)
Birkedal, L., Sieczkowski, F., Thamsborg, J.: A concurrent logical relation. In: Proc. of CSL 2012. pp. 107–121 (2012)
Bizjak, A., Birkedal, L.: Step-indexed logical relations for probability. In: Proc. of FoSSaCS 2015. pp. 279–294 (2015)
Bowman, W.J., Ahmed, A.: Noninterference for free. In: Proc. of ICFP 2015. pp. 101–113 (2015)
Brunel, A., Mazza, D., Pagani, M.: Backpropagation in the simply typed lambda-calculus with linear negation. PACMPL 4(POPL), 64:1–64:27 (2020)
Brunel, A., Terui, K.: Church \({=}>\) scott = ptime: an application of resource sensitive realizability. In: Proc. of DICE 2010. pp. 31–46 (2010)
Chaudhuri, S., Gulwani, S., Lublinerman, R.: Continuity analysis of programs. In: Proc. of POPL 2010. pp. 57–70 (2010)
Chaudhuri, S., Gulwani, S., Lublinerman, R.: Continuity and robustness of programs. Commun. ACM 55(8), 107–115 (2012)
Chaudhuri, S., Gulwani, S., Lublinerman, R., NavidPour, S.: Proving programs robust. In: Proc. of SIGSOFT/FSE 2011. pp. 102–112 (2011)
Clifford: Preliminary Sketch of Biquaternions. Proceedings of the London Mathematical Society s1-4(1), 381–395 (11 1871)
Cook, S.A., Kapron, B.M.: Characterizations of the basic feasible functionals of finite type (extended abstract). In: 30th Annual Symposium on Foundations of Computer Science, Research Triangle Park, North Carolina, USA, 30 October - 1 November 1989. pp. 154–159 (1989)
Crary, K., Harper, R.: Syntactic logical relations for polymorphic and recursive types. Electr. Notes Theor. Comput. Sci. 172, 259–299 (2007)
Crole, R.L.: Categories for Types. Cambridge mathematical textbooks, Cambridge University Press (1993)
Dreyer, D., Neis, G., Birkedal, L.: The impact of higher-order state and control effects on local relational reasoning. J. Funct. Program. 22(4–5), 477–528 (2012)
Edalat, A.: The domain of differentiable functions. Electr. Notes Theor. Comput. Sci. 40, 144 (2000)
Edalat, A., Lieutier, A.: Domain theory and differential calculus (functions of one variable). In: Proc. of LICS 2002. pp. 277–286 (2002)
Elliott, C.: The simple essence of automatic differentiation. PACMPL 2(ICFP), 70:1–70:29 (2018)
Escardó, M.H., Ho, W.K.: Operational domain theory and topology of sequential programming languages. Inf. Comput. 207(3), 411–437 (2009)
Fiore, M.P.: Semantic analysis of normalisation by evaluation for typed lambda calculus. In: Proc. of PPDP 2002. pp. 26–37 (2002)
Freeman, T., Pfenning, F.: Refinement types for ml. In: Proceedings of the ACM SIGPLAN 1991 Conference on Programming Language Design and Implementation. pp. 268–277. PLDI ’91 (1991)
Gianantonio, P.D., Edalat, A.: A language for differentiable functions. In: Proc. of FOSSACS 2013. pp. 337–352 (2013)
Girard, J.Y.: Une extension de l’interpretation de gödel a l’analyse, et son application a l’elimination des coupures dans l’analyse et la theorie des types. In: Studies in Logic and the Foundations of Mathematics, vol. 63, pp. 63–92. Elsevier (1971)
Goubault-Larrecq, J., Lasota, S., Nowak, D.: Logical relations for monadic types. In: International Workshop on Computer Science Logic. pp. 553–568. Springer (2002)
Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, second edn. (2008)
Hofmann, M.: Logical relations and nondeterminism. In: Software, Services, and Systems - Essays Dedicated to Martin Wirsing on the Occasion of His Retirement from the Chair of Programming and Software Engineering. pp. 62–74 (2015)
Huot, M., Staton, S., Vákár, M.: Correctness of automatic differentiation via diffeologies and categorical gluing (2020), to appear in Proc. of ESOP 2020 (long version available at http://arxiv.org/abs/2001.02209)
Jaber, G.: Syteci: automating contextual equivalence for higher-order programs with references. PACMPL 4(POPL), 59:1–59:28 (2020)
Jung, A., Tiuryn, J.: A new characterization of lambda definability. In: Proc. of TLCA 1993. pp. 245–257 (1993)
Kapron, B.M., Cook, S.A.: A new characterization of type-2 feasibility. SIAM J. Comput. 25(1), 117–132 (1996)
Lafont, Y.: Logiques, catégories & machines: implantation de langages de programmation guidée par la logique catégorique. Institut national de recherche en informatique et en automatique (1988)
Manzyuk, O., Pearlmutter, B.A., Radul, A.A., Rush, D.R., Siskind, J.M.: Perturbation confusion in forward automatic differentiation of higher-order functions. J. Funct. Program. 29, e12 (2019)
Mitchell, J.C.: Foundations for programming languages. Foundation of computing series, MIT Press (1996)
Owens, S., Myreen, M.O., Kumar, R., Tan, Y.K.: Functional big-step semantics. In: Proc. of ESOP 2016. pp. 589–615 (2016)
Pearlmutter, B.A., Siskind, J.M.: Lazy multivariate higher-order forward-mode AD. In: Proc. of POPL 2007. pp. 155–160 (2007)
Pearlmutter, B.A., Siskind, J.M.: Reverse-mode AD in a functional framework: Lambda the ultimate backpropagator. ACM Trans. Program. Lang. Syst. 30(2), 7:1–7:36 (2008)
Pitts, A.M., Stark, I.D.B.: Observable properties of higher order functions that dynamically create local names, or what’s new? In: Proc. of MFCS 1993. pp. 122–141 (1993)
Plotkin, G.: Lambda-definability and logical relations. Edinburgh University (1973)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Neurocomputing: Foundations of research. chap. Learning Representations by Back-propagating Errors, pp. 696–699. MIT Press (1988)
Shaikhha, A., Fitzgibbon, A., Vytiniotis, D., Peyton Jones, S.: Efficient differentiable programming in a functional array-processing language. PACMPL 3(ICFP), 97:1–97:30 (2019)
Siskind, J.M., Pearlmutter, B.A.: Nesting forward-mode AD in a functional framework. Higher-Order and Symbolic Computation 21(4), 361–376 (2008)
Spivak, M.: Calculus On Manifolds: A Modern Approach To Classical Theorems Of Advanced Calculus. Avalon Publishing (1971)
Staton, S., Yang, H., Wood, F.D., Heunen, C., Kammar, O.: Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints. In: Proc. of LICS 2016. pp. 525–534 (2016)
Tait, W.W.: Intensional interpretations of functionals of finite type i. Journal of Symbolic Logic 32(2), 198–212 (1967)
Tait, W.W.: A realizability interpretation of the theory of species. In: Logic Colloquium. pp. 240–251. Springer, Berlin, Heidelberg (1975)
Turon, A.J., Thamsborg, J., Ahmed, A., Birkedal, L., Dreyer, D.: Logical relations for fine-grained concurrency. In: Proc. of POPL 2013. pp. 343–356 (2013)
Vuillemin, J.: Exact real computer arithmetic with continued fractions. IEEE Trans. Comput. 39(8), 1087–1105 (1990)
Weihrauch, K.: Computable Analysis: An Introduction. Texts in Theoretical Computer Science. An EATCS Series, Springer, Berlin Heidelberg (2000)
Zhao, J., Zhang, Q., Zdancewic, S.: Relational parametricity for a polymorphic linear lambda calculus. In: Proc. of APLAS 2010. pp. 344–359 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2020 The Author(s)
About this paper
Cite this paper
Barthe, G., Crubillé, R., Lago, U.D., Gavazzo, F. (2020). On the Versatility of Open Logical Relations. In: Müller, P. (eds) Programming Languages and Systems. ESOP 2020. Lecture Notes in Computer Science(), vol 12075. Springer, Cham. https://doi.org/10.1007/978-3-030-44914-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-44914-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-44913-1
Online ISBN: 978-3-030-44914-8
eBook Packages: Computer ScienceComputer Science (R0)