Higher-Order and Symbolic Computation

, Volume 21, Issue 4, pp 361–376 | Cite as

Nesting forward-mode AD in a functional framework

  • Jeffrey Mark Siskind
  • Barak A. Pearlmutter


We discuss the augmentation of a functional-programming language with a derivative-taking operator implemented with forward-mode automatic differentiation (AD). The primary technical difficulty in doing so lies in ensuring correctness in the face of nested invocation of that operator, due to the need to distinguish perturbations introduced by distinct invocations. We exhibit a series of implementations of a referentially-transparent forward-mode-AD derivative-taking operator, each of which uses a different non-referentially-transparent mechanism to distinguish perturbations. Even though the forward-mode-AD derivative-taking operator is itself referentially transparent, we hypothesize that one cannot correctly formulate this operator as a function definition in current pure dialects of Haskell.


Automatic differentiation Applicative (functional) languages Referential transparency Multiple transformation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Church, A.: The Calculi of Lambda Conversion. Princeton University Press, Princeton (1941) Google Scholar
  2. 2.
    Claessen, K., Sands, D.: Observable sharing for functional circuit description. In: Proc. of Asian Computer Science Conference (ASIAN). Springer, New York (1999) Google Scholar
  3. 3.
    Clifford, W.K.: Preliminary sketch of Bi-quaternions. Proc. Lond. Math. Soc. 4, 381–95 (1873) CrossRefGoogle Scholar
  4. 4.
    Flatt, M.: PLT MzScheme: language manual. Technical Report PLT-TR05-1-v300, PLT Scheme Inc. (2005) Google Scholar
  5. 5.
    Karczmarczuk, J.: Functional differentiation of computer programs. In: Proceedings of the III ACM SIGPLAN International Conference on Functional Programming. Baltimore, MD, pp. 195–203 (1998) Google Scholar
  6. 6.
    Karczmarczuk, J.: Lazy differential algebra and its applications. In: Workshop, III International Summer School on Advanced Functional Programming. Braga, Portugal (1998) Google Scholar
  7. 7.
    Karczmarczuk, J.: Functional coding of differential forms. In: Scottish Workshop on FP (1999) Google Scholar
  8. 8.
    Karczmarczuk, J.: Functional differentiation of computer programs. Higher-Order Symb. Comput. 14, 35–57 (2001) zbMATHCrossRefGoogle Scholar
  9. 9.
    Leibnitz, G.W.: A new method for maxima and minima as well as tangents, which is impeded neither by fractional nor irrational quantities, and a remarkable type of calculus for this. Acta Eruditorum (1664) Google Scholar
  10. 10.
    Newton, I.: De quadratura curvarum. In Optiks, 1704 edition. Appendix Google Scholar
  11. 11.
    Nilsson, H.: Functional automatic differentiation with Dirac impulses. In: Proceedings of the Eighth ACM SIGPLAN International Conference on Functional Programming. Uppsala, Sweden, pp. 153–164. ACM Press, New York (2003) CrossRefGoogle Scholar
  12. 12.
    Pearlmutter, B.A., Siskind, J.M.: Lazy multivariate higher-order forward- mode AD. In: Proceedings of the 2007 Symposium on Principles of Programming Languages. Nice, France, pp. 155–160 (2007) Google Scholar
  13. 13.
    Sussman, G.J., Wisdom, J., Mayer, M.E.: Structure and Interpretation of Classical Mechanics. MIT Press, Cambridge (2001) zbMATHGoogle Scholar
  14. 14.
    Wengert, R.E.: A simple automatic derivative evaluation program. Commun. ACM 7(8), 463–464 (1964) zbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.School of Electrical and Computer EngineeringPurdue UniversityWest LafayetteUSA
  2. 2.Hamilton InstituteNUI MaynoothMaynoothIreland

Personalised recommendations