Higher-Order and Symbolic Computation

, Volume 21, Issue 3, pp 311–331

On the implementation of automatic differentiation tools

  • Christian H. Bischof
  • Paul D. Hovland
  • Boyana Norris
Article

DOI: 10.1007/s10990-008-9034-4

Cite this article as:
Bischof, C.H., Hovland, P.D. & Norris, B. Higher-Order Symb Comput (2008) 21: 311. doi:10.1007/s10990-008-9034-4

Abstract

Automatic differentiation is a semantic transformation that applies the rules of differential calculus to source code. It thus transforms a computer program that computes a mathematical function into a program that computes the function and its derivatives. Derivatives play an important role in a wide variety of scientific computing applications, including numerical optimization, solution of nonlinear equations, sensitivity analysis, and nonlinear inverse problems. We describe the forward and reverse modes of automatic differentiation and provide a survey of implementation strategies. We describe some of the challenges in the implementation of automatic differentiation tools, with a focus on tools based on source transformation. We conclude with an overview of current research and future opportunities.

Keywords

Semantic transformationAutomatic differentiation

Copyright information

© US Government 2008

Authors and Affiliations

  • Christian H. Bischof
    • 1
  • Paul D. Hovland
    • 2
  • Boyana Norris
    • 2
  1. 1.Institute for Scientific ComputingAachen University of TechnologyAachenGermany
  2. 2.Mathematics and Computer Science DivisionArgonne National LaboratoryArgonneUSA