Skip to main content
SpringerLink
Log in

Automatic Differentiation: Introduction, History and Rounding Error Estimation

  • Reference work entry
Encyclopedia of Optimization

  • 237 Accesses

Most numerical algorithms for analyzing or optimizing the performance of a nonlinear system require the partial derivatives of functions that describe a mathematical model of the system. The automatic differentiation (abbreviated as AD in the following), or its synonym, computational differentiation, is an efficient method for computing the numerical values of the derivatives. AD combines advantages of numerical computation and those of symbolic computation [2], [4].

Given a vector-valued function f: R n → R m:

(1)

of n variables represented by a big program with hundreds or thousands of program statements, one often had encountered (before the advent of AD) some difficulties in computing the partial derivatives ∂f i /∂x j with conventional methods (as will be shown below). Now, one can successfully differentiate them with AD, deriving from the program for f another program that efficiently computes the numerical values of the partial derivatives.

AD is entirely different from the...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 1,699.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baur, W., and Strassen, V.: ‘The complexity of partial derivatives’, Theoret. Computer Sci.22 (1983), 317–330.

    MathSciNet  MATH  Google Scholar 

  2. Berz, M., Bischof, Ch., Corliss, G., and Griewank, A. (eds.): Computational differentiation: Techniques, applications, and tools, SIAM, 1996.

    Google Scholar 

  3. Griewank, A.: ‘Achieving logarithmic growth of temporal and spatial complexity in reverse automatic differentiation’, Optim. Methods and Software1 (1992), 35–54.

    Google Scholar 

  4. Griewank, A., and Corliss, G.F. (eds.): Automatic differentiation of algorithms: Theory, implementation, and application, SIAM, 1991.

    Google Scholar 

  5. Hillstrom, K.E.: ‘Installation guide for JAKEF’, Techn. Memorandum Math. and Computer Sci. Div. Argonne Nat. Lab.ANL/MCS-TM-17 (1985).

    Google Scholar 

  6. Hillstrom, K.E.: ‘User guide for JAKEF’, Techn. Memorandum Math. and Computer Sci. Div. Argonne Nat. Lab.ANL/MCS-TM-16 (1985).

    Google Scholar 

  7. Iri, M.: ‘Simultaneous computation of functions, partial derivatives and estimates of rounding errors — Complexity and practicality’, Japan J. Appl. Math.1 (1984), 223–252.

    Article  MathSciNet  MATH  Google Scholar 

  8. Iri, M., Tsuchiya, T., and Hoshi, M.: ‘Automatic computation of partial derivatives and rounding error estimates with applications to large-scale systems of nonlinear equations’, J. Comput. Appl. Math.24 (1988), 365–392.

    MATH  Google Scholar 

  9. Kagiwada, H., Kalaba, R., Rasakhoo, N., and Spingarn, K.: Numerical derivatives and nonlinear analysis, Vol. 31 of Math. Concepts and Methods in Sci. and Engin., Plenum, 1986.

    Google Scholar 

  10. Kubota, K., and Iri, M.: ‘Estimates of rounding errors with fast automatic differentiation and interval analysis’, J. Inform. Process. 14 (1991), 508–515.

    MathSciNet  MATH  Google Scholar 

  11. Linnainmaa, S.: ‘Taylor expansion of the accumulated rounding error’, BIT16 (1976), 146–160.

    MathSciNet  MATH  Google Scholar 

  12. Miller, W., and Wrathall, C.: Software for roundoff analysis of matrix algorithms, Acad. Press, 1980.

    Google Scholar 

  13. Ostrovskii, G.M., Wolin, Ju.M., and Borisov, W.W.: ‘Über die Berechnung von Ableitungen’, Wiss. Z. Techn. Hochschule Chemie13 (1971), 382–384.

    Google Scholar 

  14. Rall, L.B.: Automatic differentiation-Techniques and applications, Vol. 120 of Lecture Notes Computer Sci., Springer, 1981.

    Google Scholar 

  15. Speelpenning, B.: ‘Compiling fast partial derivatives of functions given by algorithms’, Report Dept. Computer Sci. Univ. IllinoisUIUCDCS-R-80-1002 (1980).

    Google Scholar 

  16. Wengert, R.E.: ‘A simple automatic derivative evaluation program’, Comm. ACM7 (1964), 463–464.

    MATH  Google Scholar 

  17. Werbos, P.: ‘Beyond regression: New tools for prediction and analysis in the behavioral sciences’, PhD Thesis Appl. Math. Harvard Univ. (1974).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Chuo Univ., 1–13–27 Kasuga Bunkyo-ku, Tokyo, 112–8551, Japan

    Masao Iri

  2. Chuo Univ., 1–13–27 Kasuga Bunkyo-ku, Tokyo, 112–8551, Japan

    Koichi Kubota

Authors
  1. Masao Iri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Koichi Kubota
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dept. Chemical Engin., Princeton Univ., Princeton, NJ, 08544-5263, USA

    Christodoulos A. Floudas

  2. Center for Applied Optim. Dept. Industrial and Systems Engin., Univ. Florida, Gainesville, FL, 32611, USA

    Panos M. Pardalos

Rights and permissions

Reprints and permissions

Copyright information

© 2001 Kluwer Academic Publishers

About this entry

Cite this entry

Iri, M., Kubota, K. (2001). Automatic Differentiation: Introduction, History and Rounding Error Estimation . In: Floudas, C.A., Pardalos, P.M. (eds) Encyclopedia of Optimization. Springer, Boston, MA. https://doi.org/10.1007/0-306-48332-7_19

Download citation

  • DOI: https://doi.org/10.1007/0-306-48332-7_19

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-7923-6932-5

  • Online ISBN: 978-0-306-48332-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation