Optimal Checkpointing for Time-Stepping Procedures in ADOL-C

  • Andreas Kowarz
  • Andrea Walther
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3994)


Using the basic reverse mode of automatic differentiation, the memory needed for the computation of discrete adjoints is proportional to the number of operations performed. This behavior is frequently not acceptable, especially for large-scale problems that involve a kind of time-stepping procedure. Therefore, we integrate a checkpointing mechanism into ADOL-C, a tool for the automatic differentiation of C and C++ programs. This checkpointing procedure is optimal for a given number of checkpoints in the sense that it yields the minimal number of recomputations. The resulting effects on the run-time behavior are illustrated by means of the derivative computation for an ODE-based optimization problem.


Operation Count Derivative Computation Adjoint Code Checkpointing Strategy Checkpoint Size 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Giering, R., Kaminski, T.: Recipes for Adjoint Code Construction. ACM Trans. Math. Software 24, 437–474 (1998)CrossRefMATHGoogle Scholar
  2. 2.
    Griewank, A.: Achieving logarithmic growth of temporal and spatial complexity in reverse automatic differentiation. Optim. Methods Softw. 1, 35–54 (1992)CrossRefGoogle Scholar
  3. 3.
    Griewank, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Frontiers in Appl. Math., vol. 19. SIAM, Philadelphia (2000)MATHGoogle Scholar
  4. 4.
    Griewank, A., Juedes, D., Utke, J.: ADOL-C: A package for the automatic differentiation of algorithms written in C/C++. ACM Trans. Math. Software 22, 131–167 (1996)CrossRefMATHGoogle Scholar
  5. 5.
    Griewank, A., Walther, A.: Revolve: An implementation of checkpointing for the reverse or adjoint mode of computational differentiation. ACM Trans. Math. Software 26, 19–45 (2000)CrossRefMATHGoogle Scholar
  6. 6.
    Griesse, R., Walther, A.: Parametric sensitivities for optimal control problems using automatic differentiation. Optimal Control Applications and Methods 24, 297–314 (2003)CrossRefMathSciNetMATHGoogle Scholar
  7. 7.
    Hinze, M., Sternberg, J.: A-Revolve: An adaptive memory and run-time-reduced procedure for calculating adjoints; with an application to the instationary Navier-Stokes system. Optim. Methods Softw. 20, 645–663 (2005)CrossRefMathSciNetMATHGoogle Scholar
  8. 8.
    Hascoët, L., Pascual, V.: Tapenade 2.1 user’s guide. Tech. rep. 300, INRIA (2004)Google Scholar
  9. 9.
    Knauer, M., Büskens, C.: Real-Time Trajectory Planning of the Industrial Robot IRB6400. PAMM 3, 515–516 (2003)CrossRefGoogle Scholar
  10. 10.
    Kubota, K.: A Fortran 77 preprocessor for reverse mode automatic differentiation with recursive checkpointing. Optim. Methods Softw. 10, 319–335 (1998)CrossRefMathSciNetMATHGoogle Scholar
  11. 11.
    Naumann, U., Utke, J., Lyons, A., Fagan, M.: Control Flow Reversal for Adjoint Code Generation. In: Proceed. of SCAM 2004, pp. 55–64. IEEE Computer Society, Los Alamitos (2004)Google Scholar
  12. 12.
    Walther, A., Griewank, A.: Advantages of binomial checkpointing for memory-reduced adjoint calculations. In: Feistauer, M., et al. (eds.) Numerical mathematics and advanced applications, Proc. ENUMATH 2003, pp. 834–843. Springer, Heidelberg (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Andreas Kowarz
    • 1
  • Andrea Walther
    • 1
  1. 1.Institute of Scientific ComputingTechnische Universität DresdenDresdenGermany

Personalised recommendations