Advertisement

The Data-Flow Equations of Checkpointing in Reverse Automatic Differentiation

  • Benjamin Dauvergne
  • Laurent Hascoët
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3994)

Abstract

Checkpointing is a technique to reduce the memory consumption of adjoint programs produced by reverse Automatic Differentiation. However, checkpointing also uses a non-negligible memory space for the so-called “snapshots”. We analyze the data-flow of checkpointing, yielding a precise characterization of all possible memory-optimal options for snapshots. This characterization is formally derived from the structure of checkpoints and from classical data-flow equations. In particular, we select two very different options and study their behavior on a number of real codes. Although no option is uniformly better, the so-called “lazy-snapshot” option appears preferable in general.

Keywords

Minimal Solution Code Fragment Optimal Option Real Code Forward Sweep 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Aho, A., Sethi, R., Ullman, J.: Compilers: Principles, Techniques and Tools. Addison-Wesley, Reading (1986)Google Scholar
  2. 2.
    Faure, C., Naumann, U.: Minimizing the tape size. In: Corliss, G., Faure, C., Griewank, A., Hascoët, L., Naumann, U. (eds.) Automatic Differentiation of Algorithms: From Simulation to Optimization, Computer and Information Science, ch. 34, pp. 293–298. Springer, New York (2001)Google Scholar
  3. 3.
    Griewank, A.: Achieving logarithmic growth of temporal and spatial complexity in reverse automatic differentiation. Optimization Methods and Software 1, 35–54 (1992)CrossRefGoogle Scholar
  4. 4.
    Griewank, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Frontiers in Appl. Math., vol. 19. SIAM, Philadelphia (2000)MATHGoogle Scholar
  5. 5.
    Hascoët, L., Araya-Polo, M.: The adjoint data-flow analyses: Formalization, properties, and applications. In: Bücker, H.M., Corliss, G., Hovland, P., Naumann, U., Norris, B. (eds.) Automatic Differentiation: Applications, Theory, and Tools. Lecture Notes in Computational Science and Engineering. Springer, Heidelberg (2005)Google Scholar
  6. 6.
    Hascoët, L., Naumann, U., Pascual, V.: To be recorded analysis in reverse-mode automatic differentiation. Future Generation Computer Systems 21(8) (2004)Google Scholar
  7. 7.
    Hascoët, L., Pascual, V.: Tapenade 2.1 user’s guide. Technical report 0300, INRIA (2004), http://www.inria.fr/rrrt/rt-0300.html
  8. 8.
    Redfern, D.: The Maple handbook, Maple V, release 4. Springer, Heidelberg (1996)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Benjamin Dauvergne
    • 1
  • Laurent Hascoët
    • 1
  1. 1.TROPICS teamINRIA Sophia-AntipolisSophia-AntipolisFrance

Personalised recommendations