The Data-Flow Equations of Checkpointing in Reverse Automatic Differentiation
Checkpointing is a technique to reduce the memory consumption of adjoint programs produced by reverse Automatic Differentiation. However, checkpointing also uses a non-negligible memory space for the so-called “snapshots”. We analyze the data-flow of checkpointing, yielding a precise characterization of all possible memory-optimal options for snapshots. This characterization is formally derived from the structure of checkpoints and from classical data-flow equations. In particular, we select two very different options and study their behavior on a number of real codes. Although no option is uniformly better, the so-called “lazy-snapshot” option appears preferable in general.
KeywordsMinimal Solution Code Fragment Optimal Option Real Code Forward Sweep
- 1.Aho, A., Sethi, R., Ullman, J.: Compilers: Principles, Techniques and Tools. Addison-Wesley, Reading (1986)Google Scholar
- 2.Faure, C., Naumann, U.: Minimizing the tape size. In: Corliss, G., Faure, C., Griewank, A., Hascoët, L., Naumann, U. (eds.) Automatic Differentiation of Algorithms: From Simulation to Optimization, Computer and Information Science, ch. 34, pp. 293–298. Springer, New York (2001)Google Scholar
- 5.Hascoët, L., Araya-Polo, M.: The adjoint data-flow analyses: Formalization, properties, and applications. In: Bücker, H.M., Corliss, G., Hovland, P., Naumann, U., Norris, B. (eds.) Automatic Differentiation: Applications, Theory, and Tools. Lecture Notes in Computational Science and Engineering. Springer, Heidelberg (2005)Google Scholar
- 6.Hascoët, L., Naumann, U., Pascual, V.: To be recorded analysis in reverse-mode automatic differentiation. Future Generation Computer Systems 21(8) (2004)Google Scholar
- 7.Hascoët, L., Pascual, V.: Tapenade 2.1 user’s guide. Technical report 0300, INRIA (2004), http://www.inria.fr/rrrt/rt-0300.html