# On Combining Computational Differentiation and Toolkits for Parallel Scientific Computing

## Abstract

Automatic differentiation is a powerful technique for evalu- ating derivatives of functions given in the form of a high-level program- ming language such as Fortran, C, or C++. The program is treated as a potentially very long sequence of elementary statements to which the chain rule of differential calculus is applied over and over again. Combining automatic differentiation and the organizational structure of toolkits for parallel scientific computing provides a mechanism for eval- uating derivatives by exploiting mathematical insight on a higher level. In these toolkits, algorithmic structures such as BLAS-like operations, linear and nonlinear solvers, or integrators for ordinary differential equations can be identified by their standardized interfaces and recognized as high-level mathematical objects rather than as a sequence of elemen- tary statements. In this note, the differentiation of a linear solver with respect to some parameter vector is taken as an example. Mathematical insight is used to reformulate this problem into the solution of multiple linear systems that share the same coefficient matrix but differ in their right-hand sides. The experiments reported here use ADIC, a tool for the automatic differentiation of C programs, and PETSc, an object-oriented toolkit for the parallel solution of scientific problems modeled by partial differential equations.

## Keywords

Linear Solver Block Method Combine Computational Mathematical Insight Large Sparse System## Preview

Unable to display preview. Download preview PDF.

## References

- [1]Satish Balay, William D. Gropp, Lois Curfman McInnes, and Barry F. Smith. PETSc 2.0 users manual. Technical Report ANL-95/11-Revision 2.0.24, Argonne National Laboratory, 1999.Google Scholar
- [2]Satish Balay, William D. Gropp, Lois Curfman McInnes, and Barry F. Smith. PETSc home page. http://www.mcs.anl.gov/petsc, 1999.
- [3]Martin Berz, Christian Bischof, George Corliss, and Andreas Griewank.
*Computational Differentiation: Techniques, Applications, and Tools*. SIAM, Philadelphia, 1996.zbMATHGoogle Scholar - [4]Christian Bischof, Alan Carle, George Corliss, Andreas Griewank, and Paul Hovland. ADIFOR: Generating derivative codes from Fortran programs.
*Scientific Programming*, 1(1):11–29, 1992.Google Scholar - [5]Christian Bischof, Alan Carle, Peyvand Khademi, and Andrew Mauer. ADIFOR 2.0: Automatic differentiation of Fortran 77 programs.
*IEEE Computational Science & Engineering*, 3(3):18–32, 1996.CrossRefGoogle Scholar - [6]Christian Bischof, Lucas Roh, and Andrew Mauer. ADIC-An extensible automatic differentiation tool for ANSI-C.
*Software-Practice and Experience*, 27(12):1427–1456, 1997.CrossRefGoogle Scholar - [7]Roland W. Freund and Manish Malhotra. A block QMR algorithm for non-Hermitian linear systems with multiple right-hand sides.
*Linear Algebra and Its Applications*, 254:119–157, 1997.zbMATHCrossRefMathSciNetGoogle Scholar - [8]Jean-Charles Gilbert. Automatic differentiation and iterative processes.
*Optimization Methods and Software*, 1(1):13–22, 1992.CrossRefGoogle Scholar - [9]Andreas Griewank. On automatic differentiation. In
*Mathematical Programming: Recent Developments and Applications*, pages 83–108, Amsterdam, 1989. Kluwer Academic Publishers.Google Scholar - [10]Andreas Griewank.
*Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation*. SIAM, Philadelphia, 2000.zbMATHGoogle Scholar - [11]Andreas Griewank, Christian Bischof, George Corliss, Alan Carle, and Karen Williamson. Derivative convergence of iterative equation solvers.
*Optimization Methods and Software*, 2:321–355, 1993.CrossRefGoogle Scholar - [12]Andreas Griewank and George Corliss.
*Automatic Differentiation of Algorithms*. SIAM, Philadelphia, 1991.zbMATHGoogle Scholar - [13]Andreas Griewank, David Juedes, and Jean Utke. ADOL-C, a package for the automatic differentiation of algorithms written in C/C++.
*ACM Transactions on Mathematical Software*, 22(2):131–167, 1996.zbMATHCrossRefGoogle Scholar - [14]Paul Hovland, Boyana Norris, Lucas Roh, and Barry Smith. Developing a derivative-enhanced object-oriented toolkit for scientific computations. In Michael E. Henderson, Christopher R. Anderson, and Stephen L. Lyons, editors,
*Object Oriented Methods for Interoperable Scientific and Engineering Computing: Proceedings of the 1998 SIAM Workshop*, pages 129–137, Philadelphia, 1999. SIAM.Google Scholar - [15]Dianne P. O’Leary. The block conjugated gradient algorithm and related methods.
*Linear Algebra and Its Applications*, 29:293–322, 1980.zbMATHCrossRefMathSciNetGoogle Scholar - [16]Louis B. Rall.
*Automatic Differentiation: Techniques and Applications*, volume 120 of*Lecture Notes in Computer Science*. Springer Verlag, Berlin, 1981.zbMATHGoogle Scholar - [17]Yousef Saad.
*Iterative Methods for Sparse Linear Systems*. PWS Publishing Company, Boston, 1996.zbMATHGoogle Scholar