On Combining Computational Differentiation and Toolkits for Parallel Scientific Computing

  • Christian H. Bischof
  • H. Martin Bücker
  • Paul D. Hovland
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1900)


Automatic differentiation is a powerful technique for evalu- ating derivatives of functions given in the form of a high-level program- ming language such as Fortran, C, or C++. The program is treated as a potentially very long sequence of elementary statements to which the chain rule of differential calculus is applied over and over again. Combining automatic differentiation and the organizational structure of toolkits for parallel scientific computing provides a mechanism for eval- uating derivatives by exploiting mathematical insight on a higher level. In these toolkits, algorithmic structures such as BLAS-like operations, linear and nonlinear solvers, or integrators for ordinary differential equations can be identified by their standardized interfaces and recognized as high-level mathematical objects rather than as a sequence of elemen- tary statements. In this note, the differentiation of a linear solver with respect to some parameter vector is taken as an example. Mathematical insight is used to reformulate this problem into the solution of multiple linear systems that share the same coefficient matrix but differ in their right-hand sides. The experiments reported here use ADIC, a tool for the automatic differentiation of C programs, and PETSc, an object-oriented toolkit for the parallel solution of scientific problems modeled by partial differential equations.


Linear Solver Block Method Combine Computational Mathematical Insight Large Sparse System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. [1]
    Satish Balay, William D. Gropp, Lois Curfman McInnes, and Barry F. Smith. PETSc 2.0 users manual. Technical Report ANL-95/11-Revision 2.0.24, Argonne National Laboratory, 1999.Google Scholar
  2. [2]
    Satish Balay, William D. Gropp, Lois Curfman McInnes, and Barry F. Smith. PETSc home page., 1999.
  3. [3]
    Martin Berz, Christian Bischof, George Corliss, and Andreas Griewank. Computational Differentiation: Techniques, Applications, and Tools. SIAM, Philadelphia, 1996.zbMATHGoogle Scholar
  4. [4]
    Christian Bischof, Alan Carle, George Corliss, Andreas Griewank, and Paul Hovland. ADIFOR: Generating derivative codes from Fortran programs. Scientific Programming, 1(1):11–29, 1992.Google Scholar
  5. [5]
    Christian Bischof, Alan Carle, Peyvand Khademi, and Andrew Mauer. ADIFOR 2.0: Automatic differentiation of Fortran 77 programs. IEEE Computational Science & Engineering, 3(3):18–32, 1996.CrossRefGoogle Scholar
  6. [6]
    Christian Bischof, Lucas Roh, and Andrew Mauer. ADIC-An extensible automatic differentiation tool for ANSI-C. Software-Practice and Experience, 27(12):1427–1456, 1997.CrossRefGoogle Scholar
  7. [7]
    Roland W. Freund and Manish Malhotra. A block QMR algorithm for non-Hermitian linear systems with multiple right-hand sides. Linear Algebra and Its Applications, 254:119–157, 1997.zbMATHCrossRefMathSciNetGoogle Scholar
  8. [8]
    Jean-Charles Gilbert. Automatic differentiation and iterative processes. Optimization Methods and Software, 1(1):13–22, 1992.CrossRefGoogle Scholar
  9. [9]
    Andreas Griewank. On automatic differentiation. In Mathematical Programming: Recent Developments and Applications, pages 83–108, Amsterdam, 1989. Kluwer Academic Publishers.Google Scholar
  10. [10]
    Andreas Griewank. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. SIAM, Philadelphia, 2000.zbMATHGoogle Scholar
  11. [11]
    Andreas Griewank, Christian Bischof, George Corliss, Alan Carle, and Karen Williamson. Derivative convergence of iterative equation solvers. Optimization Methods and Software, 2:321–355, 1993.CrossRefGoogle Scholar
  12. [12]
    Andreas Griewank and George Corliss. Automatic Differentiation of Algorithms. SIAM, Philadelphia, 1991.zbMATHGoogle Scholar
  13. [13]
    Andreas Griewank, David Juedes, and Jean Utke. ADOL-C, a package for the automatic differentiation of algorithms written in C/C++. ACM Transactions on Mathematical Software, 22(2):131–167, 1996.zbMATHCrossRefGoogle Scholar
  14. [14]
    Paul Hovland, Boyana Norris, Lucas Roh, and Barry Smith. Developing a derivative-enhanced object-oriented toolkit for scientific computations. In Michael E. Henderson, Christopher R. Anderson, and Stephen L. Lyons, editors, Object Oriented Methods for Interoperable Scientific and Engineering Computing: Proceedings of the 1998 SIAM Workshop, pages 129–137, Philadelphia, 1999. SIAM.Google Scholar
  15. [15]
    Dianne P. O’Leary. The block conjugated gradient algorithm and related methods. Linear Algebra and Its Applications, 29:293–322, 1980.zbMATHCrossRefMathSciNetGoogle Scholar
  16. [16]
    Louis B. Rall. Automatic Differentiation: Techniques and Applications, volume 120 of Lecture Notes in Computer Science. Springer Verlag, Berlin, 1981.zbMATHGoogle Scholar
  17. [17]
    Yousef Saad. Iterative Methods for Sparse Linear Systems. PWS Publishing Company, Boston, 1996.zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Christian H. Bischof
    • 1
  • H. Martin Bücker
    • 1
  • Paul D. Hovland
    • 2
  1. 1.Institute for Scientific ComputingAachen University of TechnologyAachenGermany
  2. 2.Mathematics and Computer Science DivisionArgonne National LaboratoryArgonneUSA

Personalised recommendations