Discrete Adjoints of PETSc through dco/c++ and Adjoint MPI

  • Johannes Lotz
  • Uwe Naumann
  • Max Sagebaum
  • Michel Schanen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8097)


PETSc’s [1] robustness, scalability and portability makes it the foundation of various parallel implementations of numerical simulation codes. We formulate a least squares problem using a PETSc implementation as the model function and rely on adjoint mode Algorithmic Differentiation (AD) [2] for the accumulation of the derivative information. Various AD tools exist that apply the adjoint model to a given C/C++ code, while none is able to differentiate MPI [3] enabled code. We solved this by combining dco/c++ and the Adjoint MPI library, leading to a fully discrete adjoint implementation of PETSc. We want to underline that this work differs from accumulating derivative information through AD for PETSc algorithms (see e.g. [4]). We compute derivative information of PETSc itself opening up the possibility of an enclosing optimization problem (as needed, e.g., by [5]).


Adjoint Model Derivative Information Adjoint Mode Reverse Section Adjoint Code 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Balay, S., Brown, J., Buschelman, K., Gropp, W.D., Kaushik, D., Knepley, M.G., McInnes, L.C., Smith, B.F., Zhang, H.: PETSc Web page (2012),
  2. 2.
    Naumann, U.: The Art of Differentiating Computer Programs. An Introduction to Algorithmic Differentiation. Software, Environments, and Tools. SIAM, Philadelphia (2012)Google Scholar
  3. 3.
    MPI Forum: MPI: A Message-Passing Interface Standard. Version 2.2 (September 4, 2009), (December 2009)
  4. 4.
    Hovland, P.D., McInnes, L.C.: Parallel Simulation of Compressible Flow using Automatic Differentiation and PETSc. Parallel Computing 27(4), 503–519 (2001); Parallel Computing in AerospaceGoogle Scholar
  5. 5.
    Sagebaum, M., Gauger, N.R., Naumann, U., Lotz, J., Leppkes, K.: Algorithmic Differentiation of a Complex C++ Code with Underlying Libraries. Technical Report AIB-2013-04, RWTH Aachen (February 2013)Google Scholar
  6. 6.
    Averick, B.M., Carter, R.G., Mor, J.J.: The Minpack-2 Test Problem Collection (1991)Google Scholar
  7. 7.
    Ungermann, J., Blank, J., Lotz, J., Leppkes, K., Hoffmann, L., Guggenmoser, T., Kaufmann, M., Preusse, P., Naumann, U., Riese, M.: A 3-D Tomographic Retrieval Approach with Advection Compensation for the Air-Borne Limb-Imager GLORIA. Atmos. Meas. Tech. 4(11), 2509–2529 (2011)CrossRefGoogle Scholar
  8. 8.
    Lawson, C.L., Hanson, R.J., Kincaid, D.R., Krogh, F.T.: Basic Linear Algebra Subprograms for Fortran Usage. ACM Transactions on Mathematical Software (TOMS) 5(3), 308–323 (1979)zbMATHCrossRefGoogle Scholar
  9. 9.
    Anderson, E., Bai, Z., Bischof, C., Demmel, J., Dongarra, J., Du Croz, J., Greenbaum, A., Hammarling, S., McKenney, A., Ostrouchov, S.: Lapack users’ guide, release 2.0, vol. 326, p. 327. SIAM, Philadelphia (1995)Google Scholar
  10. 10.
    Agullo, E., Demmel, J., Dongarra, J., Hadri, B., Kurzak, J., Langou, J., Ltaief, H., Luszczek, P., Tomov, S.: Numerical Linear Algebra on Emerging Architectures: The PLASMA and MAGMA projects. Journal of Physics: Conference Series 180, 012037 (2009)Google Scholar
  11. 11.
    Gropp, W., Lusk, E., Skjellum, A.: Using MPI: Portable Parallel Programming with the Message Passing Interface. MIT Press (1994)Google Scholar
  12. 12.
    Hovland, P., Bischof, C.: Automatic Differentiation for Message-Passing Parallel Programs. In: IPPS 1998: Proceedings of the 12th International Parallel Processing Symposium on International Parallel Processing Symposium. IEEE Computer Society, Washington, DC (1998)Google Scholar
  13. 13.
    Schanen, M., Naumann, U.: A Wish List for Efficient Adjoints of One-Sided MPI Communication. In: Träff, J.L., Benkner, S., Dongarra, J.J. (eds.) EuroMPI 2012. LNCS, vol. 7490, pp. 248–257. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  14. 14.
    Schanen, M., Naumann, U., Hascoët, L., Utke, J.: Interpretative Adjoints for Numerical Simulation Codes using MPI. Procedia Computer Science 1(1), 1819 –1827 (2010); ICCS 2010Google Scholar
  15. 15.
    Utke, J., Hascoët, L., Heimbach, P., Hill, C., Hovland, P., Naumann, U.: Toward Adjoinable MPI. In: Proceedings of the 23rd IEEE International Parallel & Distributed Processing Symposium. IEEE Computer Society, Washington, DC (2009)Google Scholar
  16. 16.
    Griewank, A., Walther, A.: Algorithm 799: Revolve: An Implementation of Checkpoint for the Reverse or Adjoint Mode of Computational Differentiation. ACM Transactions on Mathematical Software 26(1), 19–45 (2000)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Johannes Lotz
    • 1
  • Uwe Naumann
    • 1
  • Max Sagebaum
    • 2
  • Michel Schanen
    • 1
  1. 1.LuFG Informatik 12: Software and Tools for Computational EngineeringRWTH Aachen UniversityGermany
  2. 2.Department of Mathematics and Center for Computational Engineering ScienceRWTH Aachen UniversityGermany

Personalised recommendations