Hierarchical Algorithmic Differentiation A Case Study

  • Johannes LotzEmail author
  • Uwe Naumann
  • Jörn Ungermann
Conference paper
Part of the Lecture Notes in Computational Science and Engineering book series (LNCSE, volume 87)


This case study in Algorithmic Differentiation (AD) discusses the semi-automatic generation of an adjoint simulation code in the context of an inverse atmospheric remote sensing problem. In-depth structural and performance analyses allow for the run time factor between the adjoint generated by overloading in C++ and the original forward simulation to be reduced to 3. 5. The dense Jacobian matrix of the underlying problem is computed at the same cost. This is achieved by a hierarchical AD using adjoint mode locally for preaccumulation and by exploiting interface contraction. For the given application this approach yields a speed-up over black-box tangent-linear and adjoint mode of more than 170. Furthermore, the memory consumption is reduced by a factor of 1,000 compared to applying black-box adjoint mode.


Atmospheric remote sensing Inverse problems Algorithmic differentiation 



Johannes Lotz is supported by the German Science Foundation (DFG grant No. 487/4-1).


  1. 1.
    Bücker, H.M., Rasch, A.: Modeling the performance of interface contraction. ACM Transactions on Mathematical Software 29(4), 440–457 (2003). DOI Google Scholar
  2. 2.
    Gebremedhin, A., Manne, F., Pothen, A.: What color is your Jacobian? Graph coloring for computing derivatives. SIAM Review 47(4), 629–705 (2005)Google Scholar
  3. 3.
    Griewank, A., Juedes, D., Utke, J.: Algorithm 755: ADOL-C: A package for the automatic differentiation of algorithms written in C/C++. ACM Transactions on Mathematical Software 22(2), 131–167 (1996). URL
  4. 4.
    Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd edn. No. 105 in Other Titles in Applied Mathematics. SIAM, Philadelphia, PA (2008). URL
  5. 5.
    Hascoët, L., Fidanova, S., Held, C.: Adjoining independent computations. In: G. Corliss, C. Faure, A. Griewank, L. Hascoët, U. Naumann (eds.) Automatic Differentiation of Algorithms: From Simulation to Optimization, Computer and Information Science,  chap. 35, pp. 299–304. Springer, New York, NY (2002)
  6. 6.
    Hovland, P.D., Bischof, C.H., Spiegelman, D., Casella, M.: Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics. SIAM Journal on Scientific Computing 18(4), 1056–1066 (1997). DOI 10.1137/S1064827595281800. URL Google Scholar
  7. 7.
    Leppkes, K., Lotz, J., Naumann, U.: dco/c++ – derivative code by overloading in C++. Tech. Rep. AIB-2011-05, RWTH Aachen (2011)Google Scholar
  8. 8.
    Naumann, U.: Optimal accumulation of Jacobian matrices by elimination methods on the dual computational graph. Math. Prog. 99(3), 399–421 (2004). DOI 10.1007/s10107-003-0456-9Google Scholar
  9. 9.
    Naumann, U.: The Art of Differentiating Computer Programs. An Introduction to Algorithmic Differentiation. SIAM (2011)Google Scholar
  10. 10.
    Nocedal, J., Wright, S.: Numerical optimization, series in operations research and financial engineering (2006)Google Scholar
  11. 11.
    Rodgers, C.: Inverse Methods for Atmospheric Sounding. World Scientific (2000)Google Scholar
  12. 12.
    Tadjouddine, M., Forth, S., Qin, N.: Elimination ad applied to jacobian assembly for an implicit compressible cfd solver. International journal for numerical methods in fluids 47(10–11), 1315–1321 (2005)Google Scholar
  13. 13.
    Tadjouddine, M., Forth, S.A., Keane, A.J.: Adjoint differentiation of a structural dynamics solver. In: H.M. Bücker, G. Corliss, P. Hovland, U. Naumann, B. Norris (eds.) Automatic Differentiation: Applications, Theory, and Implementations, Lecture Notes in Computational Science and Engineering, vol. 50, pp. 309–319. Springer, New York, NY (2005). DOI 10.1007/3-540-28438-9{ _}27Google Scholar
  14. 14.
    Tikhonov, A.N., Arsenin, V.Y.: Solutions of ill-posed problems. Winston, Washington D.C., USA (1977)Google Scholar
  15. 15.
    Ungermann, J., Blank, J., Lotz, J., Leppkes, K., Hoffmann, L., Guggenmoser, T., Kaufmann, M., Preusse, P., Naumann, U., Riese, M.: A 3-D tomographic retrieval approach with advection compensation for the air-borne limb-imager GLORIA. Atmos. Meas. Tech. 4(11), 2509–2529 (2011). DOI 10.5194/amt-4-2509-2011Google Scholar
  16. 16.
    Ungermann, J., Hoffmann, L., Preusse, P., Kaufmann, M., Riese, M.: Tomographic retrieval approach for mesoscale gravity wave observations by the premier infrared limb-sounder. Atmos. Meas. Tech. 3(2), 339–354 (2010). DOI 10.5194/amt-3-339-2010Google Scholar
  17. 17.
    Ungermann, J., Kalicinsky, C., Olschewski, F., Knieling, P., Hoffmann, L., Blank, J., Woiwode, W., Oelhaf, H., Hösen, E., Volk, C.M., Ulanovsky, A., Ravegnani, F., Weigel, K., Stroh, F., Riese, M.: CRISTA-NF measurements with unprecedented vertical resolution during the RECONCILE aircraft campaign. Atmos. Meas. Tech. 4(6), 6915–6967 (2011). DOI 10.5194/amtd-4-6915-2011Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Lehr- und Forschungsgebiet Informatik 12: Software and Tools for Computational EngineeringRWTH Aachen UniversityAachenGermany
  2. 2.Institute of Energy and Climate Research – Stratosphere (IEK-7)Research Center Jülich GmbHJülichGermany

Personalised recommendations