Efficient Computation of Sparse Higher Derivative Tensors

  • Jens DeussenEmail author
  • Uwe Naumann
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11536)


The computation of higher derivatives tensors is expensive even for adjoint algorithmic differentiation methods. In this work we introduce methods to exploit the symmetry and the sparsity structure of higher derivatives to considerably improve the efficiency of their computation. The proposed methods apply coloring algorithms to two-dimensional compressed slices of the derivative tensors. The presented work is a step towards feasibility of higher-order methods which might benefit numerical simulations in numerous applications of computational science and engineering.


Adjoints Algorithmic differentiation Coloring algorithm Higher derivative tensors Recursive coloring Sparsity 


  1. 1.
    Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. SIAM (2008)Google Scholar
  2. 2.
    Naumann, U.: The Art of Differentiating Computer Programs: An Introduction to Algorithmic Differentiation, SE24 in Software, Environments and Tools. SIAM (2012)Google Scholar
  3. 3.
    Gundersen, G., Steihaug, T.: Sparsity in higher order methods for unconstrained optimization. Optim. Methods Softw. 27, 275–294 (2012)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Ederington, L.H., Guan, W.: Higher order greeks. J. Deriv. 14, 7–34 (2007)CrossRefGoogle Scholar
  5. 5.
    Smith, R.C.: Uncertainty Quantification: Theory, Implementation, and Applications. SIAM (2013)Google Scholar
  6. 6.
    Christianson, B., Cox, M.: Automatic propagation of uncertainties. In: Bücker, M., Corliss, G., Naumann, U., Hovland, P., Norris, B. (eds.) Automatic Differentiation: Applications, Theory, and Implementations. LNCSE, vol. 50, pp. 47–58. Springer, Heidelberg (2006). Scholar
  7. 7.
    Putko, M.M., Taylor, A.C., Newman, P.A., Green, L.L.: Approach for input uncertainty propagation and robust design in CFD using sensitivity derivatives. J. Fluids Eng. 124, 60–69 (2002)CrossRefGoogle Scholar
  8. 8.
    Griewank, A., Utke, J., Walther, A.: Evaluating higher derivative tensors by forward propagation of univariate Taylor series. Math. Comput. 69, 1117–1130 (2000)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Gower, R.M., Gower, A.L.: Higher-order reverse automatic differentiation with emphasis on the third-order. Math. Program. 155, 81–103 (2016)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Jones, M.T., Plassmann, P.E.: Scalable iterative solution of sparse linear systems. Parallel Comput. 20, 753–773 (1994)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Gebremedhin, A.H., Manne, F., Pothen, A.: What color is your Jacobian? Graph coloring for computing derivatives. SIAM Rev. 47, 629–705 (2005)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Boisvert, R.F., Pozo, R., Remington, K., Barrett, R.F., Dongarra, J.J.: Matrix Market: a web resource for test matrix collections. In: Boisvert, R.F. (ed.) Quality of Numerical Software. IFIPAICT, pp. 125–137. Springer, Boston (1997). Scholar
  13. 13.
    Hascoët, L., Naumann, U., Pascual, V.: “To be recorded” analysis in reverse-mode automatic differentiation. FGCS 21, 1401–1417 (2005)CrossRefGoogle Scholar
  14. 14.
    Coleman, T.F., Moré, J.J.: Estimation of sparse Hessian matrices and graph coloring problems. Math. Program. 28, 243–270 (1984)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Coleman, T.F., Cai, J.-Y.: The cyclic coloring problem and estimation of sparse Hessian matrices. SIAM J. Algebraic Discrete Methods 7, 221–235 (1986)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Gay, D.M.: More AD of nonlinear AMPL models: computing Hessian information and exploiting partial separability. In: Computational Differentiation: Applications, Techniques, and Tools, pp. 173–184. SIAM (1996)Google Scholar
  17. 17.
    Gower, R.M., Mello, M.P.: Computing the sparsity pattern of Hessians using automatic differentiation. ACM TOMS 40, 10:1–10:15 (2014)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Gebremedhin, A.H., Nguyen, D., Patwary, M.M.A., Pothen, A.: ColPack: software for graph coloring and related problems in scientific computing. ACM TOMS 40, 1:1–1:31 (2013)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Software and Tools for Computational EngineeringRWTH Aachen UniversityAachenGermany

Personalised recommendations