Mathematical Programming

, Volume 112, Issue 2, pp 427–441 | Cite as

Optimal Jacobian accumulation is NP-complete

  • Uwe NaumannEmail author


We show that the problem of accumulating Jacobian matrices by using a minimal number of floating-point operations is NP-complete by reduction from Ensemble Computation. The proof makes use of the fact that, deviating from the state-of-the-art assumption, algebraic dependences can exist between the local partial derivatives. It follows immediately that the same problem for directional derivatives, adjoints, and higher derivatives is NP-complete, too.


Automatic differentiation Complexity NP-completeness 

Mathematics Subject Classification (2000)

26B10 68Q17 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Personal communication with Steihaug, T. Bergen University, Norway, and Griewank, A. at Humboldt University Berlin (2005)Google Scholar
  2. 2.
    Baur W., Strassen V. (1983) The complexity of partial derivatives. Theoret. Comput. Sci. 22, 317–330zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Berz, M., Bischof, C., Corliss, G., Griewank, A. (eds.) Computational differentiation: techniques, applications, and tools. In: Proceedings Series. SIAM Philadelphia (1996)Google Scholar
  4. 4.
    Bischof, C., Haghighat, M.: Hierarchical approaches to automatic differentiation. In: [3], pp. 82–94Google Scholar
  5. 5.
    Bücker, M., Corliss, G., Hovland, P., Naumann, U., Norris, B. (eds.) Automatic Differentiation: Applications, Theory, and Tools. Lecture Notes in Computational Science and Engineering, vol. 50. Springer, Berlin Heidelberg New York (2005)Google Scholar
  6. 6.
    Corliss G., Faure C., Griewank A., Hascoët L., Naumann U. (eds) (2002) Automatic Differentiation of Algorithms – From Simulation to Optimization. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  7. 7.
    Corliss, G., Griewank, A. (eds.) Automatic Differentiation: Theory, Implementation, and Application. Proceedings Series. SIAM Philadelphia (1991)Google Scholar
  8. 8.
    Garey M., Johnson D. (1979) Computers and Intractability – A Guide to the Theory of NP-completeness. W. H. Freeman and Company, San FranciscozbMATHGoogle Scholar
  9. 9.
    Giering, R., Kaminski, T.: Applying TAF to generate efficient derivative code of Fortran 77-95 programs. In: Proceedings of GAMM 2002, Augsburg, Germany (2002)Google Scholar
  10. 10.
    Griewank A. (2000) Evaluating derivatives. Principles and techniques of algorithmic differentiation Frontiers in Applied Mathematics, vol 19. SIAM, PhiladelphiaGoogle Scholar
  11. 11.
    Griewank A., Juedes D., Utke J. (1996) ADOL–C, a package for the automatic differentiation of algorithms written in C/C++. ACM Trans. Math. Softw. 22(2): 131–167zbMATHCrossRefGoogle Scholar
  12. 12.
    Griewank A., Naumann U. (2003) Accumulating Jacobians as chained sparse matrix products. Math. Prog. 3(95): 555–571CrossRefMathSciNetGoogle Scholar
  13. 13.
    Griewank, A., Reese, S.: On the calculation of Jacobian matrices by the Markovitz rule. In: [7], pp. 126–135Google Scholar
  14. 14.
    Griewank, A., Vogel, O.: Analysis and exploitation of Jacobian scarcity. In: Proceedings of HPSC Hanoi. Springer, Berlin Heidelberg New York (2003)Google Scholar
  15. 15.
    Hascoët, L., Pascual, V.: Tapenade 2.1 user’s guide. Technical report 300, INRIA (2004)Google Scholar
  16. 16.
    Heath M. (1998) Scientific Computing. An Introductory Survey. McGraw-Hill, New YorkGoogle Scholar
  17. 17.
    Kelley, C.: Solving Nonlinear Equations with Newton’s Method. SIAM Philadelphia (2003)Google Scholar
  18. 18.
    Naumann, U.: Elimination techniques for cheap Jacobians. In: [6], chap. 29, pp. 247–253 (2001)Google Scholar
  19. 19.
    Naumann U. (2002) Cheaper Jacobians by simulated annealing. SIAM J. Opt. 13(3): 660–674zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Naumann U. (2004) Optimal accumulation of Jacobian matrices by elimination methods on the dual computational graph. Math. Prog. 3(99): 399–421CrossRefMathSciNetGoogle Scholar
  21. 21.
    Naumann U., Utke J. (2005) Optimality-preserving elimination of linearities in Jacobian accumulation. Electron. Trans. Numer. Anal. (ETNA) 21, 134–150zbMATHMathSciNetGoogle Scholar
  22. 22.
    Naumann, U., Utke, J., Wunsch, C., Hill, C., Heimbach, P., Fagan, M., Tallent, N., Strout, M.: Adjoint code by source transformation with open ad/f. In: Proceedings of the European Conference on Computational Fluid Dynamics (ECCOMAS CFD 2006). TU Delft (2006)Google Scholar
  23. 23.
    Speelpenning, B.: Compiling fast partial derivatives of functions given by algorithms. Ph.D. Thesis, University of Chicago (1980)Google Scholar
  24. 24.
    Walther, A.: Program reversal schedules for single- and multi-processor machines. Ph.D. Thesis, Institute of Scientific Computing, Technical University Dresden (1999)Google Scholar
  25. 25.
    Walther, A., Griewank, A.: New results on program reversals. In: [6], chapt. 28, pp. 237–243. Springer, Berlin Heidelberg New York (2001)Google Scholar

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  1. 1.Software and Tools for Computational Engineering, Department of Computer ScienceRWTH Aachen UniversityAachenGermany

Personalised recommendations