Mathematical Programming

, Volume 28, Issue 3, pp 243–270 | Cite as

Estimation of sparse hessian matrices and graph coloring problems

  • Thomas F. Coleman
  • Jorge J. Moré
Article

Abstract

Large scale optimization problems often require an approximation to the Hessian matrix. If the Hessian matrix is sparse then estimation by differences of gradients is attractive because the number of required differences is usually small compared to the dimension of the problem. The problem of estimating Hessian matrices by differences can be phrased as follows: Given the sparsity structure of a symmetric matrixA, obtain vectorsd 1,d 2, …d p such thatAd 1,Ad 2, …Ad p determineA uniquely withp as small as possible. We approach this problem from a graph theoretic point of view and show that both direct and indirect approaches to this problem have a natural graph coloring interpretation. The complexity of the problem is analyzed and efficient practical heuristic procedures are developed. Numerical results illustrate the differences between the various approaches.

Key words

Graph Coloring Estimation of Hessian Matrices Sparsity Differentiation Numerical Differences NP-Complete Problems Unconstrained Minimization 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. T.F. Coleman and J.J. Moré, “Estimation of sparse Jacobian matrices and graph coloring problems”,SIAM Journal on Numerical Analysis 20 (1983), 187–209.MATHCrossRefMathSciNetGoogle Scholar
  2. T.F. Coleman and J.J. Moré, “Software for estimating sparse Jacobian matrices”, Technical Report ANL-82-37, Argonne National Laboratory (Argonne, Illinois, 1982).Google Scholar
  3. A.R. Curtis, M.J.D. Powell and J.K. Reid, “On the estimation of sparse Jacobian matrices”,Journal of the Institute of Mathematics and its Applications 13 (1974) 117–119.MATHGoogle Scholar
  4. S. Eisenstat, Personal communication (1980).Google Scholar
  5. G.C. Everstine, “A comparison of three resequencing algorithms for the reduction of matrix profile and wavefront”,International Journal on Numerical Methods in Engineering 14 (1979) 837–853.MATHCrossRefGoogle Scholar
  6. M.R. Garey and D.S. Johnson, Computers and intractability (W.H. Freeman, San Francisco, CA, 1979).MATHGoogle Scholar
  7. D.W. Matula, G. Marble and J.D. Isaacson, “Graph coloring algorithms”, in: R. Read, ed.,Graph theory and computing (Academic Press, New York, 1972), pp. 104–122.Google Scholar
  8. D.W. Matula and L.L. Beck, “Smallest-last ordering and clustering and graph coloring algorithms”,Journal of the Association for Computing Machinery 30 (1983) 417–427.MATHMathSciNetGoogle Scholar
  9. S.T. McCormick, “Optimal approximation of sparse Hessians and its equivalence to a graph coloring problem”,Mathematical Programming 26 (1983), 153–171.MATHCrossRefMathSciNetGoogle Scholar
  10. G.N. Newsam and J.D. Ramsdell, “Estimation of sparse Jacobian matrices”,SIAM Journal of Algebraic and Discrete Methods (1983), to appear.Google Scholar
  11. M.J.D. Powell and Ph.L. Toint, “On the estimation of sparse Hessian matrices”,SIAM Journal on Numerical Analysis 16 (1979) 1060–1074.MATHCrossRefMathSciNetGoogle Scholar
  12. G. Szekeres and H.S. Wilf, “An inequality for the chromatic number of a graph”,Journal of Combinatorial Theory 4 (1968) 1–3.MathSciNetGoogle Scholar
  13. M.N. Thapa, “Optimization of unconstrained functions with sparse Hessian matrices: Newton-type methods”, Technical Report SOL 82-8, Systems Optimization Laboratory, Stanford University (Stanford, CA, 1982).Google Scholar

Copyright information

© The Mathematical Programming Society Inc. 1984

Authors and Affiliations

  • Thomas F. Coleman
    • 1
  • Jorge J. Moré
    • 2
  1. 1.Department of Computer ScienceCornell UniversityIthacaUSA
  2. 2.Mathematics and Computer Science DivisionArgonne National LaboratoryArgonneUSA

Personalised recommendations