BIT Numerical Mathematics

, Volume 36, Issue 3, pp 494–508

On conjugate gradient-like methods for eigen-like problems

  • Alan Edelman
  • Steven T. Smith


Numerical analysts, physicists, and signal processing engineers have proposed algorithms that might be called conjugate gradient for problems associated with the computation of eigenvalues. There are many variations, mostly one eigenvalue at a time though sometimes block algorithms are proposed. Is there a correct “conjugate gradient” algorithm for the eigenvalue problem? How are the algorithms related amongst themselves and with other related algorithms such as Lanczos, the Newton method, and the Rayleigh quotient?

Key words

Conjugate gradient Lanczos Newton's Method optimization signal processing electronic structures differential geometry 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    B. M. Alsén,Multiple step gradient iterative methods for computing eigenvalues of large symmetric matrices, Tech. Rep. UMINF-15.71, University of Umeå, 1971.Google Scholar
  2. 2.
    I. Andersson,Experiments with the conjugate gradient algorithm for the determination of eigenvalues of symmetric matrices, Tech. Rep. UMINF-4.71, University of Umeå, 1971.Google Scholar
  3. 3.
    T. A. Arias, M. C. Payne, and J. D. Joannopoulous,Ab initio molecular dynamics: analytically continued energy functionals and insights into iterative solutions, Physical Review Letters, 71 (1992), pp. 1077–1080.Google Scholar
  4. 4.
    W. W. Bradbury and R. Fletcher,New iterative methods for solutions of the eigen-problem, Numer. Math., 9 (1966), pp. 259–267.Google Scholar
  5. 5.
    F. Chatelin,Simultaneous Newton's iteration for the eigenproblem, Computing, Suppl., 5 (1984), pp. 67–74.Google Scholar
  6. 6.
    F. Chatelin,Eigenvalues of Matrices, John Wiley, New York, 1993.Google Scholar
  7. 7.
    H. Chen, T. K. Sarkar, S. A. Dianat, and J. D. Brulé,Adaptive spectral estimation by the conjugate gradient method, IEEE Trans. Acoustics, Speech, and Signal Processing, ASSP-34 (1986), pp. 272–284.Google Scholar
  8. 8.
    J. K. Cullum,The simultaneous computation of a few of the algebraically largest and smallest eigenvalues of a large, sparse, symmetric matrix, BIT, 18 (1978), pp. 265–275.Google Scholar
  9. 9.
    J. K. Cullum and W. E. Donath,A block generalization of the symmetric s-step Lanczos algorithm, Tech. Rep. RC 4845, IBM Research, Yorktown Heights, NY, 1974.Google Scholar
  10. 10.
    J. K. Cullum and W. E. Donath,A block Lanczos algorithm for computing the q algebraically largest eigenvalues and a corresponding eigenspace of large, sparse, real symmetric matrices, in Proceedings of the 1974 Conference on Decision and Control, Phoeniz, Arizona, November 1974, pp. 505–509.Google Scholar
  11. 11.
    J. K. Cullum and R. A. Willoughby,Lanczos Algorithms for Large Symmetric Eigenvalue Computations, vol. 1, Theory, Birkhauser, Stuttgart, 1985.Google Scholar
  12. 12.
    J. W. Demmel,Three methods for refining estimates of invariant subspaces, Computing, 38 (1987), pp. 43–57.Google Scholar
  13. 13.
    J. J. Dongarra, C. B. Moler, and J. H. Wilkinson,Improving the accuracy of computed eigenvalues and eigenvectors, SIAM J. Numer. Anal., 20 (1983), pp. 46–58.Google Scholar
  14. 14.
    A. Edelman, T. A. Arias, and S. T. Smith,Conjugate gradient on the Stiefel and Grassmann manifolds, submitted to SIAM J. Matrix Anal. Appl.Google Scholar
  15. 15.
    R. L. Fox and M. P. Kapoor,A miminimization method for the solution of the eigenproblem arising in structural dynamics, in Proceedings of the Second Conference on Matrix Methods in Structural Mechanics, L. Berke, R. M. Bader, W. J. Mykytow, J. S. Przemieniecki, and M. H. Shirk, eds., Wright-Patterson Air Force Base, Ohio, 1969, pp. 271–306.Google Scholar
  16. 16.
    I. Fried,Gradient methods for finite element eigenproblems, AIAA J., 7 (1969), pp. 739–741.Google Scholar
  17. 17.
    I. Fried,Optimal gradient minimization scheme for finite element eigenproblems, J. Sound Vib., 20 (1972), pp. 333–342.Google Scholar
  18. 18.
    Z. Fu and E. M. Dowling,Conjugate gradient eigenstructure tracking for adaptive spectral estimation, IEEE Trans. Signal Processing, 43 (1995), pp. 1151–1160.Google Scholar
  19. 19.
    D. R. Fuhrmann and B. Liu,An iterative algorithm for locating the minimal eigenvector of a symmetric matrix, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing, 1984, pp. 45.8.1–4.Google Scholar
  20. 20.
    M. Geradin,The computational efficiency of a new minimization algorithm for eigenvalue analysis, J. Sound Vibration, 19 (1971), pp. 319–331.Google Scholar
  21. 21.
    M. J. Gillan,Calculation of the vacancy formation energy in aluminium, Journal of Physics, Condensed Matter, 1 (1989), pp. 689–711.Google Scholar
  22. 22.
    G. Golub and D. O'Leary,Some history of the conjugate gradient and Lanczos methods, SIAM Review, 31 (1989), pp. 50–102.Google Scholar
  23. 23.
    R. Haimi-Cohen and A. Cohen,Gradient-type algorithms for partial singular value decomposition, IEEE Trans. Pattern. Anal. Machine Intell., PAMI-9 (1987), pp. 137–142.Google Scholar
  24. 24.
    W. Karush,An iterative method for finding characteristic vectors of a symmetric matrix, Pacific J. Math., 1 (1951), pp. 233–248.Google Scholar
  25. 25.
    G. Kresse and J. Hafner,Ab initio molecular dynamics for liquid metals, Physical Review B, (1993), pp. 558–561.Google Scholar
  26. 26.
    J. Nocedal,Theory of algorithms for unconstrained optimization, Acta Numerica, (1992), pp. 199–242.Google Scholar
  27. 27.
    B. N. Parlett,The Symmetric Eigenvalue Problem, Prentice-Hall, Inc., Englewood Cliffs, NJ, 1980.Google Scholar
  28. 28.
    M. C. Payne, M. P. Teter, D. C. Allan, T. A. Arias, and J. D. Joannopoulos,Iterative minimization techniques for ab initio total-energy calculations: molecular dynamics and conjugate gradients, Rev. Mod. Phys, 64 (1992), pp. 1045–1097.Google Scholar
  29. 29.
    A. Perdon and G. Gambolati,Extreme eigenvalues of large sparse matrices by Rayleigh quotient and modified conjugate gradients, Comp. Methods Appl. Mech. Engin., 56 (1986), pp. 251–264.Google Scholar
  30. 30.
    A. Ruhe,Iterative eigenvalue algorithms for large symmetric matrices, in Numerische Behandlung von Eigenwertaaufgaben Oberwolfach 1972, Intl, Series Numerical Math. Volume 24, 1974, pp. 97–115.Google Scholar
  31. 31.
    A. H. Sameh and J. A. Wisniewski,A trace minimization algorithm for the generalized eigenvalue problem, SIAM J. Numer. Anal., 19 (1982), pp. 1243–1259.Google Scholar
  32. 32.
    S. T. Smith,Geometric Optimization Methods for Adaptive Filtering, PhD thesis, Harvard University, Cambridge, MA, 1993.Google Scholar
  33. 33.
    S. T. Smith,Optimization techniques on Riemannian manifolds, in Hamiltonian and Gradient Flows, Algorithms and Control, A. Bloch, ed., Fields Institute Communications, American Mathematical Society, Providence, RI, 3 (1994), pp. 113–146.Google Scholar
  34. 34.
    G. W. Stewart,Error and perturbation bounds for subspaces associated with certain eigenvalue problems, SIAM Review, 15 (1973), pp. 752–764.Google Scholar
  35. 35.
    I. Štich, R. Car, M. Parrinello, and S. Baroni,Conjugate gradient minimization of the energy functional: a new method for electronic structure calculation, Phys. Rev. B., 39 (1989), pp. 4997–5004.Google Scholar
  36. 36.
    M. W. Sung, R. Kawai, and J. Weare,Packing transitions in nanosized Liclusters, Physical Review Letter, 73 (1994), pp. 3552–3555.Google Scholar
  37. 37.
    M. P. Teter, M. C. Payne, and D. C. Allan,Solution of Schrödinger's equation for large systems, Phys. Review B, 40 (1989), pp. 12255–12263.Google Scholar
  38. 38.
    M. A. Townsend and G. E. Johnson,In favor of conjugate directions: a generalized acceptable-point algorithm for function minimization, J. Franklin Inst., 306 (1978).Google Scholar
  39. 39.
    X. Yang, T. P. Sarkar, and E. Arvas,A survey of conjugate gradient algorithms for solution of extreme eigen-problems of a symmetric matrix, IEEE Trans. Acoust., Speech, Signal Processing, 37 (1989), pp. 1550–1556.Google Scholar

Copyright information

© BIT Foundation 1996

Authors and Affiliations

  • Alan Edelman
    • 1
  • Steven T. Smith
    • 2
  1. 1.Department of MathematicsMassachusetts Institute of TechnologyCambridgeUSA
  2. 2.Lincoln LaboratoryMassachusetts Institute of TechnologyLexingtonUSA

Personalised recommendations