Find out how to access previewonly content
On conjugate gradientlike methods for eigenlike problems
 Alan Edelman,
 Steven T. Smith
 … show all 2 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
Numerical analysts, physicists, and signal processing engineers have proposed algorithms that might be called conjugate gradient for problems associated with the computation of eigenvalues. There are many variations, mostly one eigenvalue at a time though sometimes block algorithms are proposed. Is there a correct “conjugate gradient” algorithm for the eigenvalue problem? How are the algorithms related amongst themselves and with other related algorithms such as Lanczos, the Newton method, and the Rayleigh quotient?
This paper will also appear in the Proceedings of the AMS/IMS/SIAM Joint Summer Research Conference on Linear and Nonlinear Conjugate GradientRelated Methods held in Seattle, 9–13 July 1995.
Supported by a fellowship from the Alfred P. Sloan Foundation and NSF Grant 9404326CCR.
This work was sponsored by DARPA under Air Force contract F1962895C0002. Opinions, interpretations, conclusions, and recommendations are those of the author and are not necessarily endorsed by the United States Air Force.
 B. M. Alsén,Multiple step gradient iterative methods for computing eigenvalues of large symmetric matrices, Tech. Rep. UMINF15.71, University of Umeå, 1971.
 I. Andersson,Experiments with the conjugate gradient algorithm for the determination of eigenvalues of symmetric matrices, Tech. Rep. UMINF4.71, University of Umeå, 1971.
 T. A. Arias, M. C. Payne, and J. D. Joannopoulous,Ab initio molecular dynamics: analytically continued energy functionals and insights into iterative solutions, Physical Review Letters, 71 (1992), pp. 1077–1080.
 W. W. Bradbury and R. Fletcher,New iterative methods for solutions of the eigenproblem, Numer. Math., 9 (1966), pp. 259–267.
 F. Chatelin,Simultaneous Newton's iteration for the eigenproblem, Computing, Suppl., 5 (1984), pp. 67–74.
 F. Chatelin,Eigenvalues of Matrices, John Wiley, New York, 1993.
 H. Chen, T. K. Sarkar, S. A. Dianat, and J. D. Brulé,Adaptive spectral estimation by the conjugate gradient method, IEEE Trans. Acoustics, Speech, and Signal Processing, ASSP34 (1986), pp. 272–284.
 J. K. Cullum,The simultaneous computation of a few of the algebraically largest and smallest eigenvalues of a large, sparse, symmetric matrix, BIT, 18 (1978), pp. 265–275.
 J. K. Cullum and W. E. Donath,A block generalization of the symmetric sstep Lanczos algorithm, Tech. Rep. RC 4845, IBM Research, Yorktown Heights, NY, 1974.
 J. K. Cullum and W. E. Donath,A block Lanczos algorithm for computing the q algebraically largest eigenvalues and a corresponding eigenspace of large, sparse, real symmetric matrices, in Proceedings of the 1974 Conference on Decision and Control, Phoeniz, Arizona, November 1974, pp. 505–509.
 J. K. Cullum and R. A. Willoughby,Lanczos Algorithms for Large Symmetric Eigenvalue Computations, vol. 1, Theory, Birkhauser, Stuttgart, 1985.
 J. W. Demmel,Three methods for refining estimates of invariant subspaces, Computing, 38 (1987), pp. 43–57.
 J. J. Dongarra, C. B. Moler, and J. H. Wilkinson,Improving the accuracy of computed eigenvalues and eigenvectors, SIAM J. Numer. Anal., 20 (1983), pp. 46–58.
 A. Edelman, T. A. Arias, and S. T. Smith,Conjugate gradient on the Stiefel and Grassmann manifolds, submitted to SIAM J. Matrix Anal. Appl.
 R. L. Fox and M. P. Kapoor,A miminimization method for the solution of the eigenproblem arising in structural dynamics, in Proceedings of the Second Conference on Matrix Methods in Structural Mechanics, L. Berke, R. M. Bader, W. J. Mykytow, J. S. Przemieniecki, and M. H. Shirk, eds., WrightPatterson Air Force Base, Ohio, 1969, pp. 271–306.
 I. Fried,Gradient methods for finite element eigenproblems, AIAA J., 7 (1969), pp. 739–741.
 I. Fried,Optimal gradient minimization scheme for finite element eigenproblems, J. Sound Vib., 20 (1972), pp. 333–342.
 Z. Fu and E. M. Dowling,Conjugate gradient eigenstructure tracking for adaptive spectral estimation, IEEE Trans. Signal Processing, 43 (1995), pp. 1151–1160.
 D. R. Fuhrmann and B. Liu,An iterative algorithm for locating the minimal eigenvector of a symmetric matrix, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing, 1984, pp. 45.8.1–4.
 M. Geradin,The computational efficiency of a new minimization algorithm for eigenvalue analysis, J. Sound Vibration, 19 (1971), pp. 319–331.
 M. J. Gillan,Calculation of the vacancy formation energy in aluminium, Journal of Physics, Condensed Matter, 1 (1989), pp. 689–711.
 G. Golub and D. O'Leary,Some history of the conjugate gradient and Lanczos methods, SIAM Review, 31 (1989), pp. 50–102.
 R. HaimiCohen and A. Cohen,Gradienttype algorithms for partial singular value decomposition, IEEE Trans. Pattern. Anal. Machine Intell., PAMI9 (1987), pp. 137–142.
 W. Karush,An iterative method for finding characteristic vectors of a symmetric matrix, Pacific J. Math., 1 (1951), pp. 233–248.
 G. Kresse and J. Hafner,Ab initio molecular dynamics for liquid metals, Physical Review B, (1993), pp. 558–561.
 J. Nocedal,Theory of algorithms for unconstrained optimization, Acta Numerica, (1992), pp. 199–242.
 B. N. Parlett,The Symmetric Eigenvalue Problem, PrenticeHall, Inc., Englewood Cliffs, NJ, 1980.
 M. C. Payne, M. P. Teter, D. C. Allan, T. A. Arias, and J. D. Joannopoulos,Iterative minimization techniques for ab initio totalenergy calculations: molecular dynamics and conjugate gradients, Rev. Mod. Phys, 64 (1992), pp. 1045–1097.
 A. Perdon and G. Gambolati,Extreme eigenvalues of large sparse matrices by Rayleigh quotient and modified conjugate gradients, Comp. Methods Appl. Mech. Engin., 56 (1986), pp. 251–264.
 A. Ruhe,Iterative eigenvalue algorithms for large symmetric matrices, in Numerische Behandlung von Eigenwertaaufgaben Oberwolfach 1972, Intl, Series Numerical Math. Volume 24, 1974, pp. 97–115.
 A. H. Sameh and J. A. Wisniewski,A trace minimization algorithm for the generalized eigenvalue problem, SIAM J. Numer. Anal., 19 (1982), pp. 1243–1259.
 S. T. Smith,Geometric Optimization Methods for Adaptive Filtering, PhD thesis, Harvard University, Cambridge, MA, 1993.
 S. T. Smith,Optimization techniques on Riemannian manifolds, in Hamiltonian and Gradient Flows, Algorithms and Control, A. Bloch, ed., Fields Institute Communications, American Mathematical Society, Providence, RI, 3 (1994), pp. 113–146.
 G. W. Stewart,Error and perturbation bounds for subspaces associated with certain eigenvalue problems, SIAM Review, 15 (1973), pp. 752–764.
 I. Štich, R. Car, M. Parrinello, and S. Baroni,Conjugate gradient minimization of the energy functional: a new method for electronic structure calculation, Phys. Rev. B., 39 (1989), pp. 4997–5004.
 M. W. Sung, R. Kawai, and J. Weare,Packing transitions in nanosized Liclusters, Physical Review Letter, 73 (1994), pp. 3552–3555.
 M. P. Teter, M. C. Payne, and D. C. Allan,Solution of Schrödinger's equation for large systems, Phys. Review B, 40 (1989), pp. 12255–12263.
 M. A. Townsend and G. E. Johnson,In favor of conjugate directions: a generalized acceptablepoint algorithm for function minimization, J. Franklin Inst., 306 (1978).
 X. Yang, T. P. Sarkar, and E. Arvas,A survey of conjugate gradient algorithms for solution of extreme eigenproblems of a symmetric matrix, IEEE Trans. Acoust., Speech, Signal Processing, 37 (1989), pp. 1550–1556.
 Title
 On conjugate gradientlike methods for eigenlike problems
 Journal

BIT Numerical Mathematics
Volume 36, Issue 3 , pp 494508
 Cover Date
 19960901
 DOI
 10.1007/BF01731929
 Print ISSN
 00063835
 Online ISSN
 15729125
 Publisher
 Kluwer Academic Publishers
 Additional Links
 Topics
 Keywords

 Conjugate gradient
 Lanczos
 Newton's Method
 optimization
 signal processing
 electronic structures
 differential geometry
 Industry Sectors
 Authors

 Alan Edelman ^{(1)}
 Steven T. Smith ^{(2)}
 Author Affiliations

 1. Department of Mathematics, Massachusetts Institute of Technology, 02139, Cambridge, MA, USA
 2. Lincoln Laboratory, Massachusetts Institute of Technology, 02173, Lexington, MA, USA