Advertisement

Journal of Optimization Theory and Applications

, Volume 4, Issue 5, pp 303–320 | Cite as

Multiplier and gradient methods

  • Magnus R. Hestenes
Survey Paper

Abstract

The main purpose of this paper is to suggest a method for finding the minimum of a functionf(x) subject to the constraintg(x)=0. The method consists of replacingf byF=f+λg+1/2cg2, wherec is a suitably large constant, and computing the appropriate value of the Lagrange multiplier. Only the simplest algorithm is presented. The remaining part of the paper is devoted to a survey of known methods for finding unconstrained minima, with special emphasis on the various gradient techniques that are available. This includes Newton's method and the method of conjugate gradients.

Keywords

Lagrange Multiplier Conjugate Gradient Gradient Method Simple Algorithm Unconstrained Minimum 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Hestenes, M. R.,A General Problem in the Calculus of Variations with Applications to Paths of Least Time, The RAND Corporation, Research Memorandum No. RM-100, 1950.Google Scholar
  2. 2.
    Mengel, A. S.,Optimum Trajectories, The RAND Corporation, Report No. P-199, 1951.Google Scholar
  3. 3.
    Hestenes, M. R.,Numerical Methods for Obtaining Solutions of Fixed-Endpoint Problems in the Calculus of Variations, The RAND Corporation, Research Memorandum No. RM-102, 1949.Google Scholar
  4. 4.
    Stein, M. L.,On Methods for Obtaining Solutions of Fixed-Endpoint Problems in the Calculus of Variations, Journal of Research of the National Bureau of Standards, Vol. 50, No. 5, 1953.Google Scholar
  5. 5.
    Hestenes, M. R.,Iterative Computational Methods, Communications on Pure and Applied Mathematics, Vol. 8, No. 1, 1955.Google Scholar
  6. 6.
    Balakrishnan, A. V.,On a New Computing Technique in Optimal Control and Its Application to Minimal-Time Flight Profile Optimization, Journal of Optimization Theory and Applications, Vol. 4, No. 1, 1969.Google Scholar
  7. 7.
    Hestenes, M. R.,An Indirect Sufficiency Proof for the Problem of Bolza in Nonparametric Form, Transactions of the American Mathematical Society, Vol. 62, No. 3, 1947.Google Scholar
  8. 8.
    Hestenes, M. R., andStiefel, E.,Methods of Conjugate Gradients for Solving Linear Systems, Journal of Research of the National Bureau of Standards, Vol. 49, No. 6, 1952.Google Scholar
  9. 9.
    Hestenes, M. R.,The Conjugate Gradient Method for Solving Linear Systems, Proceedings of the Sixth Symposium in Applied Mathematics, Edited by J. H. Curtiss, American Mathematical Society, Providence, Rhode Island, 1956.Google Scholar
  10. 10.
    Hayes, R. M.,Iterative Methods for Solving Linear Problems in Hilbert Space, Contributions to the Solutions of Systems of Linear Equations and the Determinations of Eigenvalues, Edited by O. Tausky, National Bureau of Standards, Applied Mathematics Series, US Government Printing Office, Washington, D.C., 1954.Google Scholar
  11. 11.
    Fletcher, R., andPowell, M. J. D.,A Rapidly Convergent Descent Method for Minimization, Computer Journal, Vol. 6, No. 2, 1964.Google Scholar
  12. 12.
    Myers, G. E.,Properties of the Conjugate-Gradient and Davidon Methods, Journal of Optimization Theory and Applications, Vol. 2, No. 4, 1968.Google Scholar
  13. 13.
    Horwitz, L. B., andSarachick, P. E.,Davidon's Method in Hilbert Space, SIAM Journal on Applied Mathematics, Vol. 16, No. 4, 1968.Google Scholar

Copyright information

© Plenum Publishing Corporation 1969

Authors and Affiliations

  • Magnus R. Hestenes
    • 1
  1. 1.Department of MathematicsUniversity of California at Los AngelesLos Angeles

Personalised recommendations