Skip to main content
Log in

Multiplier and gradient methods

  • Survey Paper
  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript


The main purpose of this paper is to suggest a method for finding the minimum of a functionf(x) subject to the constraintg(x)=0. The method consists of replacingf byF=f+λg+1/2cg 2, wherec is a suitably large constant, and computing the appropriate value of the Lagrange multiplier. Only the simplest algorithm is presented. The remaining part of the paper is devoted to a survey of known methods for finding unconstrained minima, with special emphasis on the various gradient techniques that are available. This includes Newton's method and the method of conjugate gradients.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others


  1. Hestenes, M. R.,A General Problem in the Calculus of Variations with Applications to Paths of Least Time, The RAND Corporation, Research Memorandum No. RM-100, 1950.

  2. Mengel, A. S.,Optimum Trajectories, The RAND Corporation, Report No. P-199, 1951.

  3. Hestenes, M. R.,Numerical Methods for Obtaining Solutions of Fixed-Endpoint Problems in the Calculus of Variations, The RAND Corporation, Research Memorandum No. RM-102, 1949.

  4. Stein, M. L.,On Methods for Obtaining Solutions of Fixed-Endpoint Problems in the Calculus of Variations, Journal of Research of the National Bureau of Standards, Vol. 50, No. 5, 1953.

  5. Hestenes, M. R.,Iterative Computational Methods, Communications on Pure and Applied Mathematics, Vol. 8, No. 1, 1955.

  6. Balakrishnan, A. V.,On a New Computing Technique in Optimal Control and Its Application to Minimal-Time Flight Profile Optimization, Journal of Optimization Theory and Applications, Vol. 4, No. 1, 1969.

  7. Hestenes, M. R.,An Indirect Sufficiency Proof for the Problem of Bolza in Nonparametric Form, Transactions of the American Mathematical Society, Vol. 62, No. 3, 1947.

  8. Hestenes, M. R., andStiefel, E.,Methods of Conjugate Gradients for Solving Linear Systems, Journal of Research of the National Bureau of Standards, Vol. 49, No. 6, 1952.

  9. Hestenes, M. R.,The Conjugate Gradient Method for Solving Linear Systems, Proceedings of the Sixth Symposium in Applied Mathematics, Edited by J. H. Curtiss, American Mathematical Society, Providence, Rhode Island, 1956.

  10. Hayes, R. M.,Iterative Methods for Solving Linear Problems in Hilbert Space, Contributions to the Solutions of Systems of Linear Equations and the Determinations of Eigenvalues, Edited by O. Tausky, National Bureau of Standards, Applied Mathematics Series, US Government Printing Office, Washington, D.C., 1954.

    Google Scholar 

  11. Fletcher, R., andPowell, M. J. D.,A Rapidly Convergent Descent Method for Minimization, Computer Journal, Vol. 6, No. 2, 1964.

  12. Myers, G. E.,Properties of the Conjugate-Gradient and Davidon Methods, Journal of Optimization Theory and Applications, Vol. 2, No. 4, 1968.

  13. Horwitz, L. B., andSarachick, P. E.,Davidon's Method in Hilbert Space, SIAM Journal on Applied Mathematics, Vol. 16, No. 4, 1968.

Download references

Author information

Authors and Affiliations


Additional information

The preparation of this paper was sponsored by the U.S. Army Research Office, Grant No. DA-31-124-ARO(D)-355. This paper was presented at the Second International Conference on Computing Methods in Optimization Problems, San Remo, Italy, 1968.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hestenes, M.R. Multiplier and gradient methods. J Optim Theory Appl 4, 303–320 (1969).

Download citation

  • Received:

  • Issue Date:

  • DOI: