Survey Paper

Journal of Optimization Theory and Applications

, Volume 4, Issue 5, pp 303-320

First online:

Multiplier and gradient methods

  • Magnus R. HestenesAffiliated withDepartment of Mathematics, University of California at Los Angeles

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access


The main purpose of this paper is to suggest a method for finding the minimum of a functionf(x) subject to the constraintg(x)=0. The method consists of replacingf byF=f+λg+1/2cg 2, wherec is a suitably large constant, and computing the appropriate value of the Lagrange multiplier. Only the simplest algorithm is presented. The remaining part of the paper is devoted to a survey of known methods for finding unconstrained minima, with special emphasis on the various gradient techniques that are available. This includes Newton's method and the method of conjugate gradients.