Non-linearly constrained minimization

Abstract

There are two broad approaches to the solution of non-linearly constrained minimization problems. In the first, the objective function is modified so that it has an unconstrained minimum at the minimum of the original constrained problem, or so that this property is obtained at the limit of a sequence of modifications, each accompanied by an unconstrained minimization. The methods used for the unconstrained minimizations are precisely those used for any such problem (see chapter 3). We shall call these techniques transformation methods. When the modifications are performed in sequence, we shall call the methods sequential, otherwise the term exact will be used. The second approach involves linear approximation to the constraints followed by the application of a projection-type method and perhaps a correction procedure to maintain a kind of active set strategy. We shall consequently call methods of this type projection methods Both approaches can be sub-divided according to whether or not the Lagrangian function plays a fundamental role in the minimization process. Broadly speaking, the methods based on the Lagrangian function are the more recent, but we include the others for their historical significance, because they are still often used, and for the insight into, and motivation for, the newer methods that they provide. Thus transformation methods include penalty and barrier function methods and augmented Lagrangian methods, while projection methods include direct projection methods and projected Lagrangian methods. The algorithms are complex and the categorisations are not always clear cut. Often an algorithm described as belonging to one class will be seen to have the flavour of another class as well.