Mathematical Programming and Control Theory pp 119-146 | Cite as

# Some algorithms for nonlinear optimization

- 105 Downloads

## Abstract

Many algorithms have been proposed for computing the optimum (or optima) of a mathematical programming problem, but there is no universal method. The simplex method for linear programming (3.3) is a highly efficient algorithm; while the number of iterations required to reach an optimum varies widely from one problem to another, the *average* number of iterations, for problems with constraints *Ax* — *b* ∈ ∝m/+ with *A* an *m* x *n* matrix with *n* much greater than *m*, is of the order of 2*m*, much less than might be expected from the number of vertices of the constraint set. (This remark does not apply to programming restricted to integer values.) If a nonlinear programming problem can be arranged so as to be solvable by a modified simplex method, this is commonly the most efficient procedure. In particular, a problem which allows an adequate approximation by piece- wise linear functions, of not too many variables, may be computed as a *separable programming* problem (3.4). Also a problem with a quadratic objective and linear constraints can be solved by a modified simplex method (4.6 and 7.6).

## Keywords

Optimal Control Problem Mathematical Program Penalty Function Nonlinear Programming Problem Unconstrained Minimization## Preview

Unable to display preview. Download preview PDF.

## References

- Abadie, J. (Ed.) (1967),
*Nonlinear Programming*, North-Holland, Amsterdam.Google Scholar - Abadie, J. (Ed.) (1970),
*Integer and Nonlinear Programming*, North-Holland, Amsterdam.Google Scholar - Beale, E.M.L. (1967), Numerical methods, Chapter 7 of Abadie (1967).Google Scholar
- Beale, E.M.L. (1970), Advanced algorithmic features for general mathematical programming systems, Chapter 4 of Abadie (1970).Google Scholar
- Bennett, J.M. (1966), An approach to some structured linear programming problems,
*Operations Research*,**14**, 636–645.CrossRefGoogle Scholar - Box, M.J., Davies, D., and Swann, W.H. (1969),
*Non-linear Optimization Techniques*, Oliver and Boyd, Edinburgh. (I.C.I. Mathematical and Statistical Techniques for Industry, Monograph No. 5.)Google Scholar - Broyden, C.G. (1972), Quasi-Newton methods, Chapter 6 of Murray (1972).Google Scholar
- Dantzig, G.B., and Wolfe, P. (1960), A decomposition principle for linear programs,
*Operations Research*,**8**, 101–111.CrossRefGoogle Scholar - Fiacco, A.V., and McCormick, G.P. (1968),
*Nonlinear Programming: Sequential Unconstrained Minimization Techniques*, Wiley, New York.Google Scholar - Gill, P.E., and Murray, W. (Eds.) (1974),
*Numerical Methods for Constrained Optimization*, Academic Press, London.Google Scholar - Lootsma, F.A. (1969), Hessian matrices of penalty functions for solving constrained minimization problems,
*Philips Res. Reports*,**24**, 332–330.Google Scholar - Luenberger, D.G. (1969),
*Optimization by Vector Space Methods*, Wiley, New York.Google Scholar - Luenberger, D.G. (1973),
*Introduction to Linear and Nonlinear Programming*, Addison-Wesley, Reading.Google Scholar - Mangasarian, O.L. (1974),
*Nonlinear Programming, Theory and Computation*, Chapter 6 of Elmaghraby, S.E., and Moder, J.J. (Eds.),*Handbook of Operations Research*, Van Nostrand Reinhold, New York.Google Scholar - Mayne, D.Q., and Polak, E. (1975), First-order strong variation algorithms for optimal control,
*J. Optim. Theor. Appl.*,**16**, 277–301.CrossRefGoogle Scholar - Miele, A. (1975), Recent advances in gradient algorithms for optimal control problems,
*J. Optim. Theor. Appl.*,**17**, 361–430.CrossRefGoogle Scholar - McCormick, G.P., and Pearson, J.D. (1969), Variable metric methods and unconstrained optimization, in Fletcher, R. (Ed.),
*Optimization*, Academic Press, London and New York.Google Scholar - Murray, W. (ed.) (1972),
*Numerical Methods for Unconstrained Optimization*, Academic Press, London.Google Scholar - Pietrzykowski, T. (1969), An exact potential method for constrained maximum,
*SIAMJ. Numer. Anal.*,**6**, 299–304.CrossRefGoogle Scholar - Polak, E. (1971),
*Computational Methods in Optimization*, Academic Press, New York.Google Scholar - Rockafellar, R.T. (1974), Augmented Lagrange multiplier functions and duality in non-convex programming,
*SIAM J. Control*,**12**, 268–287.CrossRefGoogle Scholar - Rosen, J.B. (1963), in Graves, R.L., and Wolfe, P.,
*Recent Advances in Mathematical Programming*, McGraw-Hill, New York, 159–176.Google Scholar - Rosen, J.B., and Ornes, J.C. (1963), Solution of nonlinear programming problems by partitioning,
*Management Science*,**10**, 160–173.CrossRefGoogle Scholar - Teo, K.-L. (1977), Computational methods of optimal control problems with application to decision making of a business firm, University of New South Wales, Private communication.Google Scholar
- Teo, K.-L., Reid, D.W., and Boyd, I.E. (1977), Stochastic optimal control theory and its computational methods, University of New South Wales, Research Report.Google Scholar
- Wolfe, P. (1967), Chapter 6 of Beale (1967).Google Scholar