QN-like variable storage conjugate gradients
- 115 Downloads
Both conjugate gradient and quasi-Newton methods are quite successful at minimizing smooth nonlinear functions of several variables, and each has its advantages. In particular, conjugate gradient methods require much less storage to implement than a quasi-Newton code and therefore find application when storage limitations occur. They are, however, slower, so there have recently been attempts to combine CG and QN algorithms so as to obtain an algorithm with good convergence properties and low storage requirements. One such method is the code CONMIN due to Shanno and Phua; it has proven quite successful but it has one limitation. It has no middle ground, in that it either operates as a quasi-Newton code using O(n2) storage locations, or as a conjugate gradient code using 7n locations, but it cannot take advantage of the not unusual situation where more than 7n locations are available, but a quasi-Newton code requires an excessive amount of storage.
In this paper we present a way of looking at conjugate gradient algorithms which was in fact given by Shanno and Phua but which we carry further, emphasize and clarify. This applies in particular to Beale's 3-term recurrence relation. Using this point of view, we develop a new combined CG-QN algorithm which can use whatever storage is available; CONMIN occurs as a special case. We present numerical results to demonstrate that the new algorithm is never worse than CONMIN and that it is almost always better if even a small amount of extra storage is provided.
Key wordsMinimization Conjugate Gradient Quasi-Newton Variable Storage Reduced Storage
Unable to display preview. Download preview PDF.
- A. Buckley, “Conjugate gradient methods”, in: M.J.D. Powell, ed.,Nonlinear Optimization 1981, Proceedings of the NATO Advanced Research Institute on Nonlinear Optimization (Academic Press, London, 1982) pp. 17–22.Google Scholar
- A. Buckley, “A portable package for testing minimization algorithms”, in: John M. Mulvey, ed.,Proceedings of the COAL Conference on Mathematical Programming Software, Boulder, Colorado (Springer, New York, 1982) 226–235.Google Scholar
- R. Fletcher, “AFortran subroutine for minimization by the method of conjugate gradients”, Report R7073, U.K. A.E.R.E., Harwell, England (1972).Google Scholar
- A. Perry, “A modified conjugate gradient algorithm”, Discussion paper 229, Center for Mathematical Studies in Economics and Management Science, Northwestern University (1976).Google Scholar