QNlike variable storage conjugate gradients
 A. Buckley,
 A. Lenir
 … show all 2 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
Both conjugate gradient and quasiNewton methods are quite successful at minimizing smooth nonlinear functions of several variables, and each has its advantages. In particular, conjugate gradient methods require much less storage to implement than a quasiNewton code and therefore find application when storage limitations occur. They are, however, slower, so there have recently been attempts to combine CG and QN algorithms so as to obtain an algorithm with good convergence properties and low storage requirements. One such method is the code CONMIN due to Shanno and Phua; it has proven quite successful but it has one limitation. It has no middle ground, in that it either operates as a quasiNewton code using O(n ^{2}) storage locations, or as a conjugate gradient code using 7n locations, but it cannot take advantage of the not unusual situation where more than 7n locations are available, but a quasiNewton code requires an excessive amount of storage.
In this paper we present a way of looking at conjugate gradient algorithms which was in fact given by Shanno and Phua but which we carry further, emphasize and clarify. This applies in particular to Beale's 3term recurrence relation. Using this point of view, we develop a new combined CGQN algorithm which can use whatever storage is available; CONMIN occurs as a special case. We present numerical results to demonstrate that the new algorithm is never worse than CONMIN and that it is almost always better if even a small amount of extra storage is provided.
 A. Buckley, “A combined conjugate gradient quasiNewton minimization algorithm”,Mathematical Programming 15 (1978) 200–210. CrossRef
 A. Buckley, “Extending the relationship between the conjugate gradient and BFGS algorithms”,Mathematical Programming 15 (1978) 343–348. CrossRef
 A. Buckley, “Conjugate gradient methods”, in: M.J.D. Powell, ed.,Nonlinear Optimization 1981, Proceedings of the NATO Advanced Research Institute on Nonlinear Optimization (Academic Press, London, 1982) pp. 17–22.
 A. Buckley, “A portable package for testing minimization algorithms”, in: John M. Mulvey, ed.,Proceedings of the COAL Conference on Mathematical Programming Software, Boulder, Colorado (Springer, New York, 1982) 226–235.
 R. Fletcher and C.M. Reeves, “Function minimization by conjugate gradients”,Computer Journal 7 (1963) 163–168.
 R. Fletcher, “AFortran subroutine for minimization by the method of conjugate gradients”, Report R7073, U.K. A.E.R.E., Harwell, England (1972).
 L. Nazareth, “A relationship between the BFGS and conjugate gradient algorithms and its implications for new algorithms”,SIAM Journal on Numerical Analysis 16 (1979) 794–800. CrossRef
 J. Nocedal, “Updating quasiNewton matrices with limited storage”,Mathematics of Computation 35 (1980) 773–782. CrossRef
 S.S. Oren and E. Spedicato, “Optimal conditioning of selfscaling variable metric algorithms”,Mathematical Programming 10 (1976) 70–90. CrossRef
 A. Perry, “A modified conjugate gradient algorithm”, Discussion paper 229, Center for Mathematical Studies in Economics and Management Science, Northwestern University (1976).
 M.J.D. Powell, “Restart procedures for the conjugate gradient method”,Mathematical Programming 12 (1977) 241–254. CrossRef
 D.F. Shanno, “Conjugate gradient methods with inexact searches”,Mathematics of Operations Research 3 (1978) 244–256. CrossRef
 D.F. Shanno and K.H. Phua, “Numerical comparison of several variable metric algorithms”,Journal of Optimization Theory and Applications 25 (1978) 507–518. CrossRef
 D.F. Shanno, “Remark on Algorithm 500”,ACM Transactions on Mathematical Sofware 6 (1980) 618–622. CrossRef
 Ph. Toint, “Some numerical results using a sparse matrix updating formula in unconstrained optimization”,Mathematics of Computation 32 (1978) 839–851. CrossRef
 Title
 QNlike variable storage conjugate gradients
 Journal

Mathematical Programming
Volume 27, Issue 2 , pp 155175
 Cover Date
 19831001
 DOI
 10.1007/BF02591943
 Print ISSN
 00255610
 Online ISSN
 14364646
 Publisher
 SpringerVerlag
 Additional Links
 Topics
 Keywords

 Minimization
 Conjugate Gradient
 QuasiNewton
 Variable Storage
 Reduced Storage
 Industry Sectors
 Authors

 A. Buckley ^{(1)}
 A. Lenir ^{(1)}
 Author Affiliations

 1. Mathematics Department, Concordia University, Montreal, Quebec, Canada