# Truncated-newtono algorithms for large-scale unconstrained optimization

- 505 Downloads
- 191 Citations

## Abstract

We present an algorithm for large-scale unconstrained optimization based onNewton's method. In large-scale optimization, solving the Newton equations at each iteration can be expensive and may not be justified when far from a solution. Instead, an inaccurate solution to the Newton equations is computed using a conjugate gradient method. The resulting algorithm is shown to have strong convergence properties and has the unusual feature that the asymptotic convergence rate is a user specified parameter which can be set to anything between linear and quadratic convergence. Some numerical results on a 916 vriable test problem are given. Finally, we contrast the computational behavior of our algorithm with Newton's method and that of a nonlinear conjugate gradient algorithm.

### Key words

Unconstrained Optimization Modified Newton Methods Conjugate Gradient Algorithms## Preview

Unable to display preview. Download preview PDF.

### References

- [1]O. Axelsson, “Solution of linear systems of equations: Iterative methods”, in: V.A. Barker, ed.,
*Sparse matrix techniques*. (Springer-Verlag, New York, 1976) pp. 1–51.Google Scholar - [2]R. Chandra, “Conjugate gradient methods for partial differential equations”, Ph.D. dissertation, Yale University (New Haven, CT, 1978), Also available as Department of Computer Science Research Report No. 129.Google Scholar
- [3]R. Chandra, S.C. Eisenstat and M.H. Schultz, “The modified conjugate residual method for partial differential equations”, in: R. Vichnevetsky, ed.,
*Advance in computer methods for partial differential equations II*, Proceedings of the Second International Symposium on Computer Methods for Partial Differential Equations, Lehigh University, Bethlehem, PA (International Association for Mathematics and Computers in Simulation, June 1977) pp. 13–19.Google Scholar - [4]A.R. Curtis, M.J.D. Powell and J.K. Reid, “On the estimation of sparse Jacobian matrices”,
*Journal of the Institute of Mathematics and its Applications***13**(1974) 117–119.MATHGoogle Scholar - [5]R.S. Dembo, S.C. Eisenstat and T. Steihaug, “Inexact Newton methods”,
*SIAM Journal of Numerical Analysis***19**(1982) 400–408.MATHCrossRefMathSciNetGoogle Scholar - [6]R.S. Dembo and J.G. Klincewicz, “A scaled reduced gradient algorithm for network flow problems with convex separable costs”,
*Mathematical Programming Studies***15**(1981) 125–147.MATHMathSciNetGoogle Scholar - [7]R. S. Dembo and T. Steihaug, “A test problem for large-scale unconstrained minimization”, School of Organization and Management, Yale University (New Haven, CT) Working Paper Series B (in preparation).Google Scholar
- [8]J.E. Dennis Jr. and J.J. Moré, “Quasi-Newton methods, motivation and theory”,
*SIAM Review***19**(1977) 46–89.MATHCrossRefMathSciNetGoogle Scholar - [9]R. Fletcher, “Conjugate gradient methods for indefinite systems”, in: G.A. Watson, ed.,
*Numerical Analysis*, Proceedings of Biennial Conference, Dundee, Scotland, 1975 (Springer-Verlag, New York, 1976) pp. 73–89.CrossRefGoogle Scholar - [10]R. Fletcher,
*Unconstrained optimization*(John Wiley and Sons, New York, 1980).MATHGoogle Scholar - [11]N.K. Garg and R.A. Tapia, “QDN: A variable storage algorithm for unconstrained optimization”, Technical Report, Department of Mathematical Sciences, Rice University (Houston, TX, 1977).Google Scholar
- [12]P.E. Gill and W. Murray, “Safeguarded steplength algorithm for optimization using descent methods”, Technical Report NPL NA 37, National Physical Laboratory (1974).Google Scholar
- [13]P.E. Gill, W. Murray and S.G. Nash, “A conjugate-gradient approach to Newton-type methods”, presented at ORSA/TIMS Joint National Meeting (Colorado Springs, November 1980).Google Scholar
- [14]P.E. Gill, W. Murray and M.H. Wright
*Practical optimization*(Academic Press, New York, 1981).MATHGoogle Scholar - [15]M.R. Hestenes,
*Conjugate direction methods in optimization*(Springer-Verlag New York, 1980).MATHGoogle Scholar - [16]M.R. Hestenes and E. Stiefel, “Methods of conjugate gradients for solving linear systems”,
*Journal of Research of the National Bureau of Standards*49 (1952) 409–436.MATHMathSciNetGoogle Scholar - [17]D.G. Luenberger, “Hyperbolic pairs in the method of conjugate gradients”,
*SIAM Journal on Applied Mathematics*17 (1969) 1263–1267.MATHCrossRefMathSciNetGoogle Scholar - [18]B. Murtagh and M. Saunders, “Large-scale linearly constrained optimization”,
*Mathematical Programming*14 (1978) 41–72.MATHCrossRefMathSciNetGoogle Scholar - [19]J. Nocedal, “Updating Quasi-Newton matrices with limited storage”,
*Mathematics of Computation*35 (1980) 773–782.MATHCrossRefMathSciNetGoogle Scholar - [20]D.P. O'Leary, “A discrete Newton algorithm for minimizing a function of many variables”,
*Mathematical Programming*23 (1982) 20–33.MATHCrossRefMathSciNetGoogle Scholar - [21]J.M. Ortega and W.C. Rheinboldt
*Iterative solution of nonlinear equations in several variables*(Academic Press, New York, 1970).MATHGoogle Scholar - [22]M.J.D. Powell and Ph.L. Toint, “On the estimation of sparse Hessian matrices”,
*SIAM Journal on Numerical Analysis*16 (1979) 1060–1074.MATHCrossRefMathSciNetGoogle Scholar - [23]D.F. Shanno, “On the convergence of a new conjugate gradient method”,
*SIAM Journal on Numerical Analysis*15 (1978) 1247–1257.MATHCrossRefMathSciNetGoogle Scholar - [24]D.F. Shanno and K.H. Phua, “Algorithm 500: Minimization of unconstrained multivariate functions”,
*Transactions on Mathematical Software*2 (1976) 87–94.MATHCrossRefGoogle Scholar - [25]T. Steihaug, “Quasi-Newton methods for large scale nonlinear problems”, Ph.D. dissertation, Yale University (New Haven, CT, 1980). Also available as Working Paper, Series B No. 49.Google Scholar
- [26]Ph.L. Toint, “Some numerical results using a sparse matrix updating formula in unconstrained optimization”,
*Mathematics of Computation*32 (1978) 839–851.MATHCrossRefMathSciNetGoogle Scholar