Abstract
It has been shown by a large amount of numerical experiments that among the local algorithms for solving unconstrained optimization problems, the truncated Newton method with preconditioned conjugate gradient (PCG) subiterations is very efficient. In this paper, we investigate its efficiency from theoretical point of view. The question is, compared with Newton’s method with Cholesky factorization, how much it is more efficient in theory. We give a quantitative answer by constructing a truncated Newton method with PCG subiterations -- Algorithm II below. Suppose Newton’s method is convergent with a one-step, Q-order α rate (α≥ 2). We first prove that Algorithm II has the same convergence rate. We then study its average number of arithmetic operations per step and the corresponding number which Newton’s method needs. Ari upper bound for the ratio of these two numbers is obtained. This upper bound is a quantitative estimate of the saving which Algorithm II can achieve from theoretical point of view. Its values, which are listed in the paper, show that the saving is rather remarkable.
The work is supported by the University Grant Council of Hong Kong under the grant CityU1066/00P and the National Natural Science Foundation of China(Grant No. 10071094).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
K. E. Atkinson, An Introduction to numerical analysis, John Wiley and Sons, New York, Second Edition, 1989.
A. R. Conn, N. I. M. Gould and Ph. L. Toint, Numerical experiments with the LANCELOT package (Release A) for large-scale nonlinear optimization, Technical Report, pp. 92–075, Rutherford Appleton Laboratory, Chilton, England, 1992.
R. Dembo, S. Eisenstat, and T. Steihaug, Inexact Newton method, SIAM Journal on Numerical Analysis, 19 (1982), pp. 400–408.
L. C. W. Dixon and R. C. Price, Numerical experience with the truncated Newton method for unconstrained optimization,Journal of Optimization Theory and Applications, 56(1988), pp. 245–255.
J. E. Dennis and R. B. Schnabel, Numerical methods for unconstrained optimization and nonlinear equations, Prentice Hall, Englewood Cliffs, NJ, 1983.
N. Y. Deng and Z. Z. Wang, Theoretical efficiency of an inexact Newton method, Journal of Optimization Theory and Applications, 105(2000), pp. 97–112.
N. Y. Deng, Y. Xiao and F. J. Zhou, Nonmonoton.ic trust region algorithm, Journal of Optimization Theory and Applications, 76(1993), pp. 259–285.
S. C. Eisenstat and H. F. Walker, Choosing the forcing terms in an inexact Newton method, SIAM Journal on Scientific Computing, 17(1996), pp. 3346.
P. E. Gill, W. Murray and M. H. Wright, Practical optimization, Academic Press, London and New York, 1981.
C. T. Kelley, Iterative methods for linear and nonlinear equations,SIAM, Philadelphia, 1995.
S. G. Nash, Preconditioning of truncated-Newton methods, SIAM Journal on Scientific and Statistical Computing, 6(1985), pp. 559–616.
J. Nocedal, Large scale unconstrained optimization, Report-DEECS, Northwestern University, 1996.
J.M. Ortega and W.C. Rheinboldt, Iterative solution of nonlinear equa-tions in several variables, Academic Press, London, 1970.
[ Ostrowski, A., Solution of Equations and Systems of Equations, Academic Press, New York, 1960.
T. Steihaug, The conjugate gradient method and trust region in large scale optimization,SIAM Journal on Numerical Analysis, 20(1983), pp. 626–637.
Ph. L. Toint, Towards an efficient sparsity exploiting Newton method for minimization, in Sparse matrices and their uses (ed. I.S. Duff), Academic Press, London, 1981, pp. 57–88.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Kluwer Academic Publishers B.V.
About this chapter
Cite this chapter
Zhang, J.Z., Deng, N.Y., Wang, Z.Z. (2003). Efficiency Analysis on a Truncated Newton Method with Preconditioned Conjugate Gradient Technique for Optimization. In: Di Pillo, G., Murli, A. (eds) High Performance Algorithms and Software for Nonlinear Optimization. Applied Optimization, vol 82. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-0241-4_18
Download citation
DOI: https://doi.org/10.1007/978-1-4613-0241-4_18
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4613-7956-0
Online ISBN: 978-1-4613-0241-4
eBook Packages: Springer Book Archive