Abstract
The paper discusses several versions of the method of shortest residuals, a specific variant of the conjugate gradient algorithm, first introduced by Lemaréchal and Wolfe and discussed by Hestenes in a quadratic case. In the paper we analyze the global convergence of the versions considered. Numerical comparison of these versions of the method of shortest residuals and an implementation of a standard Polak–Ribière conjugate gradient algorithm is also provided. It supports the claim that the method of shortest residuals is a viable technique, competitive to other conjugate gradient algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)
Polak, E., Ribiére, C.: Note sur la convergence de methodes de directions conjuguées. Rev. Fr. Inform. Rech. Oper. 16, 35–43 (1969)
Shanno, D.F.: On the convergence of a new conjugate gradient algorithm. SIAM J. Numer. Anal. 15, 1247–1257 (1978)
Dai, Y., Yuan, Y.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10, 177–182 (1999)
Powell, M.J.D.: Restart procedures for the conjugate gradient method. Math. Program. 12, 241–254 (1977)
Hestenes, M.: Conjugate Direction Methods in Optimization. Springer, New York (1980)
Lemaréchal, C.: An extension of Davidson methods to nondifferentiable problem. In: Balinski, M.L., Wolfe, P. (eds.) Mathematical Programming Study 3, pp. 95–109. North-Holland, Amsterdam (1975)
Wolfe, P.: A method of conjugate subgradients for minimizing nondifferentiable functions. In: Balinski, M.L., Wolfe, P. (eds.) Mathematical Programming Study 3, pp. 145–173 North-Holland, Amsterdam (1975)
Pytlak, R.: On the convergence of conjugate gradient algorithms. IMA J. Numer. Anal. 14, 443–460 (1994)
Dai, Y., Yuan, Y.: Global convergence of the method of shortest residuals. Numer. Math. 83, 581–598 (1999)
Al-Baali, M.: Descent property and global convergence of the Fletcher–Reeves method with inexact line search. IMA J. Numer. Anal. 5, 121–124 (1985)
Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2, 21–42 (1992)
Bongartz, I., Conn, A.R., Gould, N.I.M., Toint, P.L.: CUTE: constrained and unconstrained testing environment. Research Report RC 18860, IBM T.J. Watson Research Center, Yorktown Heights, NY (1994)
Moré, J.J., Thuente, D.J.: Line search algorithms with guaranteed sufficient decrease. Trans. Math. Softw. 20, 286–307 (1994)
Pytlak, R., Tarnawski, T.: Preconditioned conjugate gradient algorithms for nonconvex problems. In: Proceedings of 43rd IEEE Conference on Decision and Control, The Bahamas, pp. 3191–3196 (2004)
Pytlak, R.: An efficient algorithm for large-scale nonlinear programming problems with simple bounds on the variables. SIAM J. Optim. 8, 532–560 (1998)
Pytlak, R., Tarnawski, T.: Preconditioned conjugate gradient algorithms for large scale problems with box constraints. Proceedings of 44rd IEEE Conference on Decision and Control, Seville, pp. 5138–5143 (2005)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by O.V. Mangasarian.
Rights and permissions
About this article
Cite this article
Pytlak, R., Tarnawski, T. On the Method of Shortest Residuals for Unconstrained Optimization. J Optim Theory Appl 133, 99–110 (2007). https://doi.org/10.1007/s10957-007-9194-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-007-9194-0