Abstract
In this article, based on the modified secant equation, we propose a modified Hestenes-Stiefel (HS) conjugate gradient method which has similar form as the CG-DESCENT method proposed by Hager and Zhang (SIAM J Optim 16:170–192, 2005). The presented method can generate sufficient descent directions without any line search. Under some mild conditions, we show that it is globally convergent with Armijo line search. Moreover, the R-linear convergence rate of the modified HS method is established. Preliminary numerical results show that the proposed method is promising, and competitive with the well-known CG-DESCENT method.
Similar content being viewed by others
References
Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10, 147–161 (2008)
Broyden, C.G., Dennis, J.E., Moré, J.J.: On the local and superlinear convergence of quasi-Newton methods. J. Inst. Math. Appl. 12, 223–246 (1973)
Byrd, R., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconst rained minimization. SIAM J. Numer. Anal. 26, 727–739 (1989)
Byrd, R., Nocedal, J., Yuan, Y.: Global convergence of a class of quasi-Newton methods on convex problems. SIAM J. Numer. Anal. 24, 1171–1189 (1987)
Bongartz, K.E., Conn, A.R., Gould, N.I.M., Toint, P.L.: CUTE: constrained and unconstrained testing environments. ACM Trans. Math. Softw. 21, 123–160 (1995)
Cheng, W.Y.: A two-term PRP-based descent method. Numer. Funct. Anal. Optim. 28, 1217–1230 (2007)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)
Dai, Y.H.: Convergence properties of the BFGS algorithm. SIAM J. Optim. 13, 693–701 (2003)
Dai, Y.H., Yuan, Y.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10, 177–182 (2000)
Dai, Z.F., Tian, B.S.: Global convergence of some modified PRP nonlinear conjugate gradient methods. Opt. Lett. (2010). doi:10.1007/s11590-010-0224-8
Fletcher, R., Reeves, C.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)
Fletcher, R.: Practical Methods of Optimization, vol. I: Unconstrained Optimization. Wiley, New York (1987)
Hestenes, M.R., Stiefel, E.L.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand., B 49, 409–432 (1952)
Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM. J. Optim. 2, 21–42 (1992)
Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 (2005)
Hager, W.W., Zhang, H.: A survey of nonlinear conjugate gradient methods. Pacific J. Optim. 2, 35–58 (2006)
Liu, Y.L., Storey, C.S.: Efficient generalized conjugate gradient algorithms, part 1: theory. J. Optim. Theory Appl. 69, 129–137 (1991)
Li, D.H., Fukushima, M.: A modified BFGS method and its global convergence in nonconvex minimization. J. Comput. Appl. Math. 129, 15–35 (2001)
Polak, B., Ribiére, G.: Note surla convergence des méthodes de directions conjuguées. Rev. Francaise Infomat Recherche Operatonelle, 3e Année 16, 35–43 (1969)
Polyak, B.T.: The conjugate gradient method in extreme problems. USSR Comput. Math. Math. Phys. 9, 94–112 (1969)
Shi, Z.J., Shen, J.: Convergence of the Polak-Ribiére-Polyak conjugate gradient method. Nonlinear Anal. 66, 1428–1441 (2007)
Yu, G.H., Zhao, Y.L., Wei, Z.X.: A descent nonlinear conjugate gradient method for large-scale unconstrained optimization. Appl. Math. Comput. 187, 636–643 (2007)
Yu, G.H., Guan, L.T., Chen, W.: Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization. Optim. Methods Softw. 23, 275–293 (2008)
Yu, G.H., Huang, J.H, Zhou, Y.: A descent spectral conjugate gradient method for impulse noise removal. Appl. Math. Lett. 23, 555–560 (2010)
Yuan, Y.: Numerical Methods for Nonlinear Programming. Shanghai Scientific & Technical Publishers (1993)
Yin, K., Xiao, Y.H., Zhang, M.L.: Nonlinear conjugate gradient method for l 1-norm regularization problems in compressive sensing. J. Comput. Infor. Sys. 7, 880–885 (2011)
Wen, F.H., Yang, X.G.: Skewness of return distribution and coeffcient of risk premium. J. Syst. Sci. Complex. 22, 360–371 (2009)
Wen, F.H., Liu, Z.F.: A copula-based correlation measure and its application in chinese stock market. Int. J. Inf. Technol. Decis. Mak. 8, 1–15 (2009)
Zhang, L., Zhou, W., Li, D.: A descent modified Polak-Ribi-re-Polyak conjugate gradient method and its global convergence. IMA J. Numer. Anal. 26, 629–640 (2006)
Zhang, L., Zhou, W., Li, D.: Some descent three-term conjugate gradient methods and their global convergence. Optim. Methods Softw. 22, 697–711 (2007)
Zoutendijk, G.: Nonlinear programming computational methods. In: Abadie, J. (ed.) Integer and Nonlinear Programming, pp. 37–86. North-Holland, Amsterdam (1970)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by the NSF of China grants (11071087 and 70971013), Hunan Natural Science Foundation (09JJ1010), and the Open Fund Project of Key Research Institute of Philosophies and Social Sciences in Hunan Universities.
Rights and permissions
About this article
Cite this article
Dai, Z., Wen, F. Global convergence of a modified Hestenes-Stiefel nonlinear conjugate gradient method with Armijo line search. Numer Algor 59, 79–93 (2012). https://doi.org/10.1007/s11075-011-9477-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-011-9477-2
Keywords
- Unconstrained optimization
- Conjugate gradient method
- Sufficient descent property
- R-linear convergence
- Global convergence