Numerical Algorithms

, Volume 59, Issue 1, pp 79–93

Global convergence of a modified Hestenes-Stiefel nonlinear conjugate gradient method with Armijo line search

Original Paper


In this article, based on the modified secant equation, we propose a modified Hestenes-Stiefel (HS) conjugate gradient method which has similar form as the CG-DESCENT method proposed by Hager and Zhang (SIAM J Optim 16:170–192, 2005). The presented method can generate sufficient descent directions without any line search. Under some mild conditions, we show that it is globally convergent with Armijo line search. Moreover, the R-linear convergence rate of the modified HS method is established. Preliminary numerical results show that the proposed method is promising, and competitive with the well-known CG-DESCENT method.


Unconstrained optimization Conjugate gradient method Sufficient descent property R-linear convergence Global convergence 

Mathematics Subject Classifications (2010)

90C30 65K05 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10, 147–161 (2008)MATHMathSciNetGoogle Scholar
  2. 2.
    Broyden, C.G., Dennis, J.E., Moré, J.J.: On the local and superlinear convergence of quasi-Newton methods. J. Inst. Math. Appl. 12, 223–246 (1973)CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Byrd, R., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconst rained minimization. SIAM J. Numer. Anal. 26, 727–739 (1989)CrossRefMATHMathSciNetGoogle Scholar
  4. 4.
    Byrd, R., Nocedal, J., Yuan, Y.: Global convergence of a class of quasi-Newton methods on convex problems. SIAM J. Numer. Anal. 24, 1171–1189 (1987)CrossRefMATHMathSciNetGoogle Scholar
  5. 5.
    Bongartz, K.E., Conn, A.R., Gould, N.I.M., Toint, P.L.: CUTE: constrained and unconstrained testing environments. ACM Trans. Math. Softw. 21, 123–160 (1995)CrossRefMATHGoogle Scholar
  6. 6.
    Cheng, W.Y.: A two-term PRP-based descent method. Numer. Funct. Anal. Optim. 28, 1217–1230 (2007)CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)CrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Dai, Y.H.: Convergence properties of the BFGS algorithm. SIAM J. Optim. 13, 693–701 (2003)CrossRefGoogle Scholar
  9. 9.
    Dai, Y.H., Yuan, Y.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10, 177–182 (2000)CrossRefGoogle Scholar
  10. 10.
    Dai, Z.F., Tian, B.S.: Global convergence of some modified PRP nonlinear conjugate gradient methods. Opt. Lett. (2010). doi:10.1007/s11590-010-0224-8 Google Scholar
  11. 11.
    Fletcher, R., Reeves, C.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)CrossRefMATHMathSciNetGoogle Scholar
  12. 12.
    Fletcher, R.: Practical Methods of Optimization, vol. I: Unconstrained Optimization. Wiley, New York (1987)Google Scholar
  13. 13.
    Hestenes, M.R., Stiefel, E.L.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand., B 49, 409–432 (1952)MATHMathSciNetGoogle Scholar
  14. 14.
    Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM. J. Optim. 2, 21–42 (1992)CrossRefMATHMathSciNetGoogle Scholar
  15. 15.
    Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 (2005)CrossRefMATHMathSciNetGoogle Scholar
  16. 16.
    Hager, W.W., Zhang, H.: A survey of nonlinear conjugate gradient methods. Pacific J. Optim. 2, 35–58 (2006)MATHMathSciNetGoogle Scholar
  17. 17.
    Liu, Y.L., Storey, C.S.: Efficient generalized conjugate gradient algorithms, part 1: theory. J. Optim. Theory Appl. 69, 129–137 (1991)CrossRefMATHMathSciNetGoogle Scholar
  18. 18.
    Li, D.H., Fukushima, M.: A modified BFGS method and its global convergence in nonconvex minimization. J. Comput. Appl. Math. 129, 15–35 (2001)CrossRefMATHMathSciNetGoogle Scholar
  19. 19.
    Polak, B., Ribiére, G.: Note surla convergence des méthodes de directions conjuguées. Rev. Francaise Infomat Recherche Operatonelle, 3e Année 16, 35–43 (1969)Google Scholar
  20. 20.
    Polyak, B.T.: The conjugate gradient method in extreme problems. USSR Comput. Math. Math. Phys. 9, 94–112 (1969)CrossRefGoogle Scholar
  21. 21.
    Shi, Z.J., Shen, J.: Convergence of the Polak-Ribiére-Polyak conjugate gradient method. Nonlinear Anal. 66, 1428–1441 (2007)CrossRefMATHMathSciNetGoogle Scholar
  22. 22.
    Yu, G.H., Zhao, Y.L., Wei, Z.X.: A descent nonlinear conjugate gradient method for large-scale unconstrained optimization. Appl. Math. Comput. 187, 636–643 (2007)CrossRefMATHMathSciNetGoogle Scholar
  23. 23.
    Yu, G.H., Guan, L.T., Chen, W.: Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization. Optim. Methods Softw. 23, 275–293 (2008)CrossRefMATHMathSciNetGoogle Scholar
  24. 24.
    Yu, G.H., Huang, J.H, Zhou, Y.: A descent spectral conjugate gradient method for impulse noise removal. Appl. Math. Lett. 23, 555–560 (2010)CrossRefMATHMathSciNetGoogle Scholar
  25. 25.
    Yuan, Y.: Numerical Methods for Nonlinear Programming. Shanghai Scientific & Technical Publishers (1993)Google Scholar
  26. 26.
    Yin, K., Xiao, Y.H., Zhang, M.L.: Nonlinear conjugate gradient method for l 1-norm regularization problems in compressive sensing. J. Comput. Infor. Sys. 7, 880–885 (2011)Google Scholar
  27. 27.
    Wen, F.H., Yang, X.G.: Skewness of return distribution and coeffcient of risk premium. J. Syst. Sci. Complex. 22, 360–371 (2009)CrossRefMathSciNetGoogle Scholar
  28. 28.
    Wen, F.H., Liu, Z.F.: A copula-based correlation measure and its application in chinese stock market. Int. J. Inf. Technol. Decis. Mak. 8, 1–15 (2009)CrossRefMATHGoogle Scholar
  29. 29.
    Zhang, L., Zhou, W., Li, D.: A descent modified Polak-Ribi-re-Polyak conjugate gradient method and its global convergence. IMA J. Numer. Anal. 26, 629–640 (2006)CrossRefMATHMathSciNetGoogle Scholar
  30. 30.
    Zhang, L., Zhou, W., Li, D.: Some descent three-term conjugate gradient methods and their global convergence. Optim. Methods Softw. 22, 697–711 (2007)CrossRefMathSciNetGoogle Scholar
  31. 31.
    Zoutendijk, G.: Nonlinear programming computational methods. In: Abadie, J. (ed.) Integer and Nonlinear Programming, pp. 37–86. North-Holland, Amsterdam (1970)Google Scholar

Copyright information

© Springer Science+Business Media, LLC. 2011

Authors and Affiliations

  1. 1.College of Mathematics and Computational ScienceChangsha University of Science and TechnologyChangshaChina
  2. 2.School of Econometrics and ManagementChangsha University of Science and TechnologyChangshaChina

Personalised recommendations