Journal of Optimization Theory and Applications

, Volume 178, Issue 3, pp 860–884 | Cite as

New Hybrid Conjugate Gradient and Broyden–Fletcher–Goldfarb–Shanno Conjugate Gradient Methods

  • Predrag S. StanimirovićEmail author
  • Branislav Ivanov
  • Snežana Djordjević
  • Ivona Brajević


Three hybrid methods for solving unconstrained optimization problems are introduced. These methods are defined using proper combinations of the search directions and included parameters in conjugate gradient and quasi-Newton methods. The convergence of proposed methods with the underlying backtracking line search is analyzed for general objective functions and particularly for uniformly convex objective functions. Numerical experiments show the superiority of the proposed methods with respect to some existing methods in view of the Dolan and Moré’s performance profile.


Global convergence Backtracking line search Unconstrained optimization Conjugate gradient methods Quasi-Newton methods 

Mathematics Subject Classification




The first author gratefully acknowledge support from the Research Project 174013 of the Serbian Ministry of Science.


  1. 1.
    Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7(2), 149–154 (1964)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Fletcher, R.: Practical Methods of Optimization. Unconstrained Optimization, vol. 1. Wiley, New York (1987)zbMATHGoogle Scholar
  3. 3.
    Dai, Y.H., Yuan, Y.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10(1), 177–182 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Hestenes, M.R., Stiefel, E.L.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Polak, E., Ribiere, G.: Note sur la convergence des mthodes de directions conjuguées. Rev. Francaise Imformat Recherche Opertionelle 16, 35–43 (1969)zbMATHGoogle Scholar
  6. 6.
    Polyak, B.T.: The conjugate gradient method in extreme problems. U.S.S.R. Comput. Math. Math. Phys. 9, 94–112 (1969)CrossRefzbMATHGoogle Scholar
  7. 7.
    Liu, Y., Storey, C.: Efficient generalized conjugate gradient algorithms, part 1: theory. J. Optim. Theory Appl. 69(1), 129–137 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Yang, X., Luo, Z., Dai, X.: A global convergence of LS-CD hybrid conjugate gradient method. In: Advances in Numerical Analysis 2013. Hindawi Publishing Corporation, Article ID 517452, 5 pp. (2013)Google Scholar
  9. 9.
    Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43(1), 87–101 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Zheng, Y., Zheng, B.: Two new Dai–Liao-type conjugate gradient methods for unconstrained optimization problems. J. Optim. Theory Appl. 175, 502–509 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Andrei, N.: An acceleration of gradient descent algorithm with backtracking for unconstrained optimization. Numer. Algorithms 42, 63–73 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Stanimirović, P.S., Miladinović, M.B.: Accelerated gradient descent methods with line search. Numer. Algorithms 54, 503–520 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Touati-Ahmed, D., Storey, C.: Efficient hybrid conjugate gradient techniques. J. Optim. Theory Appl. 64(2), 379–397 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Zhang, L., Zhou, W.: Two descent hybrid conjugate gradient methods for optimization. J. Comput. Appl. Math. 216, 251–264 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2(1), 21–42 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Dai, Y.H., Yuan, Y.: An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103, 33–47 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Zhang, L., Zhou, W.J., Li, D.H.: A descent modified Polak–Ribiére–Polyak conjugate gradient method and its global convergence. IMA J. Numer. Anal. 26, 629–640 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Zhang, L., Zhou, W.J., Li, D.H.: Global convergence of a modified Fletcher–Reeves conjugate method with Armijo-type line search. Numer. Math. 104, 561–572 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Zhang, L.: Nonlinear Conjugate Gradient Methods for Optimization Problems. Ph.D. Thesis, College of Mathematics and Econometrics, Hunan University, Changsha, China (2006)Google Scholar
  21. 21.
    Ibrahim, M.A.H., Mamat, M., Leong, W.J.: The hybrid BFGS-CG method in solving unconstrained optimization problems. In: Abstract and Applied Analysis 2014. Hindawi Publishing Corporation. Article ID 507102, 6 pp. (2014)Google Scholar
  22. 22.
    Zoutendijk, G.: Nonlinear programming, computational methods. In: Abadie, J. (ed.) Integer and Nonlinear Programming, pp. 37–86. North-Holland, Amsterdam (1970)Google Scholar
  23. 23.
    Cheng, W.: A two-term PRP-based descent method. Numer. Funct. Anal. Optim. 28(11–12), 1217–1230 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Luo, Y.Z., Tang, G.J., Zhou, L.N.: Hybrid approach for solving systems of nonlinear equations using chaos optimization and quasi-Newton method. Appl. Soft Comput. 8(2), 1068–1073 (2008)CrossRefGoogle Scholar
  25. 25.
    Han, L., Neumann, M.: Combining quasi-Newton and Cauchy directions. Int. J. Appl. Math. 12(2), 167–191 (2003)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Baluch, B., Salleh, Z., Alhawarat, A., Roslan, U.A.M.: A new modified three-term conjugate gradient method with sufficient descent property and its global convergence. J. Math. 2017, Article ID 2715854, 12 pp. (2017)Google Scholar
  27. 27.
    Khanaiah, Z., Hmod, G.: Novel hybrid algorithm in solving unconstrained optimizations problems. Int. J. Novel Res. Phys. Chem. Math. 4(3), 36–42 (2017)Google Scholar
  28. 28.
    Osman, W.F.H.W., Ibrahim, M.A.H., Mamat, M.: Hybrid DFP-CG method for solving unconstrained optimization problems. J. Phys. Conf. Ser. 890(2017), 012033 (2017)CrossRefGoogle Scholar
  29. 29.
    Byrd, R.H., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J. Numer. Anal. 26(3), 727–739 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Andrei, N.: An Unconstrained optimization test functions collection. Adv. Model. Optim. 10(1), 147–161 (2008)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Predrag S. Stanimirović
    • 1
    Email author
  • Branislav Ivanov
    • 2
  • Snežana Djordjević
    • 3
  • Ivona Brajević
    • 1
  1. 1.Faculty of Sciences and MathematicsUniversity of NišNisSerbia
  2. 2.Technical Faculty in BorUniversity of BelgradeBorSerbia
  3. 3.Faculty of Technology in LeskovacUniversity of NišLeskovacSerbia

Personalised recommendations