New Hybrid Conjugate Gradient and Broyden–Fletcher–Goldfarb–Shanno Conjugate Gradient Methods

  • Predrag S. Stanimirović
  • Branislav Ivanov
  • Snežana Djordjević
  • Ivona Brajević
Article
  • 42 Downloads

Abstract

Three hybrid methods for solving unconstrained optimization problems are introduced. These methods are defined using proper combinations of the search directions and included parameters in conjugate gradient and quasi-Newton methods. The convergence of proposed methods with the underlying backtracking line search is analyzed for general objective functions and particularly for uniformly convex objective functions. Numerical experiments show the superiority of the proposed methods with respect to some existing methods in view of the Dolan and Moré’s performance profile.

Keywords

Global convergence Backtracking line search Unconstrained optimization Conjugate gradient methods Quasi-Newton methods 

Mathematics Subject Classification

90C30 

Notes

Acknowledgements

The first author gratefully acknowledge support from the Research Project 174013 of the Serbian Ministry of Science.

References

  1. 1.
    Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7(2), 149–154 (1964)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Fletcher, R.: Practical Methods of Optimization. Unconstrained Optimization, vol. 1. Wiley, New York (1987)MATHGoogle Scholar
  3. 3.
    Dai, Y.H., Yuan, Y.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10(1), 177–182 (1999)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Hestenes, M.R., Stiefel, E.L.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Polak, E., Ribiere, G.: Note sur la convergence des mthodes de directions conjuguées. Rev. Francaise Imformat Recherche Opertionelle 16, 35–43 (1969)MATHGoogle Scholar
  6. 6.
    Polyak, B.T.: The conjugate gradient method in extreme problems. U.S.S.R. Comput. Math. Math. Phys. 9, 94–112 (1969)CrossRefMATHGoogle Scholar
  7. 7.
    Liu, Y., Storey, C.: Efficient generalized conjugate gradient algorithms, part 1: theory. J. Optim. Theory Appl. 69(1), 129–137 (1991)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Yang, X., Luo, Z., Dai, X.: A global convergence of LS-CD hybrid conjugate gradient method. In: Advances in Numerical Analysis 2013. Hindawi Publishing Corporation, Article ID 517452, 5 pp. (2013)Google Scholar
  9. 9.
    Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43(1), 87–101 (2001)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Zheng, Y., Zheng, B.: Two new Dai–Liao-type conjugate gradient methods for unconstrained optimization problems. J. Optim. Theory Appl. 175, 502–509 (2017)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Andrei, N.: An acceleration of gradient descent algorithm with backtracking for unconstrained optimization. Numer. Algorithms 42, 63–73 (2006)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Stanimirović, P.S., Miladinović, M.B.: Accelerated gradient descent methods with line search. Numer. Algorithms 54, 503–520 (2010)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Touati-Ahmed, D., Storey, C.: Efficient hybrid conjugate gradient techniques. J. Optim. Theory Appl. 64(2), 379–397 (1990)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Zhang, L., Zhou, W.: Two descent hybrid conjugate gradient methods for optimization. J. Comput. Appl. Math. 216, 251–264 (2008)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2(1), 21–42 (1992)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Dai, Y.H., Yuan, Y.: An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103, 33–47 (2001)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 (2005)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Zhang, L., Zhou, W.J., Li, D.H.: A descent modified Polak–Ribiére–Polyak conjugate gradient method and its global convergence. IMA J. Numer. Anal. 26, 629–640 (2006)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Zhang, L., Zhou, W.J., Li, D.H.: Global convergence of a modified Fletcher–Reeves conjugate method with Armijo-type line search. Numer. Math. 104, 561–572 (2006)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Zhang, L.: Nonlinear Conjugate Gradient Methods for Optimization Problems. Ph.D. Thesis, College of Mathematics and Econometrics, Hunan University, Changsha, China (2006)Google Scholar
  21. 21.
    Ibrahim, M.A.H., Mamat, M., Leong, W.J.: The hybrid BFGS-CG method in solving unconstrained optimization problems. In: Abstract and Applied Analysis 2014. Hindawi Publishing Corporation. Article ID 507102, 6 pp. (2014)Google Scholar
  22. 22.
    Zoutendijk, G.: Nonlinear programming, computational methods. In: Abadie, J. (ed.) Integer and Nonlinear Programming, pp. 37–86. North-Holland, Amsterdam (1970)Google Scholar
  23. 23.
    Cheng, W.: A two-term PRP-based descent method. Numer. Funct. Anal. Optim. 28(11–12), 1217–1230 (2007)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Luo, Y.Z., Tang, G.J., Zhou, L.N.: Hybrid approach for solving systems of nonlinear equations using chaos optimization and quasi-Newton method. Appl. Soft Comput. 8(2), 1068–1073 (2008)CrossRefGoogle Scholar
  25. 25.
    Han, L., Neumann, M.: Combining quasi-Newton and Cauchy directions. Int. J. Appl. Math. 12(2), 167–191 (2003)MathSciNetMATHGoogle Scholar
  26. 26.
    Baluch, B., Salleh, Z., Alhawarat, A., Roslan, U.A.M.: A new modified three-term conjugate gradient method with sufficient descent property and its global convergence. J. Math. 2017, Article ID 2715854, 12 pp. (2017)Google Scholar
  27. 27.
    Khanaiah, Z., Hmod, G.: Novel hybrid algorithm in solving unconstrained optimizations problems. Int. J. Novel Res. Phys. Chem. Math. 4(3), 36–42 (2017)Google Scholar
  28. 28.
    Osman, W.F.H.W., Ibrahim, M.A.H., Mamat, M.: Hybrid DFP-CG method for solving unconstrained optimization problems. J. Phys. Conf. Ser. 890(2017), 012033 (2017)CrossRefGoogle Scholar
  29. 29.
    Byrd, R.H., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J. Numer. Anal. 26(3), 727–739 (1989)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Andrei, N.: An Unconstrained optimization test functions collection. Adv. Model. Optim. 10(1), 147–161 (2008)MathSciNetMATHGoogle Scholar
  31. 31.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Predrag S. Stanimirović
    • 1
  • Branislav Ivanov
    • 2
  • Snežana Djordjević
    • 3
  • Ivona Brajević
    • 1
  1. 1.Faculty of Sciences and MathematicsUniversity of NišNisSerbia
  2. 2.Technical Faculty in BorUniversity of BelgradeBorSerbia
  3. 3.Faculty of Technology in LeskovacUniversity of NišLeskovacSerbia

Personalised recommendations