Skip to main content

An optimal parameter choice for the Dai–Liao family of conjugate gradient methods by avoiding a direction of the maximum magnification by the search direction matrix

Abstract

Based on a singular value analysis conducted on the Dai–Liao conjugate gradient method, it is shown that when the gradient approximately lies in the direction of the maximum magnification by the search direction matrix, the method may get into some computational errors and also, the convergence may occur hardly. Hence, we obtain a formula for computing the Dai–Liao parameter which makes the direction of the maximum magnification by the search direction matrix to be orthogonal to the gradient. We briefly discuss global convergence of the corresponding Dai–Liao method with and without convexity assumption on the objective function. Numerical experiments on a set of test problems of the CUTEr collection show practical effectiveness of the suggested adaptive choice of the Dai–Liao parameter in the sense of the Dolan–Moré performance profile.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

References

  1. Andrei N (2007) Numerical comparison of conjugate gradient algorithms for unconstrained optimization. Stud Inform Control 16(4):333–352

    Google Scholar 

  2. Andrei N (2011) Open problems in conjugate gradient algorithms for unconstrained optimization. Bull Malays Math Sci Soc 34(2):319–330

    Google Scholar 

  3. Andrei N (2016) An adaptive conjugate gradient algorithm for large-scale unconstrained optimization. J Comput Appl Math 292(1):83–91

    Article  Google Scholar 

  4. Andrei N (2017) A Dai–Liao conjugate gradient algorithm with clustering of eigenvalues. Numer Algorithms 77:1273–1282. https://doi.org/10.1007/s11075-017-0362-5

    Article  Google Scholar 

  5. Babaie-Kafaki S (2014) On the sufficient descent condition of the Hager–Zhang conjugate gradient methods. 4OR 12(3):285–292

    Article  Google Scholar 

  6. Babaie-Kafaki S, Ghanbari R (2014) The Dai–Liao nonlinear conjugate gradient method with optimal parameter choices. Eur J Oper Res 234(3):625–630

    Article  Google Scholar 

  7. Babaie-Kafaki S, Ghanbari R (2014) A descent family of Dai–Liao conjugate gradient methods. Optim Methods Softw 29(3):583–591

    Article  Google Scholar 

  8. Babaie-Kafaki S, Ghanbari R (2014) Two modified three-term conjugate gradient methods with sufficient descent property. Optim Lett 8(8):2285–2297

    Article  Google Scholar 

  9. Babaie-Kafaki S, Ghanbari R (2015) Two optimal Dai–Liao conjugate gradient methods. Optimization 64(11):2277–2287

    Article  Google Scholar 

  10. Babaie-Kafaki S, Ghanbari R (2017a) A class of adaptive Dai–Liao conjugate gradient methods based on the scaled memoryless BFGS update. 4OR 15(1):85–92

    Article  Google Scholar 

  11. Babaie-Kafaki S, Ghanbari R (2017b) A class of descent four-term extension of the Dai–Liao conjugate gradient method based on the scaled memoryless BFGS update. J Ind Manag Optim 3(2):649–658

    Google Scholar 

  12. Babaie-Kafaki S, Ghanbari R (2017c) Extensions of the Hestenes-Stiefel and Polak-Ribière-Polyak conjugate gradient methods with sufficient descent property. Bull Iran Math Soc 43(7):2437–2448

    Google Scholar 

  13. Babaie-Kafaki S, Ghanbari R (2017d) Two adaptive Dai–Liao nonlinear conjugate gradient methods. Iran J Sci Technol Trans Sci 42:1505–1509. https://doi.org/10.1007/s40995-017-0271-4

    Article  Google Scholar 

  14. Babaie-Kafaki S, Ghanbari R, Mahdavi-Amiri N (2010) Two new conjugate gradient methods based on modified secant equations. J Comput Appl Math 234(5):1374–1386

    Article  Google Scholar 

  15. Dai YH, Han JY, Liu GH, Sun DF, Yin HX, Yuan YX (1999) Convergence properties of nonlinear conjugate gradient methods. SIAM J Optim 10(2):348–358

    Google Scholar 

  16. Dai YH, Kou CX (2013) A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J Optim 23(1):296–320

    Article  Google Scholar 

  17. Dai YH, Liao LZ (2001) New conjugacy conditions and related nonlinear conjugate gradient methods. Appl Math Optim 43(1):87–101

    Article  Google Scholar 

  18. Dolan ED, Moré JJ (2002) Benchmarking optimization software with performance profiles. Math Program 91(2):201–213

    Article  Google Scholar 

  19. Fatemi M (2016) A new efficient conjugate gradient method for unconstrained optimization. J Comput Appl Math 300(1):207–216

    Article  Google Scholar 

  20. Fatemi M (2016) An optimal parameter for Dai–Liao family of conjugate gradient methods. J Optim Theory Appl 169(2):587–605

    Article  Google Scholar 

  21. Fatemi M, Babaie-Kafaki S (2016) Two extensions of the Dai–Liao method with sufficient desent property based on a penalization scheme. Bull Comput Appl Math 4(1):7–19

    Google Scholar 

  22. Ford JA, Narushima Y, Yabe H (2008) Multi-step nonlinear conjugate gradient methods for unconstrained minimization. Comput Optim Appl 40(2):191–216

    Article  Google Scholar 

  23. Gilbert JC, Nocedal J (1992) Global convergence properties of conjugate gradient methods for optimization. SIAM J Optim 2(1):21–42

    Article  Google Scholar 

  24. Gould NIM, Orban D, Toint PhL (2003) CUTEr: a constrained and unconstrained testing environment, revisited. ACM Trans Math Softw 29(4):373–394

    Article  Google Scholar 

  25. Hager WW, Zhang H (2005) A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J Optim 16(1):170–192

    Article  Google Scholar 

  26. Hager WW, Zhang H (2006) Algorithm 851: CG\(_{-}\)Descent, a conjugate gradient method with guaranteed descent. ACM Trans Math Softw 32(1):113–137

    Article  Google Scholar 

  27. Hager WW, Zhang H (2006) A survey of nonlinear conjugate gradient methods. Pac J Optim 2(1):35–58

    Google Scholar 

  28. Hestenes MR, Stiefel E (1952) Methods of conjugate gradients for solving linear systems. J Res Natl Bur Stand 49(6):409–436

    Article  Google Scholar 

  29. Li G, Tang C, Wei Z (2007) New conjugacy condition and related new conjugate gradient methods for unconstrained optimization. J Comput Appl Math 202(2):523–539

    Article  Google Scholar 

  30. Livieris IE, Pintelas P (2012) A descent Dai–Liao conjugate gradient method based on a modified secant equation and its global convergence. ISRN Comput Math 2012:8 Article ID 435495

    Article  Google Scholar 

  31. Narushima Y, Yabe H (2012) Conjugate gradient methods based on secant conditions that generate descent search directions for unconstrained optimization. J Comput Appl Math 236(17):4303–4317

    Article  Google Scholar 

  32. Nocedal J, Wright SJ (2006) Numerical optimization. Springer, New York

    Google Scholar 

  33. Perry A (1976) A modified conjugate gradient algorithm. Oper Res 26(6):1073–1078

    Article  Google Scholar 

  34. Peyghami MR, Ahmadzadeh H, Fazli A (2015) A new class of efficient and globally convergent conjugate gradient methods in the Dai–Liao family. Optim Methods Softw 30(4):843–863

    Article  Google Scholar 

  35. Powell MJD (1986) Convergence properties of algorithms for nonlinear optimization. SIAM Rev 28(4):487–500

    Article  Google Scholar 

  36. Sun W, Yuan YX (2006) Optimization theory and methods: nonlinear programming. Springer, New York

    Google Scholar 

  37. Watkins DS (2002) Fundamentals of matrix computations. Wiley, New York

    Book  Google Scholar 

  38. Zhou W, Zhang L (2006) A nonlinear conjugate gradient method based on the MBFGS secant condition. Optim Methods Softw 21(5):707–714

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by Research Council of Semnan University (Grant no. 139704261033). The authors thank the anonymous Reviewers and the Associate Editor for their valuable comments and suggestions helped to improve the quality of this work. They are also grateful to Professor Michael Navon for providing the line search code.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Saman Babaie-Kafaki.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Aminifard, Z., Babaie-Kafaki, S. An optimal parameter choice for the Dai–Liao family of conjugate gradient methods by avoiding a direction of the maximum magnification by the search direction matrix. 4OR-Q J Oper Res 17, 317–330 (2019). https://doi.org/10.1007/s10288-018-0387-1

Download citation

Keywords

  • Nonlinear programming
  • Unconstrained optimization
  • Conjugate gradient method
  • Maximum magnification
  • Global convergence

Mathematics Subject Classification

  • 90C53
  • 65K05
  • 65F35