An Optimal Parameter for Dai–Liao Family of Conjugate Gradient Methods

Abstract

We introduce a new efficient nonlinear conjugate gradient method for unconstrained optimization, based on minimizing a penalty function. Our penalty function combines the good properties of the linear conjugate gradient method using some penalty parameters. We show that the new method is a member of Dai–Liao family and, more importantly, propose an efficient Dai–Liao parameter by closely analyzing the penalty function. Numerical experiments show that the proposed parameter is promising.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3

References

  1. 1.

    Hestenes, M.R., Stiefel, E.: Method of conjugate gradient for solving linear system. J. Res. Natl. Bur. Stand. 49, 409–436 (1952)

    MathSciNet  Article  MATH  Google Scholar 

  2. 2.

    Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Polak, E., Ribière, G.: Not sur la convergence de méthods directions conjugées. Revue Franccaise d’Informatique et de Recherche opiérationnelle. 16, 35–43 (1969)

    MATH  Google Scholar 

  4. 4.

    Polyak, B.T.: The conjugate gradient method in extremal problems. USSR Comput. Math. Math. Phys. 9, 94–112 (1969)

    Article  MATH  Google Scholar 

  5. 5.

    Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43, 87–101 (2001)

    MathSciNet  Article  MATH  Google Scholar 

  6. 6.

    Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  7. 7.

    Dai, Y.H., Kou, C.X.: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23, 296–320 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  8. 8.

    Hager, W.W., Zhang, H.: A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2(1), 335–358 (2006)

    MathSciNet  MATH  Google Scholar 

  9. 9.

    Dai, Y.H.: Nonlinear Conjugate Gradient Methods. Wiley Encyclopedia of Operations Research and Management Science. doi:10.1002/9780470400531.eorms0183 (2011)

  10. 10.

    Gilbert, J.C., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2, 21–42 (1992)

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45, 503–528 (1989)

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    Oren, S.S.: Self-scaling metric algorithms for unconstrained minimization. Ph.D. thesis, Department of Engineering Economic Systems. Stanford University, Stanford (1972)

  13. 13.

    Perry, J.M.: A Class of Conjugate Gradient Algorithms with a Two-Step Variable-Metric Memory. Northwestern University, Evanston (1977)

    Google Scholar 

  14. 14.

    Shanno, D.F.: On the convergence of a new conjugate gradient algorithm. SIAM J. Numer. Anal. 15, 1247–1257 (1978)

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    Moré, J.J., Thuente, D.J.: Line search algorithms with guaranteed sufficient decrease. Acm Trans. Math. Soft. 20, 286–307 (1994)

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Hager, W.W., Zhang, H.: Algorithm 851: CG-DESCENT, a conjugate gradient method with guaranteed descent. Acm Trans. Math. Soft. 32, 113–137 (2006)

    MathSciNet  Article  Google Scholar 

  17. 17.

    Gould, N.I.M., Orban, D., Toint, P.L.: CUTEr ( and SifDec ), a constrained and unconstrained testing environment, revisited. Technical report TR/PA/01/04, CERFACS, Toulouse (2001)

  18. 18.

    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profile. Math. Program. 91, 201–213 (2002)

    MathSciNet  Article  MATH  Google Scholar 

Download references

Acknowledgments

The author also thank the Research Council of K. N. Toosi University of Technology for supporting this work and sincerely appreciate the helpful comments and suggestions provided by anonymous referees.

Author information

Affiliations

Authors

Corresponding author

Correspondence to M. Fatemi.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fatemi, M. An Optimal Parameter for Dai–Liao Family of Conjugate Gradient Methods. J Optim Theory Appl 169, 587–605 (2016). https://doi.org/10.1007/s10957-015-0786-9

Download citation

Keywords

  • Conjugate gradient method
  • Dai–Liao family
  • Unconstrained optimization
  • Line search

Mathematics Subject Classification

  • 90C06
  • 90C26
  • 65Y20