Optimization Letters

, Volume 10, Issue 8, pp 1789–1797 | Cite as

On optimality of two adaptive choices for the parameter of Dai–Liao method

Short Communication
First Online:
Received:
Accepted:

Abstract

Orthonormal matrices are a class of well-conditioned matrices with the least spectral condition number. Here, at first it is shown that a recently proposed choice for parameter of the Dai–Liao nonlinear conjugate gradient method makes the search direction matrix as close as possible to an orthonormal matrix in the Frobenius norm. Then, conducting a brief singular value analysis, it is shown that another recently proposed choice for the Dai–Liao parameter improves spectral condition number of the search direction matrix. Thus, theoretical justifications of the two choices for the Dai–Liao parameter are enhanced. Finally, some comparative numerical results are reported.

Keywords

Unconstrained optimization Large-scale optimization  Conjugate gradient method Orthonormal matrix Condition number 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This research was supported by the Research Council of Semnan University. The author is grateful to Professor William W. Hager for providing the line search code. He also thanks the anonymous reviewers for their valuable comments and suggestions helped to improve the presentation.

References

  1. 1.
    Andrei, N.: Numerical comparison of conjugate gradient algorithms for unconstrained optimization. Stud. Inform. Control 16(4), 333–352 (2007)MathSciNetGoogle Scholar
  2. 2.
    Andrei, N.: Open problems in conjugate gradient algorithms for unconstrained optimization. B. Malays. Math. Sci. So. 34(2), 319–330 (2011)MathSciNetMATHGoogle Scholar
  3. 3.
    Babaie-Kafaki, S.: An adaptive conjugacy condition and related nonlinear conjugate gradient methods. Int. J. Comput. Methods 11(4), 1350092 (2014)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Babaie-Kafaki, S.: On the sufficient descent condition of the Hager-Zhang conjugate gradient methods. 4OR 12(3), 285–292 (2014)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Babaie-Kafaki, S., Ghanbari, R.: The Dai–Liao nonlinear conjugate gradient method with optimal parameter choices. Eur. J. Oper. Res. 234(3), 625–630 (2014)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Babaie-Kafaki, S., Ghanbari, R.: A descent family of Dai–Liao conjugate gradient methods. Optim. Methods Softw. 29(3), 583–591 (2014)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Babaie-Kafaki, S., Ghanbari, R.: Two optimal Dai–Liao conjugate gradient methods. Optimization 64(1), 2277–2287 (2015)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Dai, Y.H., Han, J.Y., Liu, G.H., Sun, D.F., Yin, H.X., Yuan, Y.X.: Convergence properties of nonlinear conjugate gradient methods. SIAM J. Optim. 10(2), 348–358 (1999)MathSciNetMATHGoogle Scholar
  9. 9.
    Dai, Y.H., Kou, C.X.: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23(1), 296–320 (2013)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43(1), 87–101 (2001)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2, Ser. A), 201–213 (2002)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Gould, N.I.M., Orban, D., Toint, PhL: CUTEr: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16(1), 170–192 (2005)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Hager, W.W., Zhang, H.: Algorithm 851: CG\(_{-}\)Descent, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw. 32(1), 113–137 (2006)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Hager, W.W., Zhang, H.: A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2(1), 35–58 (2006)MathSciNetMATHGoogle Scholar
  16. 16.
    Perry, A.: A modified conjugate gradient algorithm. Oper. Res. 26(6), 1073–1078 (1976)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Piazza, G., Politi, T.: An upper bound for the condition number of a matrix in spectral norm. J. Comput. Appl. Math. 143(1), 141–144 (2002)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Stewart, G.W.: Matrix Algorithms, Volume 1: basic decompositions. SIAM, Philadelphia (1998)Google Scholar
  19. 19.
    Sun, W., Yuan, Y.X.: Optimization Theory and Methods: Nonlinear Programming. Springer, New York (2006)MATHGoogle Scholar
  20. 20.
    Watkins, D.S.: Fundamentals of Matrix Computations. Wiley, New York (2002)CrossRefMATHGoogle Scholar
  21. 21.
    Zhang, L., Zhou, W., Li, D.H.: Some descent three-term conjugate gradient methods and their global convergence. Optim. Methods Softw. 22(4), 697–711 (2007)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Department of Mathematics, Faculty of Mathematics, Statistics and Computer ScienceSemnan UniversitySemnanIran

Personalised recommendations