Annals of Operations Research

, Volume 103, Issue 1–4, pp 161–173 | Cite as

Global Convergence of Conjugate Gradient Methods without Line Search

  • Jie Sun
  • Jiapu Zhang


Global convergence results are derived for well-known conjugate gradient methods in which the line search step is replaced by a step whose length is determined by a formula. The results include the following cases: (1) The Fletcher–Reeves method, the Hestenes–Stiefel method, and the Dai–Yuan method applied to a strongly convex LC1 objective function; (2) The Polak–Ribière method and the Conjugate Descent method applied to a general, not necessarily convex, LC1 objective function.

conjugate gradient methods convergence of algorithms line search 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    M. Al-Baali, Descent property and global convergence of the Fletcher-Reeves method with inexact line search, IMA J. Numer. Anal. 5 (1985) 121–124.Google Scholar
  2. [2]
    Y.H. Dai, J.Y. Han, G.H. Liu, D.F. Sun, H.X. Yin and Y.X. Yuan, Convergence properties of nonlinear conjugate gradient methods, SIAM J. Optim. 10 (1999) 345–358.Google Scholar
  3. [3]
    Y.H. Dai and Y. Yuan, Convergence properties of the conjugate descent method, Advances in Mathematics 25 (1996) 552–562.Google Scholar
  4. [4]
    Y.H. Dai and Y. Yuan, Convergence properties of the Fletcher-Reeves method, IMA J. Numer. Anal. 16 (1996) 155–164.Google Scholar
  5. [5]
    Y.H. Dai and Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optimization 10 (1999) 177–182.Google Scholar
  6. [6]
    R. Fletcher, Practical Method of Optimization, Vol I: Unconstrained Optimization, 2nd edn. (Wiley, New York, 1987).Google Scholar
  7. [7]
    R. Fletcher and C. Reeves, Function minimization by conjugate gradients, Comput. J. 7 (1964) 149–154.Google Scholar
  8. [8]
    J.C. Gilbert and J. Nocedal, Global convergence properties of conjugate gradient methods for optimization, SIAM Journal on Optimization 2 (1992) 21–42.Google Scholar
  9. [9]
    M.R. Hestenes and E. Stiefel, Method of conjugate gradient for solving linear system, J. Res. Nat. Bur. Stand. 49 (1952) 409–436.Google Scholar
  10. [10]
    Y. Hu and C. Storey, Global convergence result for conjugate gradient methods, Journal of Optimization Theory and Applications 71 (1991) 399–405.Google Scholar
  11. [11]
    G. Liu, J. Han and H. Yin, Global convergence of the Fletcher-Reeves algorithm with inexact line search, Appl. Math. J. Chinese Univ. Ser. B 10 (1995) 75–82.Google Scholar
  12. [12]
    B. Polak, The conjugate gradient method in extreme problems, Comput. Math. Math. Phys. 9 (1969) 94–112.Google Scholar
  13. [13]
    B. Polak and G. Ribiière, Note sur la convergence des méthodes de directions conjuguées, Rev. Fran. Informat. Rech. Opér. 16 (1969) 35–43.Google Scholar
  14. [14]
    M.J.D. Powell, Nonconvex minimization calculations and the conjugate gradient method, in: Lecture Notes in Mathematics 1066 (1984) pp. 121–141.Google Scholar
  15. [15]
    M.J.D. Powell, Convergence properties of algorithms for nonlinear optimization, SIAM Review 28 (1986) 487–500.Google Scholar
  16. [16]
    D. Touati-Ahmed and C. Storey, Efficient hybrid conjugate gradient techniques, Journal of Optimization Theory and Applications 64 (1990) 379–397.Google Scholar
  17. [17]
    P. Wolfe, Convergence conditions for ascent methods, SIAM Review 11 (1969) 226–235.Google Scholar
  18. [18]
    G. Zoutendijk, Nonlinear programming, computational methods, in: Integer and Nonlinear Programming, ed. J. Abadie (North-Holland, 1970) pp. 37-86.Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • Jie Sun
    • 1
  • Jiapu Zhang
    • 2
  1. 1.Department of Decision SciencesNational University of SingaporeRepublic of Singapore
  2. 2.Department of MathematicsUniversity of MelbourneMelbourneAustralia

Personalised recommendations