An Analysis of the Barzilai and Borwein Gradient Method for Unsymmetric Linear Equations

  • Yu-Hong Dai
  • Li-Zhi Liao
  • Duan Li
Part of the Applied Optimization book series (APOP, volume 96)


The Barzilai and Borwein gradient method does not ensure descent in the objective function at each iteration, but performs better than the classical steepest descent method in practical computations. Combined with the technique of nonmonotone line search etc., such a method has found successful applications in unconstrained optimization, convex constrained optimization and stochastic optimization. In this paper, we give an analysis of the Barzilai and Borwein gradient method for two unsymmetric linear equations with only two variables. Under mild conditions, we prove that the convergence rate of the Barzilai and Borwein gradient method is Q-superlinear if the coefficient matrix A has the same eigenvalue; if the eigenvalues of A are different, then the convergence rate is R-superlinear.

Key words

Unsymmetric linear equations gradient method convergence Q-superlinear R-superlinear 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. H. Akaike (1959), On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method, Ann. Inst. Statist. Math. Tokyo, Vol. 11, pp. 1–17.zbMATHMathSciNetCrossRefGoogle Scholar
  2. J. Barzilai and J. M. Borwein (1988), Two-point step size gradient methods, IMA J. Numer. Anal., Vol. 8, pp. 141–148.MathSciNetGoogle Scholar
  3. E. G. Birgin, I. Chambouleyron, and J. M. Martínez (1999), Estimation of the optical constants and the thickness of thin films using unconstrained optimization, J. Comput. Phys., Vol. 151, pp. 862–880.CrossRefGoogle Scholar
  4. E. G. Birgin and Y. G. Evtushenko (1998), Automatic differentiation and spectral projected gradient methods for optimal control problems, Optim. Methods Softw., Vol. 10, pp. 125–146.MathSciNetGoogle Scholar
  5. E. G. Birgin, J. M. Martínez, and M. Raydan (2000), Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optim., Vol. 10, pp. 1196–1211.MathSciNetCrossRefGoogle Scholar
  6. A. Cauchy (1847), Méthode générale pour la résolution des systèmes d'équations simultanées, Comp. Rend. Acad. Sci. Paris, Vol. 25, pp. 46–89.Google Scholar
  7. Y. H. Dai and L.-Z. Liao (2002), R-linear convergence of the Barzilai and Borwein gradient method, IMA J. Numer. Anal., Vol. 22, No. 1, pp. 1–10.MathSciNetCrossRefGoogle Scholar
  8. R. Fletcher (1990), Low storage methods for unconstrained optimization, Lectures in Applied Mathematics (AMS), Vol. 26, pp. 165–179.zbMATHMathSciNetGoogle Scholar
  9. A. Friedlander, J. M. Martínez, B. Molina, and M. Raydan (1999), Gradient method with retards and generalizations, SIAM J. Numer. Anal., Vol. 36, 275–289.MathSciNetCrossRefGoogle Scholar
  10. W. Glunt, T. L. Hayden, and M. Raydan (1993), Molecular conformations from distance matrices, J. Comput. Chem., Vol. 14, pp. 114–120.CrossRefGoogle Scholar
  11. L. Grippo, F. Lampariello, and S. Lucidi (1986), A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal., Vol. 23, pp. 707–716.MathSciNetCrossRefGoogle Scholar
  12. W. B. Liu and Y. H. Dai (2001), Minimization Algorithms based on Supervisor and Searcher Cooperation, Journal of Optimization Theory and Applications, Vol. 111, No. 2, pp. 359–379.MathSciNetCrossRefGoogle Scholar
  13. M. Raydan (1993), On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. Numer. Anal., Vol. 13, pp. 321–326.zbMATHMathSciNetGoogle Scholar
  14. M. Raydan (1997), The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM J. Optim., Vol. 7, pp. 26–33.zbMATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  • Yu-Hong Dai
    • 1
  • Li-Zhi Liao
    • 2
  • Duan Li
    • 3
  1. 1.Institute of Computational Mathematics and Scientific/Engineering ComputingChinese Academy of SciencesP.R. China
  2. 2.Department of MathematicsHong Kong Baptist UniversityKowloon Tong, KowloonHong Kong
  3. 3.Department of Systems Engineering and Engineering ManagementChinese University of Hong KongHong Kong

Personalised recommendations