An Analysis of the Barzilai and Borwein Gradient Method for Unsymmetric Linear Equations
Part of the
book series (APOP, volume 96)
The Barzilai and Borwein gradient method does not ensure descent in the objective function at each iteration, but performs better than the classical steepest descent method in practical computations. Combined with the technique of nonmonotone line search etc., such a method has found successful applications in unconstrained optimization, convex constrained optimization and stochastic optimization. In this paper, we give an analysis of the Barzilai and Borwein gradient method for two unsymmetric linear equations with only two variables. Under mild conditions, we prove that the convergence rate of the Barzilai and Borwein gradient method is Q-superlinear if the coefficient matrix A has the same eigenvalue; if the eigenvalues of A are different, then the convergence rate is R-superlinear.
Key wordsUnsymmetric linear equations gradient method convergence Q-superlinear R-superlinear
H. Akaike (1959), On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method, Ann. Inst. Statist. Math. Tokyo
, Vol. 11, pp. 1–17.zbMATHMathSciNetCrossRefGoogle Scholar
J. Barzilai and J. M. Borwein (1988), Two-point step size gradient methods, IMA J. Numer. Anal.
, Vol. 8, pp. 141–148.MathSciNetGoogle Scholar
E. G. Birgin, I. Chambouleyron, and J. M. Martínez (1999), Estimation of the optical constants and the thickness of thin films using unconstrained optimization, J. Comput. Phys.
, Vol. 151, pp. 862–880.CrossRefGoogle Scholar
E. G. Birgin and Y. G. Evtushenko (1998), Automatic differentiation and spectral projected gradient methods for optimal control problems, Optim. Methods Softw.
, Vol. 10, pp. 125–146.MathSciNetGoogle Scholar
E. G. Birgin, J. M. Martínez, and M. Raydan (2000), Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optim.
, Vol. 10, pp. 1196–1211.MathSciNetCrossRefGoogle Scholar
A. Cauchy (1847), Méthode générale pour la résolution des systèmes d'équations simultanées, Comp. Rend. Acad. Sci. Paris
, Vol. 25, pp. 46–89.Google Scholar
Y. H. Dai and L.-Z. Liao (2002), R
-linear convergence of the Barzilai and Borwein gradient method, IMA J. Numer. Anal.
, Vol. 22, No. 1, pp. 1–10.MathSciNetCrossRefGoogle Scholar
R. Fletcher (1990), Low storage methods for unconstrained optimization, Lectures in Applied Mathematics (AMS)
, Vol. 26, pp. 165–179.zbMATHMathSciNetGoogle Scholar
A. Friedlander, J. M. Martínez, B. Molina, and M. Raydan (1999), Gradient method with retards and generalizations, SIAM J. Numer. Anal.
, Vol. 36, 275–289.MathSciNetCrossRefGoogle Scholar
W. Glunt, T. L. Hayden, and M. Raydan (1993), Molecular conformations from distance matrices, J. Comput. Chem.
, Vol. 14, pp. 114–120.CrossRefGoogle Scholar
L. Grippo, F. Lampariello, and S. Lucidi (1986), A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal.
, Vol. 23, pp. 707–716.MathSciNetCrossRefGoogle Scholar
W. B. Liu and Y. H. Dai (2001), Minimization Algorithms based on Supervisor and Searcher Cooperation, Journal of Optimization Theory and Applications
, Vol. 111, No. 2, pp. 359–379.MathSciNetCrossRefGoogle Scholar
M. Raydan (1993), On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. Numer. Anal.
, Vol. 13, pp. 321–326.zbMATHMathSciNetGoogle Scholar
M. Raydan (1997), The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM J. Optim.
, Vol. 7, pp. 26–33.zbMATHMathSciNetCrossRefGoogle Scholar
© Springer Science+Business Media, Inc. 2005