A Positive Barzilai–Borwein-Like Stepsize and an Extension for Symmetric Linear Systems
The Barzilai and Borwein (BB) gradient method has achieved a lot of attention since it performs much more better than the classical steepest descent method. In this paper, we analyze a positive BB-like gradient stepsize and discuss its possible uses. Specifically, we present an analysis of the positive stepsize for two-dimensional strictly convex quadratic functions and prove the R-superlinear convergence under some assumption. Meanwhile, we extend BB-like methods for solving symmetric linear systems and find that a variant of the positive stepsize is very useful in the context. Some useful discussions on the positive stepsize are also given.
KeywordsUnconstrained optimization Barzilai and Borwein gradient method Quadratic function R-superlinear convergence Condition number
The authors are grateful to Dr. Bo Jiang for checking an early version of this manuscript and to Ms. Liaoyuan Zeng for her editing of this paper. They also thank an anonymous referee for his/her useful suggestions and comments.
- 1.Al-Baali, M.: On alternate steps for gradient methods. Talk at 22-nd Biennial Conference on Numerical Analysis, University of Dundee, Scotland, 26–29 June 2007Google Scholar
- 4.Cauchy, A.: Méthode générale pour la résolution des systèms d’équations simultanées. Comput. Rend. Sci. Paris 25, 536–538 (1847)Google Scholar
- 12.Dai, Y.H., Liao, L.Z.: A new first-order neural network for unconstrained nonconvex optimization. Research Report, Academy of Mathematics and Systems Science, Chinese Academy of Sciences (1999)Google Scholar