On the Barzilai-Borwein Method
A review is given of the underlying theory and recent developments in regard to the Barzilai-Borwein steepest descent method for large scale unconstrained optimization. One aim is to assess why the method seems to be comparable in practical efficiency to conjugate gradient methods. The importance of using a non-monotone line search is stressed, although some suggestions are made as to why the modification proposed by Raydan (1997) often does not usually perform well for an ill-conditioned problem. Extensions for box constraints are discussed. A number of interesting open questions are put forward.
Key wordsBarzilai-Borwein method steepest descent elliptic systems unconstrained optimization
Unable to display preview. Download preview PDF.
- Birgin, E.G., Martínez, J.M. and Raydan, M., Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optimization, Vol. 10, pp. 1196–1211.Google Scholar
- Cauchy, A., (1847), Méthode générale pour la resolution des systèms d'équations simultanées, Comp. Rend. Sci. Paris, Vol. 25, pp. 536–538.Google Scholar
- Dai, Y.H. and Liao, L.-Z., (1999), R-linear convergence of the Barzilai and Borwein gradient method, Research report, (accepted by IMA J. Numerical Analysis).Google Scholar
- Friedlander, A., Martínez, J.M. and Raydan, M., (1995), A new method for large-scale box constrained convex quadratic minimization problems, Optimization Methods and Software, Vol. 5, pp. 57–74.Google Scholar
- G. H. Golub and C. F. Van Loan, (1996), Matrix Computations, 3rd Edition, The Johns Hopkins Press, Baltimore.Google Scholar
- Polak, E., (1971), Computational Methods in Optimization: A Unified Approach, Academic Press, New York.Google Scholar
- Reid, J.K., (1971), Large Sparse Sets of Linear Equations, Academic Press, London, Chapter 11, pp. 231–254.Google Scholar