Advertisement

On the Barzilai-Borwein Method

  • Roger Fletcher
Part of the Applied Optimization book series (APOP, volume 96)

Abstract

A review is given of the underlying theory and recent developments in regard to the Barzilai-Borwein steepest descent method for large scale unconstrained optimization. One aim is to assess why the method seems to be comparable in practical efficiency to conjugate gradient methods. The importance of using a non-monotone line search is stressed, although some suggestions are made as to why the modification proposed by Raydan (1997) often does not usually perform well for an ill-conditioned problem. Extensions for box constraints are discussed. A number of interesting open questions are put forward.

Key words

Barzilai-Borwein method steepest descent elliptic systems unconstrained optimization 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Akaike, H. (1959), On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method, Ann. Inst. Statist. Math. Tokyo, Vol. 11, pp. 1–17.zbMATHMathSciNetCrossRefGoogle Scholar
  2. Barzilai, J. and Borwein, J.M. (1988), Two-point step size gradient methods, IMA J. Numerical Analysis, Vol. 8, pp. 141–148.MathSciNetGoogle Scholar
  3. Birgin, E.G., Martínez, J.M. and Raydan, M., Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optimization, Vol. 10, pp. 1196–1211.Google Scholar
  4. Cauchy, A., (1847), Méthode générale pour la resolution des systèms d'équations simultanées, Comp. Rend. Sci. Paris, Vol. 25, pp. 536–538.Google Scholar
  5. Conn, A.R., Gould, N.I.M. and Toint, Ph.L., (1988, 1989), Global convergence of a class of trust region algorithms for optimization with simple bounds, SIAM J. Numerical Analysis, Vol. 25, pp. 433–460, and Vol. 26, pp. 764–767.MathSciNetCrossRefGoogle Scholar
  6. Dai, Y.H. and Liao, L.-Z., (1999), R-linear convergence of the Barzilai and Borwein gradient method, Research report, (accepted by IMA J. Numerical Analysis).Google Scholar
  7. Dembo, R.S., Eisenstat, S.C. and Steihaug, T., (1982), Inexact Newton Methods, SIAM J. Numerical Analysis, Vol. 19, pp. 400–408.MathSciNetCrossRefGoogle Scholar
  8. Fletcher, R., (1990), Low storage methods for unconstrained optimization, Lectures in Applied Mathematics (AMS) Vol. 26, pp. 165–179.zbMATHMathSciNetGoogle Scholar
  9. Fletcher, R. and Reeves, C.M., (1964), Function minimization by conjugate gradients, Computer J. Vol. 7, pp. 149–154.MathSciNetCrossRefGoogle Scholar
  10. Friedlander, A., Martínez, J.M. and Raydan, M., (1995), A new method for large-scale box constrained convex quadratic minimization problems, Optimization Methods and Software, Vol. 5, pp. 57–74.Google Scholar
  11. Friedlander, A., Martínez, J.M., Molina, B., and Raydan, M., (1999), Gradient method with retards and generalizations, SIAM J. Numerical Analysis, Vol. 36, pp. 275–289.CrossRefGoogle Scholar
  12. Glunt, W., Hayden, T.L. and Raydan, M., (1993), Molecular conformations from distance matrices, J. Comput. Chem., Vol. 14, pp. 114–120.CrossRefGoogle Scholar
  13. Glunt, W., Hayden, T.L. and Raydan, M., (1994), Preconditioners for Distance Matrix Algorithms, J. Comput. Chem., Vol. 15, pp. 227–232.CrossRefGoogle Scholar
  14. G. H. Golub and C. F. Van Loan, (1996), Matrix Computations, 3rd Edition, The Johns Hopkins Press, Baltimore.Google Scholar
  15. Grippo, L., Lampariello, F. and Lucidi, S., (1986), A nonmonotone line search technique for Newton's method, SIAM J. Numerical Analysis, Vol. 23, pp. 707–716.MathSciNetCrossRefGoogle Scholar
  16. Hestenes, M.R. and Stiefel, E.L., (1952), Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards, Sect. 5:49, pp. 409–436.MathSciNetGoogle Scholar
  17. Molina, B. and Raydan, M., (1996), Preconditioned Barzilai-Borwein method for the numerical solution of partial differential equations, Numerical Algorithms, Vol. 13, pp. 45–60.MathSciNetCrossRefGoogle Scholar
  18. Nocedal, J., (1980), Updating quasi-Newton matrices with limited storage, Math. of Comp., Vol. 35, pp. 773–782.zbMATHMathSciNetCrossRefGoogle Scholar
  19. Polak, E., (1971), Computational Methods in Optimization: A Unified Approach, Academic Press, New York.Google Scholar
  20. Shanno, D.F., and Phua, K.H., (1980), Remark on Algorithm 500: Minimization of unconstrained multivariate functions, ACM Trans. Math. Software, Vol. 6, pp. 618–622.CrossRefGoogle Scholar
  21. Raydan, M., (1993), On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. Numerical Analysis, Vol. 13, pp. 321–326.zbMATHMathSciNetGoogle Scholar
  22. Raydan, M., (1997), The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM J. Optimization, Vol. 7, pp. 26–33.zbMATHMathSciNetCrossRefGoogle Scholar
  23. Reid, J.K., (1971), Large Sparse Sets of Linear Equations, Academic Press, London, Chapter 11, pp. 231–254.Google Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  • Roger Fletcher
    • 1
  1. 1.Department of MathematicsUniversity of DundeeDundeeScotland, UK

Personalised recommendations