Relaxed Steepest Descent and Cauchy-Barzilai-Borwein Method
- 408 Downloads
The negative gradient direction to find local minimizers has been associated with the classical steepest descent method which behaves poorly except for very well conditioned problems. We stress out that the poor behavior of the steepest descent methods is due to the optimal Cauchy choice of steplength and not to the choice of the search direction. We discuss over and under relaxation of the optimal steplength. In fact, we study and extend recent nonmonotone choices of steplength that significantly enhance the behavior of the method. For a new particular case (Cauchy-Barzilai-Borwein method), we present a convergence analysis and encouraging numerical results to illustrate the advantages of using nonmonotone overrelaxations of the gradient method.
Unable to display preview. Download preview PDF.
- 1.H. Akaike, “On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method,” Ann. Inst. Statist. Math. Tokyo, vol. 11, pp. 1–16, 1959.Google Scholar
- 2.J. Barzilai and J.M. Borwein, “Two point step size gradient methods,” IMA J. Numer. Anal., vol. 8, pp. 141–148, 1988.Google Scholar
- 3.C. Brezinski, Padé—Type Approximation and General Orthogonal Polynomials, Birkhauser—Verlag: Basel, 1980.Google Scholar
- 4.A. Cauchy, “Méthodes générales pour la résolution des systèmes d'équations simultanées,” C. R. Acad. Sci. Par., vol. 25, pp. 536–538, 1847.Google Scholar
- 5.Y.H. Dai and L.Z. Liao, “R-linear convergence of the Barzilai and Borwein gradient method,” Technical Report AMSS 1999-081, Academy of Mathematics and Systems Sciences, Beijing, China, 1999.Google Scholar
- 6.A. Friedlander, J.M. Martinez, B. Molina, and M. Raydan, “Gradient method with retards and generalizations,” SIAM J. Numer. Anal., vol. 36, pp. 275–289, 1999.Google Scholar
- 7.W. Glunt, T.L. Hayden, and M. Raydan, “Molecular conformations from distance matrices,” J. Comp. Chem., vol. 14, pp. 114–120, 1993.Google Scholar
- 8.D. Luenberger, Linear and Nonlinear Programming, Addison-Wesley: Menlo Park, CA, 1984.Google Scholar
- 9.M. Raydan, “On the Barzilai and Borwein choice of steplength for the gradient method,” IMA J. Numer. Anal., vol. 13, pp. 321–326, 1993.Google Scholar