Abstract
The Barzilai and Borwein gradient method has received a significant amount of attention in different fields of optimization. This is due to its simplicity, computational cheapness, and efficiency in practice. In this research, based on spectral analysis techniques, root-linear global convergence for the Barzilai and Borwein method is proven for strictly convex quadratic problems posed in infinite-dimensional Hilbert spaces. The applicability of these results is demonstrated for two optimization problems governed by partial differential equations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8(1), 141–148 (1988)
Raydan, M.: On the Barzilai and Borwein choice of steplength for the gradient method. IMA J. Numer. Anal. 13(3), 321–326 (1993)
Dai, Y.H., Liao, L.Z.: \(R\)-Linear convergence of the Barzilai and Borwein gradient method. IMA J. Numer. Anal. 22(1), 1–10 (2002)
Dai, Y.H., Fletcher, R.: On the asymptotic behaviour of some new gradient methods. Math. Program. 103(3, Ser. A), 541–559 (2005)
Fletcher, R.: On the Barzilai-Borwein method. In: Qi, L., Teo, K., Yang, X. (eds.) Optimization and Control with Applications, Applied Optimization, vol. 96, pp. 235–256. Springer, New York (2005)
Hall, B.C.: Quantum Theory for Mathematicians, Graduate Texts in Mathematics, vol. 267. Springer, New York (2013)
Werner, D.: Funktionalanalysis, extended edn. Springer, Berlin (2000)
Molina, B., Raydan, M.: Preconditioned Barzilai–Borwein method for the numerical solution of partial differential equations. Numer. Algorithms 13(1–2), 45–60 (1996)
Axelsson, O., Karátson, J.: On the rate of convergence of the conjugate gradient method for linear operators in Hilbert space. Numer. Funct. Anal. Optim. 23(3–4), 285–302 (2002)
Axelsson, O., Karátson, J.: Mesh independent superlinear PCG rates via compact-equivalent operators. SIAM J. Numer. Anal. 45(4), 1495–1516 (2007)
Herzog, R., Sachs, E.: Superlinear convergence of Krylov subspace methods for self-adjoint problems in Hilbert space. SIAM J. Numer. Anal. 53(3), 1304–1324 (2015)
Mardal, K.A., Winther, R.: Preconditioning discretizations of systems of partial differential equations. Numer. Linear Algebra Appl. 18(1), 1–40 (2011)
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC, 2nd edn. Springer, Cham (2017)
Lions, J.L., Magenes, E.: Non-homogeneous Boundary Value Problems and Applications, vol. I. Springer, New York (1972)
Lasiecka, I., Triggiani, R.: Sharp regularity theory for second order hyperbolic equations of Neumann type. I. \(L_2\) nonhomogeneous data. Ann. Mat. Pura Appl. (4) 157, 285–367 (1990)
Lasiecka, I., Triggiani, R.: Regularity theory of hyperbolic equations with nonhomogeneous Neumann boundary conditions. II. General boundary data. J. Differ. Equ. 94(1), 112–164 (1991)
Kröner, A., Kunisch, K., Vexler, B.: Semismooth Newton methods for optimal control of the wave equation with control constraints. SIAM J. Control Optim. 49(2), 830–858 (2011)
Lions, J.L.: Optimal control of systems governed by partial differential equations. Translated from the French by S. K. Mitter. Die Grundlehren der mathematischen Wissenschaften, Band 170. Springer, New York (1971)
Mordukhovich, B.S., Raymond, J.P.: Neumann boundary control of hyperbolic equations with pointwise state constraints. SIAM J. Control Optim. 43(4), 1354–1372, (2004/2005). (Electronic)
Tröltzsch, F., Volkwein, S.: The SQP method for control constrained optimal control of the Burgers equation. ESAIM Control Optim. Calc. Var. 6, 649–674 (2001)
Volkwein, S.: Distributed control problems for the Burgers equation. Comput. Optim. Appl. 18(2), 115–140 (2001)
Curtis, F.E., Guo, W.: Handling nonpositive curvature in a limited memory steepest descent method. IMA J. Numer. Anal. 36(2), 717–742 (2016)
De Asmundis, R., Di Serafino, D., Hager, W.W., Toraldo, G., Zhang, H.: An efficient gradient method using the Yuan steplength. Comput. Optim. Appl. 59(3), 541–563 (2014)
De Asmundis, R., di Serafino, D., Riccio, F., Toraldo, G.: On spectral properties of steepest descent methods. IMA J. Numer. Anal. 33(4), 1416–1435 (2013)
di Serafino, D., Ruggiero, V., Toraldo, G., Zanni, L.: On the steplength selection in gradient methods for unconstrained optimization. Appl. Math. Comput. 318, 176–195 (2018)
Fletcher, R.: A limited memory steepest descent method. Math. Program. 135(1–2, Ser. A), 413–436 (2012)
Yuan, Y.X.: A new stepsize for the steepest descent method. J. Comput. Math. 24(2), 149–156 (2006)
Zheng, Y., Zheng, B.: A new modified Barzilai–Borwein gradient method for the quadratic minimization problem. J. Optim. Theory Appl. 172(1), 179–186 (2017)
Acknowledgements
The work of Karl Kunisch was supported the ERC Advanced Grant 668998 (OCLOC) under the EU’s H2020 research program.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Roland Herzog.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Azmi, B., Kunisch, K. Analysis of the Barzilai-Borwein Step-Sizes for Problems in Hilbert Spaces. J Optim Theory Appl 185, 819–844 (2020). https://doi.org/10.1007/s10957-020-01677-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-020-01677-y