Abstract
We establish the weak convergence of inertial Krasnoselskii-Mann iterations towards a common fixed point of a family of quasi-nonexpansive operators, along with estimates for the non-asymptotic rate at which the residuals vanish. Strong and linear convergence are obtained in the quasi-contractive setting. In both cases, we highlight the relationship with the non-inertial case, and show that passing from one regime to the other is a continuous process in terms of the hypotheses on the parameters. Numerical illustrations are provided for an inertial primal-dual method and an inertial three-operator splitting algorithm, whose performance is superior to that of their non-inertial counterparts.
Article PDF
Similar content being viewed by others
References
Alvarez, F.: On the minimizing property of a second order dissipative system in Hilbert spaces. SIAM J. Control Optim. 38(4), 1102–1119 (2000)
Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Valued Anal. 9(1), 3–11 (2001)
Attouch, H., Cabot, A.: Convergence of a relaxed inertial forward–backward algorithm for structured monotone inclusions. Appl. Math. Optim. 80(3), 547–598 (2019)
Attouch, H., Peypouquet, J.: The rate of convergence of Nesterov’s accelerated forward-backward method is actually faster than \(1/k^{2}\). SIAM J. Optim. 26(3), 1824–1834 (2016)
Attouch, H., Peypouquet, J.: Convergence of inertial dynamics and proximal algorithms governed by maximally monotone operators. Math. Program. 174, 391–432 (2019)
Attouch, H., Briceno-Arias, L.M., Combettes, P.L.: A parallel splitting method for coupled monotone inclusions. SIAM J. Control Optim. 48(5), 3246–3270 (2010)
Attouch, H., Chbani, Z., Peypouquet, J., Redont, P.: Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Math. Program. 168(1–2), 123–175 (2018)
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC. Springer, Cham (2017). With a foreword by Hédy Attouch
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)
Boţ, R.I., Csetnek, E.R., Hendrich, C.: Inertial Douglas–Rachford splitting for monotone inclusion problems. Appl. Math. Comput. 256, 472–487 (2015)
Brézis, H., Lions, P.L.: Produits infinis de résolvantes. Isr. J. Math. 29(4), 329–345 (1978)
Briceno-Arias, L.M.: Forward-Douglas–Rachford splitting and forward-partial inverse method for solving monotone inclusions. Optimization 64(5), 1239–1261 (2015)
Briceño-Arias, L., Roldán, F.: Primal-dual splittings as fixed point iterations in the range of linear operators. J. Glob. Optim. (2022)
Cauchy, A., et al.: Méthode générale pour la résolution des systemes d’équations simultanées. C. R. Sci. Paris 25(1847), 536–538 (1847)
Chambolle, A., Dossal, C.: On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. J. Optim. Theory Appl. 166(3), 968–982 (2015)
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)
Chambolle, A., Caselles, V., Cremers, D., Novaga, M., Pock, T.: An introduction to total variation for image analysis. Theor. Found. Numer. Methods Sparse Recovery 9, 263–340 (2010)
Combettes, P.L.: Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization 53(5–6), 475–504 (2004)
Combettes, P.L., Glaudin, L.E.: Quasi-nonexpansive iterations on the affine hull of orbits: from Mann’s mean value algorithm to inertial methods. SIAM J. Optim. 27(4), 2356–2380 (2017)
Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)
Condat, L.: A primal–dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)
Cortild, D., Peypouquet, J.: Krasnoselskii-Mann iterations: Inertia, perturbations and approximation (2024). arXiv:2401.16870
Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57(11), 1413–1457 (2004)
Davis, D., Yin, W.: A three-operator splitting scheme and its optimization applications. Set-Valued Var. Anal. 25(4), 829–858 (2017)
Dong, Y.: New inertial factors of the Krasnosel’skiı-Mann iteration. Set-Valued Var. Anal. 29, 145–161 (2021)
Dong, Q.-L., Yuan, H.-b.: Accelerated Mann and cq algorithms for finding a fixed point of a nonexpansive mapping. Fixed Point Theory Appl. 2015(1), 1 (2015)
Dong, Q.-L., Cho, Y.J., Rassias, T.M.: General inertial Mann algorithms and their convergence analysis for nonexpansive mappings. Appl. Nonlinear Anal., 175–191 (2018)
Dong, Q.-L., Cho, Y.J., He, S., Pardalos, P.M., Rassias, T.M.: The Krasnosel’skii-Mann Iterative Method: Recent Progress and Applications. Springer, Berlin (2022)
Douglas, J. Jr., Rachford, H.H. Jr.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)
Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)
Goldstein, A.A.: Convex programming in Hilbert space. Bull. Am. Math. Soc. 70(5), 709–710 (1964)
Güler, O.: On the convergence of the proximal point algorithm for convex minimization. SIAM J. Control Optim. 29(2), 403–419 (1991)
Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images. Fundamentals of Algorithms, vol. 3. SIAM, Philadelphia (2006). Matrices, spectra, and filtering
Iyiola, O.S., Shehu, Y.: New convergence results for inertial Krasnoselskii–Mann iterations in Hilbert spaces with applications. Results Math. 76(2), 1–25 (2021)
Krasnosel’skii, M.A.: Two comments on the method of successive approximations. Usp. Math. Nauk 10, 123–127 (1955)
Levitin, E.S., Polyak, B.T.: Constrained minimization methods. USSR Comput. Math. Math. Phys. 6(5), 1–50 (1966)
Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)
Lorenz, D.A., Pock, T.: An inertial forward-backward algorithm for monotone inclusions. J. Math. Imaging Vis. 51(2), 311–325 (2014)
Maingé, P.-E.: Convergence theorems for inertial KM-type algorithms. J. Comput. Appl. Math. 219(1), 223–236 (2008)
Mann, W.R.: Mean value methods in iteration. Proc. Am. Math. Soc. 4(3), 506–510 (1953)
Martinet, B.: Regularisation, d’inéquations variationelles par approximations succesives. Revue Française d’informatique et de Recherche operationelle (1970)
Moudafi, A.: A reflected inertial Krasnoselskii-type algorithm for Lipschitz pseudo-contractive mappings. Bull. Iran. Math. Soc. 44, 1109–1115 (2018)
Moudafi, A., Oliny, M.: Convergence of a splitting inertial proximal method for monotone operators. J. Comput. Appl. Math. 155(2), 447–454 (2003)
Nesterov, Y.: A method for solving the convex programming problem with convergence rate \(\mathcal{O}(1/k^{2})\). Proc. Acad. Sci. USSR 269, 543–547 (1983)
Opial, Z.: Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull. Am. Math. Soc. 73(4), 591–597 (1967)
Passty, G.B.: Ergodic convergence to a zero of the sum of monotone operators in Hilbert space. J. Math. Anal. Appl. 72(2), 383–390 (1979)
Peypouquet, J.: Convex Optimization in Normed Spaces. SpringerBriefs in Optimization. Springer, Cham (2015)
Polyak, B.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4, 1–17 (1964)
Pustelnik, N.: Méthodes proximales pour la résolution de problèmes inverses: application à la tomographie par émission de positrons. PhD thesis, Université Paris-Est (2010)
Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)
Shehu, Y.: Convergence rate analysis of inertial Krasnoselskii–Mann type iteration with applications. Numer. Funct. Anal. Optim. 39(10), 1077–1091 (2018)
Shehu, Y., Gibali, A., Sagratella, S.: Inertial projection-type methods for solving quasi-variational inequalities in real Hilbert spaces. J. Optim. Theory Appl. 184(3), 877–894 (2020)
Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38(3), 667–681 (2013)
Author information
Authors and Affiliations
Ethics declarations
Competing Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Maulén, J.J., Fierro, I. & Peypouquet, J. Inertial Krasnoselskii-Mann Iterations. Set-Valued Var. Anal 32, 10 (2024). https://doi.org/10.1007/s11228-024-00713-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11228-024-00713-7
Keywords
- Krasnoselskii-Mann iterations
- Fixed points
- Nonexpansive operators
- Monotone inclusions
- Convex optimization
- Inertial methods
- Acceleration