Abstract
Our interest in this paper is to introduce a Halpern-type algorithm with both inertial terms and errors for approximating fixed point of a nonexpansive mapping. We obtain strong convergence of the sequence generated by our proposed method in real Hilbert spaces under some reasonable assumptions on the sequence of parameters. As applications, we present some strong convergence results for monotone inclusion, variational inequality problem, linear inverse problem, and LASSO problem in Compressed Sensing. Our result improves the rate of convergence of existing Halpern method for monotone inclusion, variational inequality problem, linear inverse problem and LASSO problem in compressed sensing as illustrated in our numerical examples both in finite and infinite dimensional Hilbert spaces.
Similar content being viewed by others
References
Alvarez, F.: Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in Hilbert space. SIAM J. Optim. 14, 773–782 (2004)
Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Valued Anal. 9, 3–11 (2001)
Ansari, Q.H., Köbis, E., Yao, J.-C.: Vector Variational Inequalities and Vector Optimization, Theory and Applications, Vector Optimization. Springer, Cham (2018). xiii+ 509 pp
Attouch, H., Peypouquet, J.: The rate of convergence of Nesterov’s accelerated forward-backward method is actually faster than \(\frac {1}{k^{2}}\). SIAM J. Optim. 26, 1824–1834 (2016)
Aubin, J.-P., Ekeland, I.: Applied Nonlinear Analysis. Wiley, New York (1984)
Baiocchi, C., Capelo, A.: Variational and Quasivariational Inequalities; Applications to Free Boundary Problems. Wiley, New York (1984)
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, CMS Books in Mathematics. Springer, New York (2011)
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)
Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms and Engineering Applications, MPS-SIAM Series on Optimization. SIAM, Philadelphia (2001)
van den Berg, E., Friedlander, M.P.: SPGL1: a Solver for Large-scale Sparse Reconstruction, http://www.cs.ubc.ca/labs/scl/spgl1 (2007)
Boikanyo, O.A., Morosanu, G.: Four parameter proximal point algorithms. Nonlinear Anal. 74, 544–555 (2011)
Bot, R.I., Csetnek, E.R., Meier, D.: Inducing strong convergence into the asymptotic behaviour of proximal splitting algorithms in Hilbert spaces. Optim. Methods Software, https://doi.org/10.1080/10556788.2018.1457151
Bréziz, H.: Operateur maximaux monotones. In: Mathematics Studies, vol. 5, North-Holland (1973)
Chambolle, A., Dossal, C.h.: On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. J. Optim. Theory. Appl. 166, 968–982 (2015)
Chen, P., Huang, J., Zhang, X.: A primal-dual fixed point algorithm for convex separable minimization with applications to image restoration. Inverse Probl. 29, 025011 (2013)
Chen, S., Donoho, D.L., Saunders, M.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20, 33–61 (1998)
Chidume, C.E., Chidume, C.O.: Iterative approximation of fixed points of nonexpansive mappings. J. Math. Anal. Appl. 318, 288–295 (2006)
Cholamjiak, W., Cholamjiak, P., Suantai, S.: An inertial forward-backward splitting method for solving inclusion problems in Hilbert spaces. J. Fixed Point Theory Appl. 20(2), Art. 68, 21 pp (2018)
Combettes, P.L., Wajs, V.: Signal recovery by proximal forward-backward splitting. SIAM Multiscale Model Simul. 4, 1168–1200 (2005)
Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57, 1413–1457 (2004)
Dong, Q.L., Jiang, D., Cholamjiak, P., Shehu, Y.: A strong convergence result involving an inertial forward-backward algorithm for monotone inclusions. J. Fixed Point Theory Appl. 19, 3097–3118 (2017)
Dong, Q.L., Yuan, H.B., Cho, Y.J., Rassias, Th.M.: Modified inertial Mann algorithm and inertial CQ-algorithm for nonexpansive mappings. Optim. Lett. 12(1), 87–102 (2018)
Duchi, J., Singer, Y.: Efficient online and batch learning using forward backward splitting. J. Mach. Learn. Res. 10, 2899–2934 (2009)
Facchinei, F., Pang, J. -S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. II. Springer Series in Operations Research, Springer, New York (2003)
Glowinski, R., Lions, J.-L., Trémolières, R.: Numerical Analysis of Variational Inequalities. North-Holland, Amsterdam (1981)
Goebel, K., Kirk, W.A.: On Metric Fixed Point Theory. Cambridge University Press, Cambridge (1990)
Halpern, B.: Fixed points of nonexpansive maps. Bull. Amer. Math. Soc. 73, 957–961 (1967)
Iiduka, H.: Iterative algorithm for triple-hierarchical constrained nonconvex optimization problem and its application to network bandwidth allocation. SIAM J. Optim. 22, 862–878 (2012)
Iiduka, H.: Fixed point optimization algorithms for distributed optimization in networked systems. SIAM J. Optim. 23, 1–26 (2013)
Kanzow, C., Shehu, Y.: Generalized Krasnoselskii–Mann-type iterations for nonexpansive mappings in Hilbert spaces. Comput. Optim. Appl. 67, 595–620 (2017)
Kanzow, C., Shehu, Y.: Strong convergence of a double projection-type method for monotone variational inequalities in Hilbert spaces. J. Fixed Point Theory Appl. 20(1), Art. 51, 24 pp (2018)
Khatibzadeh, H., Ranjbar, S.: Halpern type iterations for strongly quasi-nonexpansive sequences and its applications. Taiwanese J. math. 19, 1561–1576 (2015)
Kinderlehrer, D., Stampacchia, G.: An Introduction to Variational Inequalities and Their Applications. Academic Press, New York (1980)
Konnov, I.V.: Combined Relaxation Methods for Variational Inequalities. Springer-Verlag, Berlin (2001)
Korpelevich, G.M.: An extragradient method for finding saddle points and for other problems. Ékonom. i Mat. Metody 12, 747–756 (1976)
Kraikaew, R., Saejung, S.: Strong convergence of the Halpern subgradient extragradient method for solving variational inequalities in Hilbert spaces. J. Optim. Theory Appl. 163, 399–412 (2014)
Kraikaew, R., Saejung, S.: On split common fixed point problems. J. Math. Anal. Appl. 415, 513–524 (2014)
Lemaire, B.: Which fixed point does the iteration method select?. In: Recent Advances in Optimization, vol. 452, pp 154–157. Springer, Berlin (1997)
Lions, P.-L.: Approximation de points fixes de contractions. C. R. Acad. Sci. Paris Sér. A-B 284, A1357–A1359 (1977)
Lorenz, D.A., Pock, T.: An inertial forward-backward algorithm for monotone inclusions. J. Math. Imaging Vis. 51, 311–325 (2015)
Lorenz, D.A.: Constructing test instances for basis pursuit denoising. IEEE Trans. Signal Process. 61, 1210–1214 (2013)
Maingé, P.-E.: Approximation methods for common fixed points of nonexpansive mappings in Hilbert spaces. J. Math. Anal. Appl. 325, 469–479 (2007)
Maingé, P.-E.: Strong convergence of projected subgradient methods for nonsmooth and nonstrictly convex minimization. Set-Valued Anal. 16, 899–912 (2008)
Maingé, P. -E., Gobinddass, M.L.: Convergence of one-step projected gradient methods for variational inequalities. J. Optim. Theory Appl. 171, 146–168 (2016)
Malitsky, Y.u.V.: Projected reflected gradient methods for monotone variational inequalities. SIAM J. Optim. 25, 502–520 (2015)
Marino, G., Xu, H.K.: Convergence of generalized proximal point algorithm. Comm. Pure Appl. Anal. 3, 791–808 (2004)
Micchelli, C.A., Shen, L., Xu, Y.: Proximity algorithms for image models: denoising. Inverse Probl. 27, 045009 (2011)
Moudafi, A., Oliny, M.: Convergence of a splitting inertial proximal method for monotone operators. J. Comput. Appl. Math. 155, 447–454 (2003)
Nagurney, A.: Network Economics: a Variational Inequality Approach. Kluwer Academic Publishers, Dordrecht (1999)
Nakajo, K., Takahashi, W.: Strong convergence theorems for nonexpansive mappings and nonexpansive semigroups. J. Math. Anal. Appl. 279, 372–379 (2003)
Nesterov, Y.: A method for solving the convex programming problem with convergence rate \(O(\frac {1}{k^{2}})\). Dokl. Akad. Nauk SSSR 269, 543–547 (1983)
Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1, 123–231 (2013)
Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. Zh. Vychisl. Mat. Mat. Fiz. 4, 1–17 (1964)
Qin, X., Petrusel, A., Yao, J.-C.: CQ iterative algorithms for fixed points of nonexpansive mappings and split feasibility problems in Hilbert spaces. J. Nonlinear Convex Anal. 19, 157–165 (2018)
Raguet, H., Fadili, J., Peyré, G.: A generalized forward-backward splitting. SIAM J. Imaging Sci. 6, 1199–1226 (2013)
Reich, S.: Strong convergence theorems for resolvents of accretive operators in Banach spaces. J. Math. Anal. Appl. 75, 287–292 (1980)
Reich, S.: Some problems and results in fixed point theory. Contemp. Math. 21, 179–187 (1983)
Saejung, S.: Halpern’s iteration in Banach spaces. Nonlinear Anal. 73, 3431–3439 (2010)
Shehu, Y.: Iterative approximations for zeros of sum of accretive operators in Bannach spaces. J. Funct. Spaces 5973468, 9 (2016)
Shehu, Y., Cai, G.: Strong convergence result of forward-backward splitting methods for accretive operators in Banach spaces with application. RACSAM 11, 71–87 (2018)
Shioji, N., Takahashi, W.: Strong convergence of approximated sequences for nonexpansive mapping in Banach spaces. Proc. Amer. Math. Soc. 125, 3641–3645 (1997)
Suzuki, T.: Strong convergence of Krasnoselskii and Mann’s type sequences for one-parameter nonexpansive semigroups without Bochner integrals. J. Math. Anal. Appl. 305, 227–239 (2005)
Tibshirami, R.: Strong convergence of approximated sequences for nonexpansive mapping in Banach spaces. Roy. Statist. Soc. Ser. B 58, 267–288 (1996)
Wang, F., Cui, H.: On the contraction-proximal point algorithms with multi-parameters. J. Global Optim. 54, 485–491 (2012)
Wang, Y., Wang, F., Xu, H.K.: Error sensitivity for strongly convergent modifications of the proximal point algorithm. J. Optim. Theory Appl. 168, 901–916 (2016)
Wang, Y., Xu, H.-K.: Strong convergence for the proximal-gradient method. J. Nonlinear Convex Anal. 15, 581–593 (2014)
Wittmann, R.: Approximation of fixed points of nonexpansive mappings. Arch. Math. 58, 486–491 (1992)
Xu, H.K.: Iterative algorithm for nonlinear operators. J. London Math. Soc. 66, 240–256 (2002)
Xu, H.K.: Another control condition in an iterative method for nonexpansive mappings. Bull. Astral. Math. Soc. 65, 109–113 (2002)
Acknowledgments
The research of Y. Shehu is supported by the Alexander von Humboldt-Foundation and Postdoctoral Fellowship from the Institute of Science and Technology (IST), Klosterneuburg, Vienna, Austria.
The research of F. U. Ogbuisi is wholly supported by the National Research Foundation (NRF) of South Africa (S& F-DST/NRF Innovation Postdoctoral Fellowship; Grant Number: 111992). Opinions expressed and conclusions arrived are those of the authors and are not necessarily to be attributed to the NRF.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Shehu, Y., Iyiola, O.S. & Ogbuisi, F.U. Iterative method with inertial terms for nonexpansive mappings: applications to compressed sensing. Numer Algor 83, 1321–1347 (2020). https://doi.org/10.1007/s11075-019-00727-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-019-00727-5