Skip to main content
SpringerLink
Log in

Convergence rate analysis for fixed-point iterations of generalized averaged nonexpansive operators

  • Published:
Journal of Fixed Point Theory and Applications Aims and scope Submit manuscript

Abstract

We estimate convergence rates for fixed-point iterations of a class of nonlinear operators which are partially motivated by convex optimization problems. We introduce the notion of the generalized averaged nonexpansive (GAN) operator with a positive exponent, and provide convergence rate analysis of the fixed-point iteration of the GAN operator. The proposed generalized averaged nonexpansiveness is weaker than averaged nonexpansiveness while stronger than nonexpansiveness. We show that the fixed-point iteration of a GAN operator with a positive exponent converges to its fixed-point and estimate the local convergence rate (the convergence rate in terms of the distance between consecutive iterates) depending on the range of the exponent. We prove that the fixed-point iteration of a GAN operator with a positive exponent strictly smaller than 1 can achieve an exponential global convergence rate (the convergence rate in terms of the distance between an iterate and the solution). Furthermore, we establish the global convergence rate of the fixed-point iteration of a GAN operator, depending on both the exponent of generalized averaged nonexpansiveness and the exponent of the H\(\ddot{\text {o}}\)lder regularity, if the GAN operator is also H\(\ddot{\text {o}}\)lder regular. We then apply the established theory to three types of convex optimization problems that appear often in data science to design fixed-point iterative algorithms for solving these optimization problems and to analyze their convergence properties.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data availability statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Agarwal, R.P., Meehan, M., O’regan, D.: Fixed Point Theory and Applications, vol. 141. Cambridge University Press, Cambridge (2001)

    Book  MATH  Google Scholar 

  2. Ahn, S., Fessler, J.A.: Globally convergent image reconstruction for emission tomography using relaxed ordered subsets algorithms. IEEE Trans. Med. Imaging 22(5), 613–626 (2003)

    Article  Google Scholar 

  3. Bailion, J.-B., Bruck, R.E., Reich, S.: On the asymptotic behavior of nonexpansive mappings and semigroups in Banach spaces. Houst. J. Math. 4(1), 1–9 (1978)

    MathSciNet  Google Scholar 

  4. Baillon, J.-B., Haddad, G.: Quelques propriétés des opérateurs angle-bornés etn-cycliquement monotones. Isr. J. Math. 26(2), 137–150 (1977)

    Article  MATH  Google Scholar 

  5. Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38(3), 367–426 (1996)

    Article  MathSciNet  Google Scholar 

  6. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Space, 2nd edn. Springer, New York (2017)

    Book  MATH  Google Scholar 

  7. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bërdëllima, A., Steidl, G.: On \(\alpha \)-firmly nonexpansive operators in \(r\)-uniformly convex spaces. Results Math. 76, 172 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bertsekas, D.P.: On the Goldstein–Levitin–Polyak gradient projection method. IEEE Trans. Autom. Control 21(2), 174–184 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  10. Bertsekas, D.P.: Convex Optimization Algorithms. Athena Scientific, Belmont (2015)

    MATH  Google Scholar 

  11. Borwein, J.M., Li, G., Tam, M.K.: Convergence rate analysis for averaged fixed point iterations in common fixed point problems. SIAM J. Optim. 27(1), 1–33 (2017)

    Article  MathSciNet  Google Scholar 

  12. Borwein, J.M., Li, G., Yao, L.: Analysis of the convergence rate for the cyclic projection algorithm applied to basic semialgebraic convex sets. SIAM J. Optim. 24(1), 498–527 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  13. Bruck, R.E., Reich, S.: Nonexpansive projections and resolvents of accretive operators in Banach spaces. Houst. J. Math. 3(4), 459–470 (1977)

    MathSciNet  MATH  Google Scholar 

  14. Byrne, C.: A unified treatment of some iterative algorithms in signal processing and image reconstruction. Inverse Probl. 20(1), 103–120 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  15. Cai, J.-F., Chan, R.H., Shen, Z.: A framelet-based image inpainting algorithm. Appl. Comput. Harmon. Anal. 24(2), 131–149 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  16. Cegielski, A.: Iterative Methods for Fixed Point Problems in Hilbert Spaces. Lecture Notes in Mathematics, vol. 2057. Springer, Heidelberg (2012)

  17. Chan, R.H., Chan, T.F., Shen, L., Shen, Z.: Wavelet algorithms for high-resolution image reconstruction. SIAM J. Sci. Comput. 24(4), 1408–1432 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  18. Chen, B., Wang, J., Zhao, H., Zheng, N., Príncipe, J.C.: Convergence of a fixed-point algorithm under maximum correntropy criterion. IEEE Signal Process. Lett. 22(10), 1723–1727 (2015)

    Article  Google Scholar 

  19. Chen, G.H.-G., Rockafellar, R.T.: Convergence rates in forward-backward splitting. SIAM J. Optim. 7(2), 421–444 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  20. Chen, P., Huang, J., Zhang, X.: A primal-dual fixed point algorithm for convex separable minimization with applications to image restoration. Inverse Probl. 29(2), 025011 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward–backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  22. Davis, D., Yin, W.: Convergence rate analysis of several splitting schemes. In: Splitting Methods in Communication. Imaging, Science, and Engineering, pp. 115–163. Springer, New York (2016)

  23. Fessler, J.A.: Penalized weighted least-squares image reconstruction for positron emission tomography. IEEE Trans. Med. Imaging 13(2), 290–300 (1994)

    Article  Google Scholar 

  24. Figueiredo, M.A.T., Nowak, R.D.: An EM algorithm for wavelet-based image restoration. IEEE Trans. Image Process. 12(8), 906–916 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  25. Figueiredo, M.A.T., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Select. Top. Signal Process. 1(4), 586–597 (2007)

    Article  Google Scholar 

  26. Hicks, T.L., Kubicek, J.D.: On the Mann iteration process in a Hilbert space. J. Math. Anal. Appl. 59(3), 498–504 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  27. Kazimierz, G., Reich, S.: Uniform Convexity, Hyperbolic Geometry, and Nonexpansive Mappings. Marcel Dekker, New York (1984)

    MATH  Google Scholar 

  28. Komodakis, N., Pesquet, J.-C.: Playing with duality: an overview of recent primal-dual approaches for solving large-scale optimization problems. IEEE Signal Process. Mag. 32(6), 31–54 (2015)

    Article  Google Scholar 

  29. Krol, A., Li, S., Shen, L., Xu, Y.: Preconditioned alternating projection algorithms for maximum a posteriori ECT reconstruction. Inverse Probl. 28(11), 115005 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  30. Li, Q., Shen, L., Xu, Y., Zhang, N.: Multi-step fixed-point proximity algorithms for solving a class of optimization problems arising from image processing. Adv. Comput. Math. 41(2), 387–422 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  31. Li, Q., Zhang, N.: Fast proximity-gradient algorithms for structured convex optimization problems. Appl. Comput. Harmon. Anal. 41(2), 491–517 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  32. Li, Z., Song, G., Xu, Y.: A fixed-point proximity approach to solving the support vector regression with the group lasso regularization. Int. J. Numer. Anal. Model. 15, 154–169 (2018)

    MathSciNet  MATH  Google Scholar 

  33. Li, Z., Song, G., Xu, Y.: A two-step fixed-point proximity algorithm for a class of non-differentiable optimization models in machine learning. J. Sci. Comput. 81(2), 923–940 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  34. Lin, Y., Schmidtlein, C.R., Li, Q., Li, S., Xu, Y.: A Krasnoselskii–Mann algorithm with an improved EM preconditioner for PET image reconstruction. IEEE Trans. Med. Imaging 38(9), 2114–2126 (2019)

    Article  Google Scholar 

  35. Lu, J., Shen, L., Xu, C., Xu, Y.: Multiplicative noise removal in imaging: an exp-model and its fixed-point proximity algorithm. Appl. Comput. Harmon. Anal. 41(2), 518–539 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  36. Măruşter, L., Măruşter, Ş: Strong convergence of the Mann iteration for \(\alpha \)-demicontractive mappings. Math. Comput. Model. 54(9–10), 2486–2492 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  37. Micchelli, C.A., Shen, L., Xu, Y.: Proximity algorithms for image models: denoising. Inverse Probl. 27(4), 045009 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  38. Micchelli, C.A., Shen, L., Xu, Y., Zeng, X.: Proximity algorithms for the L1/TV image denoising model. Adv. Comput. Math. 38(2), 401–426 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  39. Moreau, J.J.: Fonctions convexes duales et points proximaux dans un espace hilbertien. Comptes rendus hebdomadaires des séances de l’Académie des sciences 255, 2897–2899 (1962)

    MathSciNet  MATH  Google Scholar 

  40. Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87. Springer Science & Business Media, New York (2003)

    MATH  Google Scholar 

  41. Opial, Z.: Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull. Am. Math. Soc. 73(4), 591–597 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  42. Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 127–239 (2014)

    Article  Google Scholar 

  43. Polson, N.G., Scott, J.G., Willard, B.T.: Proximal algorithms in statistics and machine learning. Stat. Sci. 30(4), 559–581 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  44. Reich, S.: Weak convergence theorems for nonexpansive mappings in Banach spaces. J. Math. Anal. Appl. 67(2), 274–276 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  45. Reich, S.: Averaged mappings in the Hilbert ball. J. Math. Anal. Appl. 109(1), 199–206 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  46. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  47. Schmidtlein, C.R., Lin, Y., Li, S., Krol, A., Beattie, B.J., Humm, J.L., Xu, Y.: Relaxed ordered subset preconditioned alternating projection algorithm for PET reconstruction with automated penalty weight selection. Med. Phys. 44(8), 4083–4097 (2017)

    Article  Google Scholar 

  48. Ruder, S.: An overview of gradient descent optimization algorithms. arXiv preprintarXiv:1609.04747 (2016)

  49. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 60(1–4), 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  50. Shen, L., Xu, Y., Zeng, X.: Wavelet inpainting with the \(\ell _0\) sparse regularization. Appl. Comput. Harmon. Anal. 41(1), 26–53 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  51. Song, Y., Chai, X.: Halpern iteration for firmly type nonexpansive mappings. Nonlinear Anal. Theory Methods Appl. 71(10), 4500–4506 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  52. Song, Y., Li, Q.: Successive approximations for quasi-firmly type nonexpansive mappings. Math. Commun. 16(1), 251–264 (2011)

    MathSciNet  MATH  Google Scholar 

  53. Sra, S., Nowozin, S., Wright, S.J.: Optimization for Machine Learning. The MIT Press, Cambridge (2012)

    Google Scholar 

  54. Tseng, P.: A modified forward-backward splitting method for maximal monotone mappings. SIAM J. Control Optim. 38(2), 431–446 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  55. Zhang, T.: Solving large scale linear prediction problems using stochastic gradient descent algorithms. In Proceedings of the Twenty-First International Conference on Machine Learning, p. 116 (2004)

  56. Zheng, W., Li, S., Krol, A., Schmidtlein, C.R., Zeng, X., Xu, Y.: Sparsity promoting regularization for effective noise suppression in SPECT image reconstruction. Inverse Probl. 35(11), 115011 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  57. Zhu, Y., Wu, J., Yu, G.: A fast proximal point algorithm for \(\ell _1\)-minimization problem in compressed sensing. Appl. Math. Comput. 270, 777–784 (2015)

    MathSciNet  MATH  Google Scholar 

  58. Zorich, V.A.: Mathematical Analysis I, 2nd edn. Springer, Berlin (2015)

    Book  MATH  Google Scholar 

Download references

Acknowledgements

Yizun Lin was supported in part by Guangdong Basic and Applied Basic Research Foundation under Grant 2021A1515110541, by the Fundamental Research Funds for the Central Universities of China under Grant 21620352, by the Opening Project of Guangdong Province Key Laboratory of Computational Science at the Sun Yat-sen University under Grant 2021006, and by National Natural Science Foundation of China under Grant 62176103. Yuesheng Xu was supported in part by US National Science Foundation under grants DMS-1912958 and DMS-2208386, and by National Natural Science Foundation of China under grant 11771464. All correspondence should be addressed to Y. Xu.

Author information

Authors and Affiliations

  1. Department of Mathematics, College of Information Science and Technology, Jinan University, Guangzhou, 510632, China

    Yizun Lin

  2. Department of Mathematics and Statistics, Old Dominion University, Norfolk, VA 23529, USA

    Yuesheng Xu

Authors
  1. Yizun Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuesheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuesheng Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, Y., Xu, Y. Convergence rate analysis for fixed-point iterations of generalized averaged nonexpansive operators. J. Fixed Point Theory Appl. 24, 61 (2022). https://doi.org/10.1007/s11784-022-00972-7

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11784-022-00972-7

Keywords

Mathematics Subject Classification

Search

Navigation