# Arbitrarily Slow Convergence of Sequences of Linear Operators: A Survey

Chapter
Part of the Springer Optimization and Its Applications book series (SOIA, volume 49)

## Abstract

This is a survey (without proofs except for verifying a few new facts) of the slowest possible rate of convergence of a sequence of linear operators that converges pointwise to a linear operator. A sequence of linear operators (L n ) is said to converge to a linear operator Larbitrarily slowly (resp., almost arbitrarily slowly) provided that (L n ) converges to L pointwise, and for each sequence of real numbers (ϕ(n)) converging to 0, there exists a point x = x ϕ such that $$\|{L}_{n}(x) - L(x)\| \geq \phi (n)$$ for all n (resp., for infinitely many n). Two main “lethargy” theorems are prominent in this study, and they have numerous applications. The first lethargy theorem (Theorem  11.16) characterizes almost arbitrarily slow convergence. Applications of this lethargy theorem include the fact that a large class of polynomial operators (e.g., Bernstein, Hermite–Fejer, Landau, Fejer, and Jackson operators) all converge almost arbitrarily slowly to the identity operator. Also all the classical quadrature rules (e.g., the composite Trapezoidal Rule, composite Simpson’s Rule, and Gaussian quadrature) converge almost arbitrarily slowly to the integration functional. The second lethargy theorem (Theorem 11.21) gives useful sufficient conditions that guarantee arbitrarily slow convergence. In the particular case when the sequence of linear operators is generated by the powers of a single linear operator, there is a “dichotomy” theorem (Theorem 11.27) which states that either there is linear (fast) convergence or arbitrarily slow convergence; no other type of convergence is possible. Some applications of the dichotomy theorem include generalizations and sharpening of (1) the von Neumann-Halperin cyclic projections theorem, (2) the rate of convergence for intermittently (i.e., “almost” randomly) ordered projections, and (3) a theorem of Xu and Zikatanov.

### Keywords

Arbitrarily slow convergence Higher powers of linear operators Cyclic projections Alternating projections Randomly ordered projections Intermittently ordered projections Subspace corrections Finite elements Domain decomposition Multigrid method Rate of convergence Bernstein polynomial operators Hermite–Fejer operators Landau operators Fejer operators Jackson operators The Trapezoidal rule Simpson’s rule Gaussian quadrature

## Notes

### Acknowledgements

We are grateful to the two referees for raising some points that helped us to make the paper more complete and readable. We are also grateful to Heinz Bauschke who originally pointed out the paper [3] to us that we were unaware of at the time.

### References

1. 1.
Amemiya, I., Ando, T.: Convergence of random products of contractions in Hilbert space. Acta Sci. Math. (Szeged) 26, 239–244 (1965)Google Scholar
2. 2.
Aronszajn, N.: Theory of reproducing kernels. Trans. Amer. Math. Soc., 68, 337–403 (1950)
3. 3.
Badea, C., Grivaux, S., Müller, V.: The rate of convergence in the method of alternating projections. St. Petersburg Math. J. 22, (2010). Announced in C. R. Math. Acad. Sci. Paris 348, 53–56 (2010)Google Scholar
4. 4.
Bai, Z.D., Yin, Y.Q.: Necessary and sufficient conditions for almost sure convergence of the largest eigenvalue of a Wigner matrix. Ann. Probability 16, 1729–1741 (1988)
5. 5.
Bauschke, H.H.: A norm convergence result on random products of relaxed projections in Hilbert space. Trans. Amer. Math. Soc. 347, 1365–1373 (1995)
6. 6.
Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Review 38, 367–426 (1996)
7. 7.
Bauschke, H.H., Borwein, J.M., Lewis, A.S.: The method of cyclic projections for closed convex sets in Hilbert space. Contemporary Mathematics 204, 1–38 (1997)
8. 8.
Bauschke, H.H., Borwein, J.M., Li, W.: The strong conical hull intersection property, bounded linear regularity, Jameson’s property(G), and error bounds in convex optimization. Math. Programming (Series A) 86, 135–160 (1999)Google Scholar
9. 9.
Bauschke, H.H., Deutsch, F., Hundal, H.: Characterizing arbitrarily slow convergence in the method of alternating projections. Intl. Trans. in Op. Res. 16, 413–425 (2009)
10. 10.
Bernstein, S.N.: On the inverse problem of the theory of the best approximation of continuous functions. Sochineniya II, 292–294 (1938)Google Scholar
11. 11.
Boland, J.M., Nicolaides, R.A.: Stable and semistable low order finite elements for viscous flows. SIAM J. Numer. Anal. 22, 474–492 (1985)
12. 12.
Cheney, W.: Analysis for Applied Mathematics. Graduate Texts in Mathematics #208, Springer, New York (2001)Google Scholar
13. 13.
Combettes, P.L.: Fejér-monotonicity in convex optimization. In: C.A. Floudas and P.M. Pardalos (eds.) Encyclopedia of Optimization, Kluwer Acad. Pub. (2000)Google Scholar
14. 14.
Cover, T.M.: Rates of convergence for nearest neighbor procedures. Proc. Hawaii Intl. Conf. Systems Sciences, 413–415 (1968)Google Scholar
15. 15.
Davis, P.J.: Interpolation and Approximation. Blaisdell, New York (1963)
16. 16.
Deroïan, F.: Formation of social networks and diffusion of innovations. Research Policy 31, 835–846 (2002)
17. 17.
Deutsch, F.: The method of alternating orthogonal projections. In: S.P. Singh (ed.) Approximation Theory, Spline Functions and Applications. Kluwer Academic Publishers, The Netherlands, 105–121 (1992)Google Scholar
18. 18.
Deutsch, F.: The role of the strong conical hull intersection property in convex optimization and approximation. In: C.K. Chui and L.L. Schumaker (eds.) Approximation Theory IX, Vanderbilt University Press, Nashville, TN, 143–150 (1998)Google Scholar
19. 19.
Deutsch, F.: Best Approximation in Inner Product Spaces. Springer, New York (2001)
20. 20.
Deutsch, F., Hundal, H.: Slow convergence of sequences of linear operators I: Almost arbitrarily slow convergence. J. Approx. Theory 162, 1701–1716 (2010)
21. 21.
Deutsch, F., Hundal, H.: Slow convergence of sequences of linear operators II: Arbitrarily slow convergence. J. Approx. Theory 162, 1717–1738 (2010)
22. 22.
Deutsch, F., Ubhaya, V.A., Ward, J.D., Xu, Y.: Constrained best approximation in Hilbert space III. Applications to n-convex functions. Constr. Approx. 12, 361–384 (1996)
23. 23.
Deutsch, F., Li, W., Ward, J.D.: A dual approach to constrained interpolation from a convex subset of Hilbert space. J. Approx. Theory 80, 381–405 (1997)
24. 24.
DeVore, R.: The Approximation of Continuous Functions by Positive Linear Operators. Lecture Notes in Mathematics # 293, Springer, New York (1972)Google Scholar
25. 25.
Devroye, L.: On arbitrarily slow rates of global convergence in density estimation. Probability Theory and Related Fields 62, 475–483 (1983)
26. 26.
Devroye, L.: Another proof of a slow convergence result of Birgé. Statistics and Probability Letters 23, 63–67 (1995)
27. 27.
Devroye, L., Györfi, L., Logosi, G.: A Probabilistic Theory of Pattern Recognition. Springer, New York (1996)
28. 28.
Dye, J., Khamsi, M.A., Reich, S.: Random products of contractions in Banach spaces. Trans. Amer. Math. Soc. 325, 87–99 (1991)
29. 29.
Ghosal, S.: Convergence rates for density estimation with Bernstein polynomials. Ann. Statistics 29, 1264–1280 (2001)
30. 30.
Golightly, A., Wilkinson, D.J.: Bayesian inference for nonlinear multivariate diffusion models observed with error. Computational Statistics and Data Analysis 52, 1674–1693 (2008)
31. 31.
Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography. J. Theoretical Biol. 29, 471–481 (1970)
32. 32.
Halperin, I.: The product of projection operators. Acta Sci. Math. (Szeged) 23, 96–99 (1962)Google Scholar
33. 33.
Hanke, M., Neubauer, A., Scherzer, O.: A convergence analysis of the Landweber iteration for nonlinear ill-posed problems. Numerische Math. 72, 21–37 (1995)
34. 34.
Hewitt, E., Stromberg, K.: Real and Abstract Analysis. Springer, New York, (1965)
35. 35.
Hounsfield, G.N.: Computerized transverse axial scanning (tomography); Part I Description of system. British J. Radiol. 46, 1016–1022 (1973)
36. 36.
Hundal, H., Deutsch, F.: Two generalizations of Dykstra’s cyclic projections algorithm. Math. Programming 77, 335–355 (1997)
37. 37.
Jahnke, H.N. (ed.): A History of Analysis. History of Mathematics 24. Amer. Math. Soc., Providence, RI, London Math. Soc., London (2003)Google Scholar
38. 38.
Kincaid, D., Cheney, W.: Numerical Analysis, 2nd edn. Brooks/Cole, New York (1996)
39. 39.
Korovkin, P.P.: Linear Operators and Approximation Theory. Hindustan Publ. Corp. (India), Delhi (1960)Google Scholar
40. 40.
Müller, V.: Power bounded operators and supercyclic vectors II. Proc.Amer. Math. Soc. 133, 2997–3004 (2005)
41. 41.
Müller, V.: Spectral Theory of Linear Operators and Spectral Systems in Banach Algebras, 2nd edn. Operator Theory: Advances and Applications 139, Birkhauser, Basel (2007)Google Scholar
42. 42.
Nakano, H.: Spectal Theory in the Hilbert Space. Japan Soc. Promotion Sc., Tokyo (1953)Google Scholar
43. 43.
Neubauer, A.: On converse and saturation results for Tikhhonov regularization of linear ill-posed problems. SIAM J. Numer. Anal. 34, 517–527 (1997)
44. 44.
von Neumann, J.: On rings of operators. Reduction theory. Ann. of Math. 50, 401–485 (1949)Google Scholar
45. 45.
von Neumann, J.: Functional Operators-Vol. II. The Geometry of Orthogonal Spaces. Annals of Math. Studies #22, Princeton University Press, Princeton, NJ (1950) [This is a reprint of mimeographed lecture notes first distributed in 1933.]Google Scholar
46. 46.
Olshevsky, A., Tsitsiklis, J.N.: Convergence rates in distributed consensus and averaging. Proc. IEEE Conf. Decision Control, San Diego, CA, 3387–3392 (2006)Google Scholar
47. 47.
Powell, M.J.D.: A new algorithm for unconstrained optimization. In: J.B. Rosen, O.L. Mangasarian, and K. Ritter (eds.) Nonlinear Programming, Academic, New York (1970)Google Scholar
48. 48.
Ratschek, H., Rokne, J.G.: Efficiency of a global optimization algorithm. SIAM J. Numer. Anal. 24, 1191–1201 (1987)
49. 49.
Rhee, W., Talagrand, M.: Bad rates of convergence for the central limit theorem in Hilbert space. Ann. Prob. 12, 843–850 (1984)
50. 50.
Schock, E.: Arbitrarily slow convergence, uniform convergence and superconvergence of Galerkin-like methods. IMA Jour. Numerical Anal. 5, 153–160 (1985)
51. 51.
Schock, E.: Semi-iterative methods for the approximated solutions of ill-posed problems. Numer. Math. 50, 263–271 (1987)
52. 52.
Timan, A.F.: Theory of Approximation of Functions of a Real Variable. MacMillan, New York (1963)
53. 53.
Wiener, N.: On the factorization of matrices. Comment. Math. Helv. 29, 97–111 (1955)
54. 54.
Xu, J., Zikatanov, L.: The method of alternating projections and the method of subspace corrections in Hilbert space. J. Amer. Math. Soc. 15, 573–597 (2002)