Arbitrarily Slow Convergence of Sequences of Linear Operators: A Survey

  • Frank Deutsch
  • Hein Hundal
Part of the Springer Optimization and Its Applications book series (SOIA, volume 49)


This is a survey (without proofs except for verifying a few new facts) of the slowest possible rate of convergence of a sequence of linear operators that converges pointwise to a linear operator. A sequence of linear operators (L n ) is said to converge to a linear operator Larbitrarily slowly (resp., almost arbitrarily slowly) provided that (L n ) converges to L pointwise, and for each sequence of real numbers (ϕ(n)) converging to 0, there exists a point x = x ϕ such that \(\|{L}_{n}(x) - L(x)\| \geq \phi (n)\) for all n (resp., for infinitely many n). Two main “lethargy” theorems are prominent in this study, and they have numerous applications. The first lethargy theorem (Theorem  11.16) characterizes almost arbitrarily slow convergence. Applications of this lethargy theorem include the fact that a large class of polynomial operators (e.g., Bernstein, Hermite–Fejer, Landau, Fejer, and Jackson operators) all converge almost arbitrarily slowly to the identity operator. Also all the classical quadrature rules (e.g., the composite Trapezoidal Rule, composite Simpson’s Rule, and Gaussian quadrature) converge almost arbitrarily slowly to the integration functional. The second lethargy theorem (Theorem 11.21) gives useful sufficient conditions that guarantee arbitrarily slow convergence. In the particular case when the sequence of linear operators is generated by the powers of a single linear operator, there is a “dichotomy” theorem (Theorem 11.27) which states that either there is linear (fast) convergence or arbitrarily slow convergence; no other type of convergence is possible. Some applications of the dichotomy theorem include generalizations and sharpening of (1) the von Neumann-Halperin cyclic projections theorem, (2) the rate of convergence for intermittently (i.e., “almost” randomly) ordered projections, and (3) a theorem of Xu and Zikatanov.


Arbitrarily slow convergence Higher powers of linear operators Cyclic projections Alternating projections Randomly ordered projections Intermittently ordered projections Subspace corrections Finite elements Domain decomposition Multigrid method Rate of convergence Bernstein polynomial operators Hermite–Fejer operators Landau operators Fejer operators Jackson operators The Trapezoidal rule Simpson’s rule Gaussian quadrature 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



We are grateful to the two referees for raising some points that helped us to make the paper more complete and readable. We are also grateful to Heinz Bauschke who originally pointed out the paper [3] to us that we were unaware of at the time.


  1. 1.
    Amemiya, I., Ando, T.: Convergence of random products of contractions in Hilbert space. Acta Sci. Math. (Szeged) 26, 239–244 (1965)Google Scholar
  2. 2.
    Aronszajn, N.: Theory of reproducing kernels. Trans. Amer. Math. Soc., 68, 337–403 (1950)MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Badea, C., Grivaux, S., Müller, V.: The rate of convergence in the method of alternating projections. St. Petersburg Math. J. 22, (2010). Announced in C. R. Math. Acad. Sci. Paris 348, 53–56 (2010)Google Scholar
  4. 4.
    Bai, Z.D., Yin, Y.Q.: Necessary and sufficient conditions for almost sure convergence of the largest eigenvalue of a Wigner matrix. Ann. Probability 16, 1729–1741 (1988)MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Bauschke, H.H.: A norm convergence result on random products of relaxed projections in Hilbert space. Trans. Amer. Math. Soc. 347, 1365–1373 (1995)MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Review 38, 367–426 (1996)MathSciNetMATHCrossRefGoogle Scholar
  7. 7.
    Bauschke, H.H., Borwein, J.M., Lewis, A.S.: The method of cyclic projections for closed convex sets in Hilbert space. Contemporary Mathematics 204, 1–38 (1997)MathSciNetGoogle Scholar
  8. 8.
    Bauschke, H.H., Borwein, J.M., Li, W.: The strong conical hull intersection property, bounded linear regularity, Jameson’s property(G), and error bounds in convex optimization. Math. Programming (Series A) 86, 135–160 (1999)Google Scholar
  9. 9.
    Bauschke, H.H., Deutsch, F., Hundal, H.: Characterizing arbitrarily slow convergence in the method of alternating projections. Intl. Trans. in Op. Res. 16, 413–425 (2009)MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Bernstein, S.N.: On the inverse problem of the theory of the best approximation of continuous functions. Sochineniya II, 292–294 (1938)Google Scholar
  11. 11.
    Boland, J.M., Nicolaides, R.A.: Stable and semistable low order finite elements for viscous flows. SIAM J. Numer. Anal. 22, 474–492 (1985)MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Cheney, W.: Analysis for Applied Mathematics. Graduate Texts in Mathematics #208, Springer, New York (2001)Google Scholar
  13. 13.
    Combettes, P.L.: Fejér-monotonicity in convex optimization. In: C.A. Floudas and P.M. Pardalos (eds.) Encyclopedia of Optimization, Kluwer Acad. Pub. (2000)Google Scholar
  14. 14.
    Cover, T.M.: Rates of convergence for nearest neighbor procedures. Proc. Hawaii Intl. Conf. Systems Sciences, 413–415 (1968)Google Scholar
  15. 15.
    Davis, P.J.: Interpolation and Approximation. Blaisdell, New York (1963)MATHGoogle Scholar
  16. 16.
    Deroïan, F.: Formation of social networks and diffusion of innovations. Research Policy 31, 835–846 (2002)CrossRefGoogle Scholar
  17. 17.
    Deutsch, F.: The method of alternating orthogonal projections. In: S.P. Singh (ed.) Approximation Theory, Spline Functions and Applications. Kluwer Academic Publishers, The Netherlands, 105–121 (1992)Google Scholar
  18. 18.
    Deutsch, F.: The role of the strong conical hull intersection property in convex optimization and approximation. In: C.K. Chui and L.L. Schumaker (eds.) Approximation Theory IX, Vanderbilt University Press, Nashville, TN, 143–150 (1998)Google Scholar
  19. 19.
    Deutsch, F.: Best Approximation in Inner Product Spaces. Springer, New York (2001)MATHGoogle Scholar
  20. 20.
    Deutsch, F., Hundal, H.: Slow convergence of sequences of linear operators I: Almost arbitrarily slow convergence. J. Approx. Theory 162, 1701–1716 (2010)MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Deutsch, F., Hundal, H.: Slow convergence of sequences of linear operators II: Arbitrarily slow convergence. J. Approx. Theory 162, 1717–1738 (2010)MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    Deutsch, F., Ubhaya, V.A., Ward, J.D., Xu, Y.: Constrained best approximation in Hilbert space III. Applications to n-convex functions. Constr. Approx. 12, 361–384 (1996)MathSciNetMATHGoogle Scholar
  23. 23.
    Deutsch, F., Li, W., Ward, J.D.: A dual approach to constrained interpolation from a convex subset of Hilbert space. J. Approx. Theory 80, 381–405 (1997)MathSciNetGoogle Scholar
  24. 24.
    DeVore, R.: The Approximation of Continuous Functions by Positive Linear Operators. Lecture Notes in Mathematics # 293, Springer, New York (1972)Google Scholar
  25. 25.
    Devroye, L.: On arbitrarily slow rates of global convergence in density estimation. Probability Theory and Related Fields 62, 475–483 (1983)MathSciNetMATHGoogle Scholar
  26. 26.
    Devroye, L.: Another proof of a slow convergence result of Birgé. Statistics and Probability Letters 23, 63–67 (1995)MathSciNetMATHCrossRefGoogle Scholar
  27. 27.
    Devroye, L., Györfi, L., Logosi, G.: A Probabilistic Theory of Pattern Recognition. Springer, New York (1996)MATHGoogle Scholar
  28. 28.
    Dye, J., Khamsi, M.A., Reich, S.: Random products of contractions in Banach spaces. Trans. Amer. Math. Soc. 325, 87–99 (1991)MathSciNetMATHCrossRefGoogle Scholar
  29. 29.
    Ghosal, S.: Convergence rates for density estimation with Bernstein polynomials. Ann. Statistics 29, 1264–1280 (2001)MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Golightly, A., Wilkinson, D.J.: Bayesian inference for nonlinear multivariate diffusion models observed with error. Computational Statistics and Data Analysis 52, 1674–1693 (2008)MathSciNetMATHCrossRefGoogle Scholar
  31. 31.
    Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography. J. Theoretical Biol. 29, 471–481 (1970)CrossRefGoogle Scholar
  32. 32.
    Halperin, I.: The product of projection operators. Acta Sci. Math. (Szeged) 23, 96–99 (1962)Google Scholar
  33. 33.
    Hanke, M., Neubauer, A., Scherzer, O.: A convergence analysis of the Landweber iteration for nonlinear ill-posed problems. Numerische Math. 72, 21–37 (1995)MathSciNetMATHCrossRefGoogle Scholar
  34. 34.
    Hewitt, E., Stromberg, K.: Real and Abstract Analysis. Springer, New York, (1965)MATHGoogle Scholar
  35. 35.
    Hounsfield, G.N.: Computerized transverse axial scanning (tomography); Part I Description of system. British J. Radiol. 46, 1016–1022 (1973)CrossRefGoogle Scholar
  36. 36.
    Hundal, H., Deutsch, F.: Two generalizations of Dykstra’s cyclic projections algorithm. Math. Programming 77, 335–355 (1997)MathSciNetMATHGoogle Scholar
  37. 37.
    Jahnke, H.N. (ed.): A History of Analysis. History of Mathematics 24. Amer. Math. Soc., Providence, RI, London Math. Soc., London (2003)Google Scholar
  38. 38.
    Kincaid, D., Cheney, W.: Numerical Analysis, 2nd edn. Brooks/Cole, New York (1996)MATHGoogle Scholar
  39. 39.
    Korovkin, P.P.: Linear Operators and Approximation Theory. Hindustan Publ. Corp. (India), Delhi (1960)Google Scholar
  40. 40.
    Müller, V.: Power bounded operators and supercyclic vectors II. Proc.Amer. Math. Soc. 133, 2997–3004 (2005)MathSciNetMATHCrossRefGoogle Scholar
  41. 41.
    Müller, V.: Spectral Theory of Linear Operators and Spectral Systems in Banach Algebras, 2nd edn. Operator Theory: Advances and Applications 139, Birkhauser, Basel (2007)Google Scholar
  42. 42.
    Nakano, H.: Spectal Theory in the Hilbert Space. Japan Soc. Promotion Sc., Tokyo (1953)Google Scholar
  43. 43.
    Neubauer, A.: On converse and saturation results for Tikhhonov regularization of linear ill-posed problems. SIAM J. Numer. Anal. 34, 517–527 (1997)MathSciNetMATHCrossRefGoogle Scholar
  44. 44.
    von Neumann, J.: On rings of operators. Reduction theory. Ann. of Math. 50, 401–485 (1949)Google Scholar
  45. 45.
    von Neumann, J.: Functional Operators-Vol. II. The Geometry of Orthogonal Spaces. Annals of Math. Studies #22, Princeton University Press, Princeton, NJ (1950) [This is a reprint of mimeographed lecture notes first distributed in 1933.]Google Scholar
  46. 46.
    Olshevsky, A., Tsitsiklis, J.N.: Convergence rates in distributed consensus and averaging. Proc. IEEE Conf. Decision Control, San Diego, CA, 3387–3392 (2006)Google Scholar
  47. 47.
    Powell, M.J.D.: A new algorithm for unconstrained optimization. In: J.B. Rosen, O.L. Mangasarian, and K. Ritter (eds.) Nonlinear Programming, Academic, New York (1970)Google Scholar
  48. 48.
    Ratschek, H., Rokne, J.G.: Efficiency of a global optimization algorithm. SIAM J. Numer. Anal. 24, 1191–1201 (1987)MathSciNetMATHCrossRefGoogle Scholar
  49. 49.
    Rhee, W., Talagrand, M.: Bad rates of convergence for the central limit theorem in Hilbert space. Ann. Prob. 12, 843–850 (1984)MathSciNetMATHCrossRefGoogle Scholar
  50. 50.
    Schock, E.: Arbitrarily slow convergence, uniform convergence and superconvergence of Galerkin-like methods. IMA Jour. Numerical Anal. 5, 153–160 (1985)MathSciNetMATHCrossRefGoogle Scholar
  51. 51.
    Schock, E.: Semi-iterative methods for the approximated solutions of ill-posed problems. Numer. Math. 50, 263–271 (1987)MathSciNetMATHCrossRefGoogle Scholar
  52. 52.
    Timan, A.F.: Theory of Approximation of Functions of a Real Variable. MacMillan, New York (1963)MATHGoogle Scholar
  53. 53.
    Wiener, N.: On the factorization of matrices. Comment. Math. Helv. 29, 97–111 (1955)MathSciNetMATHCrossRefGoogle Scholar
  54. 54.
    Xu, J., Zikatanov, L.: The method of alternating projections and the method of subspace corrections in Hilbert space. J. Amer. Math. Soc. 15, 573–597 (2002)MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of MathematicsPenn State UniversityUniversity ParkUSA

Personalised recommendations