Skip to main content
SpringerLink
Log in

Almost Sure Convergence of the Kaczmarz Algorithm with Random Measurements

  • Published:
Journal of Fourier Analysis and Applications Aims and scope Submit manuscript

Abstract

The Kaczmarz algorithm is an iterative method for reconstructing a signal x∈ℝd from an overcomplete collection of linear measurements y n =〈x,φ n 〉, n≥1. We prove quantitative bounds on the rate of almost sure exponential convergence in the Kaczmarz algorithm for suitable classes of random measurement vectors \(\{\varphi_{n}\}_{n=1}^{\infty} \subset {\mathbb {R}}^{d}\). Refined convergence results are given for the special case when each φ n has i.i.d. Gaussian entries and, more generally, when each φ n /∥φ n ∥ is uniformly distributed on \(\mathbb{S}^{d-1}\). This work on almost sure convergence complements the mean squared error analysis of Strohmer and Vershynin for randomized versions of the Kaczmarz algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Editorial comment on the two following letters to the editor: concerning Kaczmarz algorithms. J. Fourier Anal. Appl. 15(4) 429–430 (2009)

  2. Censor, Y., Herman, G., Jiang, M.: A note on the behavior of the randomized Kaczmarz algorithm of Strohmer and Vershynin. J. Fourier Anal. Appl. 15(4), 431–436 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  3. Ehler, M., Okoudjou, K.: Probabilistic frames: an overview. In: Casazza, P., Kutyniok, G. (eds.) Finite Frames: Theory and Applications. Birkhäuser, to appear

  4. Eldar, Y., Needell, D.: Acceleration of randomized Kaczmarz method via the Johnson-Lindenstrauss lemma. Numer. Algorithms 58(2), 163–177 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  5. Haller, R., Szwarc, R.: Kaczmarz algorithm in Hilbert space. Studia Math. 169(2), 123–132 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  6. Hwang, C.-R., Sheu, S.-J.: On the geometrical convergence of the Gibbs sampler in R d∗. J. Multivar. Anal. 66, 22–37 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  7. Kaczmarz, S.: Angenäherte Auflösung von Systemen linearer Gleichungen. Bull. Int. Acad. Pol. Sci. A, 355–357 (1937). English translation in: Approximate solution of systems of linear equations. Int. J. Control 57, 1269–1271 (1993)

    Google Scholar 

  8. Kwapień, S., Mycielski, J.: On the Kaczmarz algorithm of approximation in infinite-dimensional spaces. Studia Math. 148(1), 75–86 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  9. Leventhal, D., Lewis, A.S.: Randomized methods for linear constraints: convergence rates and conditioning. Math. Oper. Res. 35(3), 641–654 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  10. Milnor, J.: Hyperbolic geometry: the first 150 years. Bull. Am. Math. Soc. 6(1), 9–24 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  11. Natterer, F.: The Mathematics of Computerized Tomography. Wiley, New York (1986)

    MATH  Google Scholar 

  12. Needell, D.: Randomized Kaczmarz solver for noisy linear systems. BIT Numer. Math. 50(2), 395–403 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  13. Powell, A.M.: Mean squared error bounds for the Rangan-Goyal soft thresholding algorithm. Appl. Comput. Harmon. Anal. 29(3), 251–271 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Rangan, S., Goyal, V.K.: Recursive consistent estimation with bounded noise. IEEE Trans. Inf. Theory 47(1), 457–464 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  15. Rudin, W.: Real and Complex Analysis, 3rd edn. McGraw-Hill, New York (1987)

    MATH  Google Scholar 

  16. Saff, E., Totik, V.: Logarithmic Potentials with External Fields, 1st edn. Springer, New York (1997)

    MATH  Google Scholar 

  17. Sezan, K.M., Stark, H.: Applications of convex projection theory to image recovery in tomography and related areas. In: Stark, H. (ed.) Image Recovery: Theory and application, pp. 415–462. Academic Press, San Diego (1987)

    Google Scholar 

  18. Shiryaev, A.N.: Probability, 2nd edn. Springer, New York (1989)

    Google Scholar 

  19. Sra, S., Tropp, J.: Row-action methods for compressed sensing. In: Proceedings of the 2006 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Toulouse, vol. 3, pp. 868–871 (2006)

    Google Scholar 

  20. Strohmer, T., Vershynin, R.: A randomized solver for linear systems with exponential convergence. In: Approximation, Randomization and Combinatorial Optimization. Lecture Notes in Computer Science, vol. 4110, pp. 499–507. Springer, Berlin (2006)

    Chapter  Google Scholar 

  21. Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15(2), 262–278 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  22. Strohmer, T., Vershynin, R.: Comments on the randomized Kaczmarz method. J. Fourier Anal. Appl. 15(4), 437–440 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  23. Szwarc, R.: Kaczmarz algorithm in Hilbert space and tight frames. Appl. Comput. Harmon. Anal. 22(3), 382–385 (2007)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

A. Powell was supported in part by NSF DMS Grant 0811086, and gratefully acknowledges the Academia Sinica Institute of Mathematics (Taipei, Taiwan) for its hospitality and support. He especially thanks Professors Yunshyong Chou, Chii-Ruey Hwang, and Shuenn-Jyi Sheu for valuable discussions related to the material.

The authors thank the anonymous reviewers for their helpful comments and especially for pointing out the reference [9].

Author information

Authors and Affiliations

  1. Department of Mathematics, Vanderbilt University, Nashville, TN, 37240, USA

    Xuemei Chen & Alexander M. Powell

Authors
  1. Xuemei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander M. Powell
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander M. Powell.

Additional information

Communicated by Roman Vershynin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, X., Powell, A.M. Almost Sure Convergence of the Kaczmarz Algorithm with Random Measurements. J Fourier Anal Appl 18, 1195–1214 (2012). https://doi.org/10.1007/s00041-012-9237-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00041-012-9237-2

Keywords

Search

Navigation