BIT Numerical Mathematics

, Volume 50, Issue 2, pp 395–403 | Cite as

Randomized Kaczmarz solver for noisy linear systems



The Kaczmarz method is an iterative algorithm for solving systems of linear equations Ax=b. Theoretical convergence rates for this algorithm were largely unknown until recently when work was done on a randomized version of the algorithm. It was proved that for overdetermined systems, the randomized Kaczmarz method converges with expected exponential rate, independent of the number of equations in the system. Here we analyze the case where the system Ax=b is corrupted by noise, so we consider the system Axb+r where r is an arbitrary error vector. We prove that in this noisy version, the randomized method reaches an error threshold dependent on the matrix A with the same rate as in the error-free case. We provide examples showing our results are sharp in the general context.


Randomized algorithms Kaczmarz method Algebraic reconstruction technique 

Mathematics Subject Classification (2000)

65F10 65F20 65F22 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cenker, C., Feichtinger, H.G., Mayer, M., Steier, H., Strohmer, T.: New variants of the POCS method using affine subspaces of finite codimension, with applications to irregular sampling. In: Proc. SPIE: Visual Communications and Image Processing, pp. 299–310 (1992) Google Scholar
  2. 2.
    Censor, Y., Herman, G.T., Jiang, M.: A note on the behavior of the randomized Kaczmarz algorithm of Strohmer and Vershynin. J. Fourier Anal. Appl. 15, 431–436 (2009) MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Deutsch, F., Hundal, H.: The rate of convergence for the method of alternating projections. J. Math. Anal. Appl. 205(2), 381–405 (1997) MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Galàntai, A.: On the rate of convergence of the alternating projection method in finite dimensional spaces. J. Math. Anal. Appl. 310(1), 30–44 (2005) MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Hanke, M., Niethammer, W.: On the acceleration of Kaczmarz’s method for inconsistent linear systems. Linear Algebra Appl. 130, 83–98 (1990) MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Herman, G.T., Meyer, L.B.: Algebraic reconstruction techniques can be made computationally efficient. IEEE Trans. Med. Imaging 12(3), 600–609 (1993) CrossRefGoogle Scholar
  7. 7.
    Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge Univ. Press, Cambridge (1985) MATHGoogle Scholar
  8. 8.
    Kaczmarz, S.: Approximate solution of systems of linear equations. Bull. Acad. Pol. Sci., Lett. A 35, 335–357 (1937) (in German); English transl.: Int. J. Control 57(6), 1269–1271 (1993) Google Scholar
  9. 9.
    Natterer, F.: The Mathematics of Computerized Tomography. Wiley, New York (1986) MATHGoogle Scholar
  10. 10.
    Shapiro, A.: Upper bounds for nearly optimal diagonal scaling of matrices. Linear Multilinear Algebra 29, 145–147 (1991) MATHCrossRefGoogle Scholar
  11. 11.
    Strohmer, T., Vershynin, R.: A randomized solver for linear systems with exponential convergence. In: RANDOM 2006 (10th International Workshop on Randomization and Computation). Lecture Notes in Computer Science, vol. 4110, pp. 499–507. Springer, Berlin (2006) Google Scholar
  12. 12.
    Strohmer, T., Vershynin, R.: Comments on the randomized Kaczmarz method. J. Fourier Anal. Appl. 15, 437–440 (2009) MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009) MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    van der Sluis, A.: Condition numbers and equilibration of matrices. Numer. Math. 14, 14–23 (1969) MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science + Business Media B.V. 2010

Authors and Affiliations

  1. 1.Department of StatisticsStanford UniversityStanfordUSA

Personalised recommendations