Exponential Convergence of a Randomized Kaczmarz Algorithm with Relaxation

  • Yong Cai
  • Yang Zhao
  • Yuchao Tang
Part of the Advances in Intelligent and Soft Computing book series (AINSC, volume 145)


The Kaczmarz method is a well-known iterative algorithm for solving linear system of equations Ax = b. Recently, a randomized version of the algorithm has been introduced. It was proved that for the system Ax = b or Ax ≈ b + r, where r is an arbitrary error vector, the randomized Kaczmarz algorithm converges with expected exponential rate. In the present paper, we study the randomized Kaczmarz algorithm with relaxation and prove that it converges with expected exponential rate for the system of Ax = b and Ax ≈ b + r. The numerical experiments of the randomized Kaczmarz algorithm with relaxation are provided to demonstrate the convergence results.


Iteration Number Relaxation Parameter Exponential Rate Exponential Convergence Fourier Anal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Björck, A., Elfving, T.: Accelerared projection methods for computing pseudoinverse solutions of systems of linear equations. BIT 19(2), 145–163 (1979)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Censor, Y., Gordan, D., Gordan, R.: Component averaging: An efficient iterative parallel algorithm for large sparse unstructured problems. Parallel Comput. 27, 777–808 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Censor, Y., Herman, G.T., Jiang, M.: A note on the behavior of the randomized Kaczmarz algorithm of Strohmer and Vershynin. J. Fourier Anal. Appl. 15, 431–436 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Cimmino, G.: Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari, La Ricerca Scientifica, XVI, Series II, Anno IX, 326–333 (1938)Google Scholar
  5. 5.
    Feichtinger, H.G., Cenker, C., Mayer, M., Steier, H., Strohmer, T.: New variants of the POCS method using affine subspaces of finite codimension, with applications to irregular sampling. In: Proc. SPIE: Visual Communications and Image Processing, pp. 299–310 (1992)Google Scholar
  6. 6.
    Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (ART) for the three-dimensional electron microscopy and X-ray photography. J. Theor. Biol. 29, 471–481 (1970)CrossRefGoogle Scholar
  7. 7.
    Herman, G.T., Meyer, L.B.: Algebraic reconstruction techniques can be made computationally efficient. IEEE Transactions on Medical Imaging 12(3), 600–609 (1993)CrossRefGoogle Scholar
  8. 8.
    Kaczmarz, S.: Angenäherte auflösung von systemen linearer gleichungen. Bull. Internat. Acad. Polon. Sci. Lettres A, 335–357 (1937)Google Scholar
  9. 9.
    Needell, D.: Randomized kaczmarz solver for noisty linear systems. BIT Numer. Math. 50(2), 395–403 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Strohmer, T., Vershynin, R.: A Randomized Solver for Linear Systems with Exponential Convergence. In: Díaz, J., Jansen, K., Rolim, J.D.P., Zwick, U. (eds.) APPROX 2006 and RANDOM 2006. LNCS, vol. 4110, pp. 499–507. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Strohmer, T., Vershynin, R.: Comments on the randomized Kaczmarz method. J. Fourier Anal. Appl. 15, 437–440 (2009)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Berlin Heidelberg 2012

Authors and Affiliations

  • Yong Cai
    • 1
  • Yang Zhao
    • 1
  • Yuchao Tang
    • 2
  1. 1.Department of MathematicsNanChang UniversityNanchangP.R. China
  2. 2.Institute for Information and System Science, Faculty of ScienceXi’an Jiaotong UniversityXi’anP.R. China

Personalised recommendations