Skip to main content
SpringerLink
Log in

The randomized Kaczmarz algorithm with the probability distribution depending on the angle

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

In this paper we propose a new probability distribution for the randomized Kaczmarz (RK) algorithm where each row of the coefficient matrix is selected in the current iteration with the probability proportional to the square of the sine of the angle between it and the chosen row in the previous iteration. This probability distribution is helpful to accelerate the convergence of the RK algorithm. We obtain the linear convergence rate estimation in expectation for the RK algorithm with the new probability distribution (RKn), which is different from the existing results. In order to avoid the calculation and storage of probability matrix when solving the large-scale linear systems, one practical probability distribution is also proposed. Furthermore, an acceleration for the RKn algorithm and its simplified version are introduced respectively, whose core is to replace the projection onto one hyperplane with that onto the intersection of two hyperplanes. The accelerated schemes are proved to have faster convergence rates. Finally, numerical experiments are provided to illustrate the performance of the proposed algorithms by comparing with the RK algorithm and Motzkin’s algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Bai, Z.Z., Wu, W.T.: On greedy randomized Kaczmarz method for solving large sparse linear systems. SIAM J. Sci. Comput. 40, A592–A606 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)

  3. Byrne, C.L.: Iterative Optimization in Inverse Problems. CRC Press, Boca Raton (2014)

    Book  MATH  Google Scholar 

  4. Dai, L., Soltanalian, M., Pelckmans, K: On the randomized kaczmarz algorithm. IEEE Si gnal Process. Lett. 21(3), 330–333 (2014)

    Article  Google Scholar 

  5. Davis, T.A., Hu, Y: The university of florida sparse matrix collection. ACM Trans. Math. Software 38, 1–25 (2011)

    MathSciNet  MATH  Google Scholar 

  6. Goebel, K, Reich, S: Uniform convexity, hyperbolic geometry and nonexpansive mappings. Dekker, New York (1984)

  7. Gower, R.M., Richtárik, P.: Stochastic dual ascent for solving linear systems. arXiv:1512.06890 (2015)

  8. Haddock, J., Needell, D: On Motzkins method for inconsistent linear systems. Bit Numer. Math. 59, 387–401 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  9. He, S., Yang, C., Duan, P.: Realization of the hybrid method for Mann iterations. Appl. Math. Comput. 217, 4239–4247 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  10. Kaczmarz, S.: Angenäherte auflösung von Systemen linearer Gleichungen. Bull. Int. Acad. Polon. Sci. Lett. A 35, 355–357 (1937)

    MATH  Google Scholar 

  11. Liu, J., Wright, S.J.: An accelerated randomized Kaczmarz algorithm. Math. of Comput. 85, 153–178 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Ma, A., Needell, D., Ramdas, A.: Convergence properties of the randomized extended Gauss–Seidel and Kaczmarz methods. SIAM J. Matrix Anal. Appl. 36, 1590–1604 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  13. Needell, D., Ward, R.: Two-subspace projection method for coherent overdetermined systems. J. Fourier Anal. Appl. 19, 256–269 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  14. Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to express their deep gratitude to the anonymous referees for their valuable comments and suggestions, which led to a large improvement of the manuscript.

Funding

This work was supported by the Open Fund of Tianjin Key Lab for Advanced Signal Processing (2022ASP-TJ01)

Author information

Authors and Affiliations

  1. Tianjin Key Laboratory for Advanced Signal Processing and College of Science, Civil Aviation University of China, Tianjin, 300300, China

    Songnian He & Qiao-Li Dong

  2. College of Science, Civil Aviation University of China, Tianjin, 300300, China

    Xiaoxiao Li

Authors
  1. Songnian He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiao-Li Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoxiao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiao-Li Dong.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, S., Dong, QL. & Li, X. The randomized Kaczmarz algorithm with the probability distribution depending on the angle. Numer Algor 93, 415–440 (2023). https://doi.org/10.1007/s11075-022-01422-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-022-01422-8

Keywords

Mathematics Subject Classification (2010)

Search

Navigation