Skip to main content
SpringerLink
Log in

On Multi-step Partially Randomized Extended Kaczmarz Method for Solving Large Sparse Inconsistent Linear Systems

  • Original Paper
  • Published:
Communications on Applied Mathematics and Computation Aims and scope Submit manuscript

Abstract

To enhance the computational performance of the partially randomized extended Kaczmarz (PREK) method, we propose the multi-step PREK (MPREK) method. By iteratively updating at each step, we establish a non-smooth inner-outer iteration scheme to solve the large, sparse, and inconsistent linear systems. For the MPREK method, a proof of its convergence and an upper bound on the convergence rate are given. Moreover, we show that this upper bound can be lower than that of the PREK method and the multi-step randomized extended Kaczmarz (MREK) method for certain typical choices of the inner iteration step size. Numerical experiments also indicate that, for an appropriate choice of the number of inner iteration steps, the MPREK method has a more efficient computational performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bai, Z.-Z.: Several splittings for non-Hermitian linear systems. Sci. China Ser. A Math. 51, 1339–1348 (2008)

    Article  MathSciNet  Google Scholar 

  2. Bai, Z.-Z., Pan, J.-Y.: Matrix Analysis and Computations. SIAM, Philadelphia (2021)

    Book  Google Scholar 

  3. Bai, Z.-Z., Wang, L.: On multi-step randomized extended Kaczmarz method for solving large sparse inconsistent linear systems. Appl. Numer. Math. 192, 197–213 (2023)

    Article  MathSciNet  Google Scholar 

  4. Bai, Z.-Z., Wu, W.-T.: On convergence rate of the randomized Kaczmarz method. Linear Algebra Appl. 553, 252–269 (2018)

    Article  MathSciNet  Google Scholar 

  5. Bai, Z.-Z., Wu, W.-T.: On greedy randomized Kaczmarz method for solving large sparse linear systems. SIAM J. Sci. Comput. 40, A592–A606 (2018)

    Article  MathSciNet  Google Scholar 

  6. Bai, Z.-Z., Wu, W.-T.: On relaxed greedy randomized Kaczmarz methods for solving large sparse linear systems. Appl. Math. Lett. 83, 21–26 (2018)

    Article  MathSciNet  Google Scholar 

  7. Bai, Z.-Z., Wu, W.-T.: On partially randomized extended Kaczmarz method for solving large sparse overdetermined inconsistent linear systems. Linear Algebra Appl. 578, 225–250 (2019)

    Article  MathSciNet  Google Scholar 

  8. Bai, Z.-Z., Wu, W.-T.: On greedy randomized augmented Kaczmarz method for solving large sparse inconsistent linear systems. SIAM J. Sci. Comput. 43, A3892–A3911 (2021)

    Article  MathSciNet  Google Scholar 

  9. Bai, Z.-Z., Wu, W.-T.: Randomized Kaczmarz iteration methods: algorithmic extensions and convergence theory. Jpn. J. Ind. Appl. Math. 40, 1421–1443 (2023)

    Article  MathSciNet  Google Scholar 

  10. Byrne, C.: A unifed treatment of some iterative algorithms in signal processing and image re-construction. Inverse Prob. 20, 103–120 (2003)

    Article  Google Scholar 

  11. Censor, Y.: Row-action methods for huge and sparse systems and their applications. SIAM Rev. 23, 444–466 (1981)

    Article  MathSciNet  Google Scholar 

  12. Censor, Y.: Parallel application of block-iterative methods in medical imaging and radiation therapy. Math. Program. 42, 307–325 (1988)

    Article  MathSciNet  Google Scholar 

  13. Du, K.: Tight upper bounds for the convergence of the randomized extended Kaczmarz and Gauss-Seidel algorithms. Numer. Linear Algebra Appl. 26, e2233 (2019)

    Article  MathSciNet  Google Scholar 

  14. Eggermont, P.P.B., Herman, G.T., Lent, A.: Iterative algorithms for large partitioned linear systems, with applications to image reconstruction. Linear Algebra Appl. 40, 37–67 (1981)

    Article  MathSciNet  Google Scholar 

  15. Eldar, Y.C., Needell, D.: Acceleration of randomized Kaczmarz methods via the Johnson-Lindenstrauss lemma. Numer. Algorithms 58, 163–177 (2011)

    Article  MathSciNet  Google Scholar 

  16. Feichtinger, H.G., Cenker, C., Mayer, M., Steier, H., Strohmer, T.: New variants of the POCS method using affine subspaces of finite codimension with applications to irregular sampling. Vis. Commun. Image Process. 1992, 299–310 (1818)

    Google Scholar 

  17. Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography. J. Theor. Biol. 29, 471–481 (1970)

    Article  Google Scholar 

  18. Guenther, R.B., Kerber, C.W., Killian, E.K., Smith, K.T., Wagner, S.L.: Reconstruction of objects from radiographs and the location of brain tumors. Proc. Natl. Acad. Sci. 71, 4884–4886 (1974)

    Article  MathSciNet  Google Scholar 

  19. Herman, G.T.: Fundamentals of Computerized Tomography: Image Reconstruction from Projections. Springer, Berlin (2009)

    Book  Google Scholar 

  20. Herman, G.T., Davidi, R.: Image reconstruction from a small number of projections. Inverse Prob. 24, 045011 (2008)

    Article  MathSciNet  Google Scholar 

  21. Kaczmarz, S.: Angenäherte auflösung von systemen linearer gleichungen. Bull. Int. Acad. Polon. Sci. Lett. 35, 355–357 (1937)

  22. Kak, A.C., Slaney, M.: Principles of Computerized Tomographic Imaging. SIAM, Philadelphia, PA (2001)

  23. Leventhal, D., Lewis, A.S.: Randomized methods for linear constraints: convergence rates and conditioning. Math. Oper. Res. 35, 641–654 (2010)

    Article  MathSciNet  Google Scholar 

  24. Liu, Y., Gu, C.-Q.: Variant of greedy randomized Kaczmarz for ridge regression. Appl. Numer. Math. 143, 223–246 (2019)

    Article  MathSciNet  Google Scholar 

  25. Needell, D.: Randomized Kaczmarz solver for noisy linear systems. BIT Numer. Math. 50, 395–403 (2010)

    Article  MathSciNet  Google Scholar 

  26. Needell, D., Zhao, R., Zouzias, A.: Randomized block Kaczmarz method with projection for solving least squares. Linear Algebra Appl. 484, 322–343 (2015)

    Article  MathSciNet  Google Scholar 

  27. Popa, C.: Least-squares solution of overdetermined inconsistent linear systems using Kaczmarz’s relaxation. Int. J. Comput. Math. 55, 79–89 (1995)

    Article  Google Scholar 

  28. Popa, C.: Extensions of block-projections methods with relaxation parameters to inconsistent and rank-deficient least-squares problems. BIT 38, 151–176 (1998)

    Article  MathSciNet  Google Scholar 

  29. Popa, C., Zdunek, R.: Kaczmarz extended algorithm for tomographic image reconstruction from limited data. Math. Comput. Simul. 65, 579–598 (2004)

    Article  MathSciNet  Google Scholar 

  30. Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)

    Article  MathSciNet  Google Scholar 

  31. Zhang, J.-J.: A new greedy Kaczmarz algorithm for the solution of very large linear systems. Appl. Math. Lett. 91, 207–212 (2019)

    Article  MathSciNet  Google Scholar 

  32. Zouzias, A., Freris, N.M.: Randomized extended Kaczmarz for solving least squares. SIAM J. Matrix Anal. Appl. 34, 773–793 (2013)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

Supported by the R &D Program of Beijing Municipal Education Commission, China (Grant No. KM202011232019).

Author information

Authors and Affiliations

  1. School of Applied Science, Beijing Information Science and Technology University, Beijing, 100192, China

    Jin-Feng Mao & Fang Chen

Authors
  1. Jin-Feng Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fang Chen.

Ethics declarations

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mao, JF., Chen, F. On Multi-step Partially Randomized Extended Kaczmarz Method for Solving Large Sparse Inconsistent Linear Systems. Commun. Appl. Math. Comput. (2024). https://doi.org/10.1007/s42967-024-00385-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42967-024-00385-y

Keywords

Mathematics Subject Classification

Search

Navigation