Skip to main content
SpringerLink
Log in

Enhancement of the Kaczmarz algorithm with projection adjustment

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

Projection adjustment is a technique that improves the rate of convergence, as measured by the number of iterations needed to achieve a given level of performance, of the Kaczmarz algorithm (KA) for iteratively solving a system of consistent linear equations, however at the cost of requiring additional time per iteration and increased storage. This hinders the applicability of the previously published Kaczmarz algorithm with projection adjustment (KAPA) to large-scale problems. An enhancement EKAPA of KAPA that uses projection adjustment only for a small subset of the equations is proposed for significantly reducing the time and storage requirements. An analysis of the behavior of EKAPA is provided. An illustration is given to show that EKAPA using a small subset of the equations for projection adjustment can achieve a speed-up over KA similar to that of KAPA in terms of the number of iterations, but requires much less computer time and storage; hence, it is more suitable for large-scale problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. SNARK14 may be downloaded free of charge from http://turing.iimas.unam.mx/SNARK14M/.

References

  1. Allaire, G., Kaber, S.M.: Numerical Linear Algebra. Springer, New York (2008)

    Book  MATH  Google Scholar 

  2. Appleby, G., Smolarski, D.C.: A linear acceleration row action method for projecting onto subspaces. Electron. Trans. Numer. Anal. 20, 253–275 (2005)

    MathSciNet  MATH  Google Scholar 

  3. Arroyo, F., Arroyo, E., Li, X., Zhu, J.: The convergence of the block cyclic projection with an overrelaxation parameter for compressed sensing based tomography. J. Comput. Appl. Math. 280, 59–67 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bai, Z., Wu, W.: On greedy randomized Kaczmarz method for solving large sparse linear systems. SIAM J. Sci. Comput. 40(1), A592–A606 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38(3), 367–426 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  6. Brezinski, C., Redivo-Zaglia, M.: Convergence acceleration of Kaczmarz’s method. J. Eng. Math. 93(1), 3–19 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  7. Censor, Y.: Row-action methods for huge and sparse systems and their applications. SIAM Rev. 23(4), 444–466 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  8. Censor, Y.: Finite series-expansion reconstruction methods. Proc. IEEE 71 (3), 409–419 (1983)

    Article  Google Scholar 

  9. Censor, Y., Gordon, D., Gordon, R.: Component averaging: an efficient iterative parallel algorithm for large and sparse unstructured problems. Parallel Comput. 27(6), 777–808 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  10. Censor, Y., Herman, G.T., Jiang, M.: A note on the behavior of the randomized Kaczmarz algorithm of Strohmer and Vershynin. J. Fourier Anal. Appl. 15, 431–436 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  11. D’Apuzzo, M., Lapegna, M.: A parallel row projection solver for large sparse linear systems. In: 16th Euromicro Conference on Parallel, Distributed and Network-Based Processing (PDP 2008), pp 432–441 (1995)

  12. Davidi, R., Herman, G.T., Langthaler, O., Sardana, S., Ye, Z.: SNARK14: A programming system for the reconstruction of 2D images from 1D projections. http://turing.iimas.unam.mx/SNARK14M/SNARK14.pdf

  13. De Loera, J.A., Haddock, J., Needell, D.: A sampling Kaczmarz-Motzkin algorithm for linear feasibility. SIAM J. Sci. Comput. 39(5), S66–S87 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  14. dos Santos, L.T.: A parallel subgradient projections method for the convex feasibility problem. J. Comput. Appl. Math. 18(3), 307–320 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  15. Gearhart, W.B., Koshy, M.: Acceleration schemes for the method of alternating projections. J. Comput. Appl. Math. 26(3), 235–249 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  16. Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography. J. Theoret. Biol. 29(3), 471–481 (1970)

    Article  Google Scholar 

  17. Herman, G.T.: Fundamentals of Computerized Tomography: Image Reconstruction from Projections, 2nd edn. Springer, London (2009)

    Book  MATH  Google Scholar 

  18. Jiang, M., Wang, G.: Convergence of the simultaneous algebraic reconstruction technique (SART). IEEE Trans. Image Process. 12(8), 957–961 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  19. Kaczmarz, S.: Angenäherte Auflösung von Systemen linearer Gleichungen. Bull. Internat. Acad. Polo. Sci. Lettres. A. 35, 355–357 (1937)

    MATH  Google Scholar 

  20. Li, H., Haltmeier, M.: The averaged Kaczmarz iteration for solving inverse problems. SIAM J. Imaging Sci. 11(1), 618–642 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  21. Lin, C., Zang, J., Qing, A.: A fast algebraic reconstruction method for inverse problem. In: IEEE Antennas and Propagation Society International Symposium, pp 912–913 (2015)

  22. Needell, D., Srebro, N., Ward, R.: Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm. Math. Program. 155(1-2), 549–573 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  23. Needell, D., Zhao, R., Zouzias, A.: Randomized block Kaczmarz method with projection for solving least squares. Linear Algebra Appl. 484, 322–343 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  24. Nikazad, T., Abbasi, M.: An acceleration scheme for cyclic subgradient projections method. Comput. Optim. Appl. 54(1), 77–91 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  25. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  26. Petra, S., Popa, C.: Single projection Kaczmarz extended algorithms. Numer. Algor. 73(3), 791–806 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  27. Pomparău, I., Popa, C.: Supplementary projections for the acceleration of Kaczmarz algorithm. Appl. Math. Comput. 232, 104–116 (2014)

    MathSciNet  MATH  Google Scholar 

  28. Popa, C.: A hybrid Kaczmarz-conjugate gradient algorithm for image reconstruction. Math. Comput. Simul. 80(12), 2272–2285 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  29. Popa, C.: Convergence rates for Kaczmarz-type algorithms. Numer. Algor. 79 (1), 1–17 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  30. Popa, C., Zdunek, R.: Kaczmarz extended algorithm for tomographic image reconstruction from limited-data. Math. Comput. Simul. 65(6), 579–598 (2004)

    Article  MathSciNet  Google Scholar 

  31. Scolnik, H., Echebest, N., Guardarucci, M.T., Vacchino, M.C.: A class of optimized row projection methods for solving large nonsymmetric linear systems. Appl. Numer. Math. 41(4), 499–513 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  32. Sidky, E.Y., Pan, X.: Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization. Phys. Med. Biol. 53(17), 4777–4807 (2008)

    Article  Google Scholar 

  33. Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15(2), 262–278 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  34. Tanabe, K.: Projection method for solving a singular system of linear equations and its applications. Numer. Math. 17(3), 203–214 (1971)

    Article  MathSciNet  MATH  Google Scholar 

  35. Walter, E.: Numerical Methods and Optimization. Springer, Cham (2014)

    MATH  Google Scholar 

  36. Zouzias, A., Freris, N.M.: Randomized extended Kaczmarz for solving least squares. SIAM J. Matrix Anal. Appl. 34(2), 773–793 (2013)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

We are grateful to Ze Ye for programming. We thank Yair Censor for references and comments on the line acceleration methods.

Funding

The research of C. Lin was supported by the China Scholarship Council, the Fundamental Research Funds for the Central Universities (No. 2682018CX20), and the National Key R&D Program of China (No. 2018YFC0809500).

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Southwest Jiaotong University, Chengdu, 611756, Sichuan, China

    Chuan Lin

  2. PhD Program in Computer Science, The Graduate Center, City University of New York, New York, NY, 10016, USA

    Gabor T. Herman

  3. Center for Biomedical Imaging, NYU School of Medicine, New York University, New York, NY, 10016, USA

    Marcelo V. W. Zibetti

Authors
  1. Chuan Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gabor T. Herman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcelo V. W. Zibetti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chuan Lin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A preliminary report on generalizing KAPA was submitted as an abstract by Chuan Lin, Anyong Qing, and Jiefeng Zang to the 2016 Progress in Electromagnetic Research Symposium.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, C., Herman, G.T. & Zibetti, M.V.W. Enhancement of the Kaczmarz algorithm with projection adjustment. Numer Algor 85, 713–736 (2020). https://doi.org/10.1007/s11075-019-00834-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-019-00834-3

Keywords

Mathematics Subject Classification (2010)

Search

Navigation