Skip to main content
Log in

Two-Subspace Projection Method for Coherent Overdetermined Systems

  • Published:
Journal of Fourier Analysis and Applications Aims and scope Submit manuscript

Abstract

We present a Projection onto Convex Sets (POCS) type algorithm for solving systems of linear equations. POCS methods have found many applications ranging from computer tomography to digital signal and image processing. The Kaczmarz method is one of the most popular solvers for overdetermined systems of linear equations due to its speed and simplicity. Here we introduce and analyze an extension of the Kaczmarz method that iteratively projects the estimate onto a solution space given by two randomly selected rows. We show that this projection algorithm provides exponential convergence to the solution in expectation. The convergence rate improves upon that of the standard randomized Kaczmarz method when the system has correlated rows. Experimental results confirm that in this case our method significantly outperforms the randomized Kaczmarz method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Cenker, C., Feichtinger, H.G., Mayer, M., Steier, H., Strohmer, T.: New variants of the POCS method using affine subspaces of finite codimension, with applications to irregular sampling. In: Conf. SPIE 92, Boston, pp. 299–310 (1992)

    Google Scholar 

  2. Censor, Y., Eggermont, P.P.B., Gordon, D.: Strong underrelaxation in Kaczmarz’s method for inconsistent systems. Numer. Math. 41(1), 83–92 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  3. Deutsch, F., Hundal, H.: The rate of convergence for the method of alternating projections. J. Math. Anal. Appl. 205(2), 381–405 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  4. Drineas, P., Mahoney, M.W., Muthukrishnan, S., Sarlós, T.: Faster least squares approximation. Numer. Math. 117(2), 217–249 (2011)

    Article  Google Scholar 

  5. Elfving, T., Nikazad, T., Hansen, P.C.: Semi-convergence and relaxation parameters for a class of SIRT algorithms. Electron. Trans. Numer. Anal. 37, 321–336 (2010)

    MathSciNet  MATH  Google Scholar 

  6. Feichtinger, H.G., Strohmer, T.: A Kaczmarz-based approach to nonperiodic sampling on unions of rectangular lattices. In: Proc. Conf. SampTA-95, pp. 32–37 (1995)

    Google Scholar 

  7. Galàntai, A.: On the rate of convergence of the alternating projection method in finite dimensional spaces. J. Math. Anal. Appl. 310(1), 30–44 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  8. Hanke, M., Niethammer, W.: On the acceleration of Kaczmarz’s method for inconsistent linear systems. Linear Algebra Appl. 130, 83–98 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  9. Herman, G.T., Meyer, L.B.: Algebraic reconstruction techniques can be made computationally efficient. IEEE Trans. Med. Imaging 12(3), 600–609 (1993)

    Article  Google Scholar 

  10. Kaczmarz, S.: Angenäherte Auflösung von Systemen linearer Gleichungen. Bull. Int. Acad. Polon. Sci. Lett. A A35, 335–357 (1937)

    Google Scholar 

  11. Leventhal, D., Lewis, A.S.: Randomized methods for linear constraints: convergence rates and conditioning. Math. Oper. Res. 35(3), 641–654 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  12. Natterer, F.: The Mathematics of Computerized Tomography. Wiley, New York (1986)

    MATH  Google Scholar 

  13. Needell, D.: Randomized Kaczmarz solver for noisy linear systems. BIT Numer. Math. 50(2), 395–403 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Needell, D., Tropp, J.A.: Paved with good intentions: Analysis of a randomized block Kaczmarz method (2012). Submitted

  15. Needell, D., Ward, R.: Two-subspace projection method for coherent overdetermined systems. Technical report, Claremont McKenna College (2012)

  16. Powell, A., Chen, X.: Almost sure convergence for the Kaczmarz algorithm with random measurements. J. Fourier Anal. Appl. (to appear)

  17. Recht, B., Re, C.: Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case studies, and consequences. In: Proc. of COLT (2012)

    Google Scholar 

  18. Sezan, K.M., Stark, H.: Applications of convex projection theory to image recovery in tomography and related areas. In: Stark, H. (ed.) Image Recovery: Theory and Application, pp. 415–462. Academic Press, San Diego (1987)

    Google Scholar 

  19. Strohmer, T., Vershynin, R.: A randomized solver for linear systems with exponential convergence. In: RANDOM 2006 (10th International Workshop on Randomization and Computation). Lecture Notes in Computer Science, vol. number 4110, pp. 499–507. Springer, Berlin (2006)

    Google Scholar 

  20. Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Deanna Needell.

Additional information

Communicated by Roman Vershynin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Needell, D., Ward, R. Two-Subspace Projection Method for Coherent Overdetermined Systems. J Fourier Anal Appl 19, 256–269 (2013). https://doi.org/10.1007/s00041-012-9248-z

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00041-012-9248-z

Keywords

Mathematics Subject Classification

Navigation