Skip to main content
SpringerLink
Log in

On the relation between the randomized extended Kaczmarz algorithm and coordinate descent

  • Published:
BIT Numerical Mathematics Aims and scope Submit manuscript

Abstract

In this note we compare the randomized extended Kaczmarz (EK) algorithm and randomized coordinate descent (CD) for solving the full-rank overdetermined linear least-squares problem and prove that CD needs fewer operations for satisfying the same residual-related termination criteria. For the general least-squares problems, we show that first running CD to compute the residual and then standard Kaczmarz on the resulting consistent system is more efficient than EK.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Hanke, M., Niethammer, W.: On the acceleration of Kaczmarzs method for inconsistent linear systems. Linear Algebra Appl. 130, 83–98 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  2. Leventhal, D., Lewis, A.S.: Randomized methods for linear constraints: convergence rates and conditioning. Math. Oper. Res. 35(3), 641–654 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  3. Needell, D.: Randomized Kaczmarz solver for noisy linear systems. BIT Numer. Math. 50, 395–403 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  4. Nesterov, Yu.: Efficiency of coordinate descent methods on huge scale optimization problems. SIAM J. Optim. 22(2), 341–362 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  5. Popa, C.: Least-squares solution of overdetermined inconsistent linear systems using Kaczmarzs relaxation. Int. J. Comput. Math. 55, 79–89 (1995)

    Article  MATH  Google Scholar 

  6. Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  7. Zouzias, A., Freris, N.M.: Randomized extended Kaczmarz for solving least squares. SIAM J. Matrix Anal. Appl. 34(2), 773–793 (2013)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The author is indebted to Liang Dai for stimulating discussions on the Kaczmarz algorithm and for pointing out [7] and to the reviewers for their constructive comments.

Author information

Authors and Affiliations

  1. Department of Automatic Control and Computers, University Politehnica of Bucharest, Spl. Independenţei 313, 060042, Bucharest, Romania

    Bogdan Dumitrescu

  2. Department of Signal Processing, Tampere University of Technology, Tampere, Finland

    Bogdan Dumitrescu

Authors
  1. Bogdan Dumitrescu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bogdan Dumitrescu.

Additional information

Communicated by Rosemary Renaut.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dumitrescu, B. On the relation between the randomized extended Kaczmarz algorithm and coordinate descent. Bit Numer Math 55, 1005–1015 (2015). https://doi.org/10.1007/s10543-014-0526-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10543-014-0526-9

Keywords

Mathematics Subject Classification

Search

Navigation