Abstract
We present a Projection onto Convex Sets (POCS) type algorithm for solving systems of linear equations. POCS methods have found many applications ranging from computer tomography to digital signal and image processing. The Kaczmarz method is one of the most popular solvers for overdetermined systems of linear equations due to its speed and simplicity. Here we introduce and analyze an extension of the Kaczmarz method that iteratively projects the estimate onto a solution space given by two randomly selected rows. We show that this projection algorithm provides exponential convergence to the solution in expectation. The convergence rate improves upon that of the standard randomized Kaczmarz method when the system has correlated rows. Experimental results confirm that in this case our method significantly outperforms the randomized Kaczmarz method.
Similar content being viewed by others
References
Cenker, C., Feichtinger, H.G., Mayer, M., Steier, H., Strohmer, T.: New variants of the POCS method using affine subspaces of finite codimension, with applications to irregular sampling. In: Conf. SPIE 92, Boston, pp. 299–310 (1992)
Censor, Y., Eggermont, P.P.B., Gordon, D.: Strong underrelaxation in Kaczmarz’s method for inconsistent systems. Numer. Math. 41(1), 83–92 (1983)
Deutsch, F., Hundal, H.: The rate of convergence for the method of alternating projections. J. Math. Anal. Appl. 205(2), 381–405 (1997)
Drineas, P., Mahoney, M.W., Muthukrishnan, S., Sarlós, T.: Faster least squares approximation. Numer. Math. 117(2), 217–249 (2011)
Elfving, T., Nikazad, T., Hansen, P.C.: Semi-convergence and relaxation parameters for a class of SIRT algorithms. Electron. Trans. Numer. Anal. 37, 321–336 (2010)
Feichtinger, H.G., Strohmer, T.: A Kaczmarz-based approach to nonperiodic sampling on unions of rectangular lattices. In: Proc. Conf. SampTA-95, pp. 32–37 (1995)
Galàntai, A.: On the rate of convergence of the alternating projection method in finite dimensional spaces. J. Math. Anal. Appl. 310(1), 30–44 (2005)
Hanke, M., Niethammer, W.: On the acceleration of Kaczmarz’s method for inconsistent linear systems. Linear Algebra Appl. 130, 83–98 (1990)
Herman, G.T., Meyer, L.B.: Algebraic reconstruction techniques can be made computationally efficient. IEEE Trans. Med. Imaging 12(3), 600–609 (1993)
Kaczmarz, S.: Angenäherte Auflösung von Systemen linearer Gleichungen. Bull. Int. Acad. Polon. Sci. Lett. A A35, 335–357 (1937)
Leventhal, D., Lewis, A.S.: Randomized methods for linear constraints: convergence rates and conditioning. Math. Oper. Res. 35(3), 641–654 (2010)
Natterer, F.: The Mathematics of Computerized Tomography. Wiley, New York (1986)
Needell, D.: Randomized Kaczmarz solver for noisy linear systems. BIT Numer. Math. 50(2), 395–403 (2010)
Needell, D., Tropp, J.A.: Paved with good intentions: Analysis of a randomized block Kaczmarz method (2012). Submitted
Needell, D., Ward, R.: Two-subspace projection method for coherent overdetermined systems. Technical report, Claremont McKenna College (2012)
Powell, A., Chen, X.: Almost sure convergence for the Kaczmarz algorithm with random measurements. J. Fourier Anal. Appl. (to appear)
Recht, B., Re, C.: Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case studies, and consequences. In: Proc. of COLT (2012)
Sezan, K.M., Stark, H.: Applications of convex projection theory to image recovery in tomography and related areas. In: Stark, H. (ed.) Image Recovery: Theory and Application, pp. 415–462. Academic Press, San Diego (1987)
Strohmer, T., Vershynin, R.: A randomized solver for linear systems with exponential convergence. In: RANDOM 2006 (10th International Workshop on Randomization and Computation). Lecture Notes in Computer Science, vol. number 4110, pp. 499–507. Springer, Berlin (2006)
Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Roman Vershynin.
Rights and permissions
About this article
Cite this article
Needell, D., Ward, R. Two-Subspace Projection Method for Coherent Overdetermined Systems. J Fourier Anal Appl 19, 256–269 (2013). https://doi.org/10.1007/s00041-012-9248-z
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00041-012-9248-z