Abstract
In a recent paper by Strohmer and Vershynin (J. Fourier Anal. Appl. 15:262–278, 2009) a “randomized Kaczmarz algorithm” is proposed for solving consistent systems of linear equations {〈a i,x〉=b i } m i=1 . In that algorithm the next equation to be used in an iterative Kaczmarz process is selected with a probability proportional to ‖a i‖2. The paper illustrates the superiority of this selection method for the reconstruction of a bandlimited function from its nonuniformly spaced sampling values.
In this note we point out that the reported success of the algorithm of Strohmer and Vershynin in their numerical simulation depends on the specific choices that are made in translating the underlying problem, whose geometrical nature is “find a common point of a set of hyperplanes”, into a system of algebraic equations. If this translation is carefully done, as in the numerical simulation provided by Strohmer and Vershynin for the reconstruction of a bandlimited function from its nonuniformly spaced sampling values, then indeed good performance may result. However, there will always be legitimate algebraic representations of the underlying problem (so that the set of solutions of the system of algebraic equations is exactly the set of points in the intersection of the hyperplanes), for which the selection method of Strohmer and Vershynin will perform in an inferior manner.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38, 367–426 (1996)
Bauschke, H.H., Borwein, J.M., Lewis, A.S.: The method of cyclic projections for closed convex sets in Hilbert space. Contemp. Math. 204, 1–38 (1997)
Bregman, L.M.: The method of successive projections for finding a common point of convex sets. Sov. Math. Dokl. 6, 688–692 (1965)
Butnariu, D., Davidi, R., Herman, G.T., Kazantsev, I.G.: Stable convergence behavior under summable perturbations of a class of projection methods for convex feasibility and optimization problems. IEEE J. Sel. Top. Signal Process. 1, 540–547 (2007)
Byrne, C.L.: Applied Iterative Methods. A.K. Peters, Wellesley (2008)
Censor, Y.: Row-action methods for huge and sparse systems and their applications. SIAM Rev. 23, 444–466 (1981)
Censor, Y., Zenios, S.A.: Parallel Optimization: Theory, Algorithms, and Applications. Oxford University Press, New York (1997)
Censor, Y., Elfving, T., Herman, G.T., Nikazad, T.: On diagonally-relaxed orthogonal projection methods. SIAM J. Sci. Comput. 30, 473–504 (2008)
Combettes, P.L.: The convex feasibility problem in image recovery. Adv. Imaging Electron Phys. 95, 155–270 (1996)
Demmel, J.W.: The probability that a numerical analysis problem is difficult. Math. Comput. 50, 449–480 (1988)
Deutsch, F., Hundal, H.: The rate of convergence for the cyclic projections algorithm I: angles between convex sets. J. Approx. Theory 142, 36–55 (2006)
Eggermont, P.P.B., Herman, G.T., Lent, A.: Iterative algorithms for large partitioned linear systems, with applications to image reconstruction. Linear Algebra Appl. 40, 37–67 (1981)
Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography. J. Theor. Biol. 29, 471–481 (1970)
Gordon, D., Mansour, R.: A geometric approach to quadratic optimization: an improved method for solving strongly underdetermined systems in CT. Inverse Probl. Sci. Eng. 15, 811–826 (2007)
Gubin, L., Polyak, B., Raik, E.: The method of projections for finding the common point of convex sets. USSR Comput. Math. Math. Phys. 7, 1–24 (1967)
Hamaker, C., Solmon, D.: The angles between the null space of X-rays. J. Math. Anal. Appl. 62, 1–23 (1978)
Herman, G.T.: The Fundamentals of Computerized Tomography: Image Reconstruction from Projections, 2nd edn. Springer, Berlin (2009)
Herman, G.T.: Algebraic reconstruction techniques in medical imaging. In: Leondes, C.T. (ed.) Medical Imaging Systems Techniques and Applications: Computational Techniques, pp. 1–42. Gordon and Breach, New York (1998)
Herman, G.T., Meyer, L.B.: Algebraic reconstruction techniques can be made computationally efficient. IEEE Trans. Med. Imag. 12, 600–609 (1993)
Jiang, M., Wang, G.: Convergence studies on iterative algorithms for image reconstruction. IEEE Trans. Med. Imag. 22, 569–579 (2003)
Kaczmarz, S.: Angenäherte Auflösung von Systemen linearer Gleichungen. Bull. Acad. Pol. Sci. Lett. A 35, 355–357 (1937)
Kayalar, S., Weinert, H.L.: Error bounds for the method of alternating projections. Math. Control Signals Syst. 1, 43–59 (1988)
Strohmer, T., Vershynin, R.: A randomized solver for linear systems with exponential convergence. In: Diaz, J., Jansen, K., Rolim, J.D.P., Zwick, U. (eds.) Approximation, Randomization and Combinatorial Optimization: Algorithms and Techniques. Lecture Notes in Computer Science, vol. 4110, pp. 499–507. Springer, Berlin (2006)
Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)
Youla, D.C.: Mathematical theory of image restoration by the method of convex projections. In: Stark, H. (ed.) Image Recovery: Theory and Applications, pp. 29–77. Academic Press, San Diego (1987)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Hans Feichtinger.
Rights and permissions
About this article
Cite this article
Censor, Y., Herman, G.T. & Jiang, M. A Note on the Behavior of the Randomized Kaczmarz Algorithm of Strohmer and Vershynin. J Fourier Anal Appl 15, 431–436 (2009). https://doi.org/10.1007/s00041-009-9077-x
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00041-009-9077-x