Numerical Algorithms

, Volume 77, Issue 4, pp 1141–1157 | Cite as

A derandomization approach to recovering bandlimited signals across a wide range of random sampling rates

  • Dan Gordon
Original Paper


Reconstructing bandlimited functions from random sampling is an important problem in signal processing. Strohmer and Vershynin obtained good results for this problem by using a randomized version of the Kaczmarz algorithm (RK) and assigning to every equation a probability weight proportional to the average distance of the sample from its two nearest neighbors. However, their results are valid only for moderate to high sampling rates; in practice, it may not always be possible to obtain many samples. Experiments show that the number of projections required by RK and other Kaczmarz variants rises seemingly exponentially when the equations/variables ratio (EVR) falls below 5. CGMN, which is a CG acceleration of Kaczmarz, provides very good results for low values of EVR and it is much better than CGNR and CGNE. A derandomization method, based on an extension of the bit-reversal permutation, is combined with the weights and shown to improve the performance of CGMN and the regular (cyclic) Kaczmarz, which even outperforms RK. A byproduct of our results is the finding that signals composed mainly of high-frequency components are easier to recover.


Bandlimited functions Bit-reversal CGMN Derandomization Extended bit-reversal Low sampling rates Randomized Kaczmarz RK Signal processing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



The author would like to thank the anonymous reviewers for their helpful comments. Section 4 was added in response to the issues raised by one of the reviewers.


  1. 1.
    Björck, Å, Elfving, T: Accelerated projection methods for computing pseudoinverse solutions of systems of linear equations. BIT, 19, 145–163 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Blum, M., Floyd, R.W., Pratt, V., Rivest, R.L., Tarjan, R.E.: Linear time bounds for median computations Proc.4th Annual ACM Symp.on Theory of Computing, STOC ’72, pp. 119–124. ACM, New York (1972)Google Scholar
  3. 3.
    Byrne, C.: A unified treatment of some iterative algorithms in signal processing and image reconstruction. Inverse Probl. 20, 103–120 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Cenker, C., Feichtinger, H., Mayer, M., Steier, H., Strohmer, T.: New variants of the POCS method using affine subspaces of finite codimension, with applications to irregular sampling. In: Maragos, P. (ed.) Visual Communications and Image Processing ’92, pp. 299–310. SPIE (1992)Google Scholar
  5. 5.
    Censor, Y., Elfving, T., Herman, G.T.: Averaging strings of sequential iterations for convex feasibility problems. In: Butnariu, D., Censor, Y., Reich, S. (eds.) Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications, volume 8 of Studies in Computational Mathematics, pp. 101–113. Elsevier, Amsterdam (2001)Google Scholar
  6. 6.
    Censor, Y., Herman, G.T., Jiang, M.: A note on the behavior of the randomized Kaczmarz algorithm of Strohmer and Vershynin. J. Fourier Anal. Appl. 15, 431–436 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Cimmino, G.: Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari. La Ricerca Scientifica XVI, Series II Anno IX(1), 326–333 (1938)zbMATHGoogle Scholar
  8. 8.
    Cooley, J.W., Tukey, J.W.: An algorithm for the machine calculation of complex Fourier series. Math. Comput. 19, 297–301 (1967)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Demmel, J.: The probability that a numerical analysis problem is difficult. Math. Comput. 50(182), 449–480 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Eldar, Y.C., Needell, D.: Acceleration of randomized Kaczmarz method via the Johnson-Lindenstrauss lemma. Numer. Algor. 58, 163–177 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Feichtinger, H., Gröchenig, K.: Theory and practice of irregular sampling. In: Frazier, M. (ed.) Wavelets: Mathematics and Applications, pp. 305–363. CRC Press, Boca Raton (1994)Google Scholar
  12. 12.
    Feichtinger, H.G., Gröchenig, K., Strohmer, T.: Efficient numerical methods in non-uniform sampling theory. Numer. Math. 69, 423–440 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Gordon, D., Gordon, R.: Component-averaged row projections: A robust, block-parallel scheme for sparse linear systems. SIAM J. Sci. Comput. 27, 1092–1117 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Gordon, D., Gordon, R.: CGMN revisited: robust and efficient solution of stiff linear systems derived from elliptic partial differential equations. ACM Trans. Math. Softw. 35(3), 18,1–18,27 (2008)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Gordon, D., Gordon, R.: Solution methods for linear systems with large off-diagonal elements and discontinuous coefficients. Comput. Model. Eng. Sci. 53 (1), 23–45 (2009)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Gordon, D., Gordon, R.: CARP-CG: A robust and efficient parallel solver for linear systems, applied to strongly convection-dominated PDEs. Parallel Comput. 36(9), 495–515 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Gordon, D., Gordon, R., Turkel, E.: Compact high order schemes with gradient-direction derivatives for absorbing boundary conditions. J. Comput. Phys. 297(9), 295–315 (Sept. 2015)Google Scholar
  18. 18.
    Herman, G.T.: Fundamentals of Computerized Tomography: Image Reconstruction From Projections, 2nd edn. Springer (2009)Google Scholar
  19. 19.
    Herman, G.T., Meyer, L.B.: Algebraic reconstruction techniques can be made computationally efficient. IEEE Trans. Med. Imaging MI-12, 600–609 (1993)CrossRefGoogle Scholar
  20. 20.
    Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49, 409–436 (1952)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Kaczmarz, S.: Angenäherte Auflösung von Systemen linearer Gleichungen. Bulletin de l’Académie Polonaise des Sciences et Lettres A35, 355–357 (1937)zbMATHGoogle Scholar
  22. 22.
    Liu, J., Wright, S.J.: An accelerated randomized Kaczmarz algorithm. Math. Comput. 85(297), 153–178 (Jan. 2016)Google Scholar
  23. 23.
    Margolis, E., Eldar, Y.C.: Nonuniform sampling of periodic bandlimited signals. IEEE Trans. Signal Process. 56(7), 2728–2745 (Jul. 2008)Google Scholar
  24. 24.
    Mayer, M.: POCS-Methoden. PhD thesis, University of Vienna, Austria. (2000)
  25. 25.
    Needell, D.: Randomized Kaczmarz solver for noisy linear systems. BIT – Numer. Math. 50, 395–403 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Nesterov, Y.: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. Optim. 22(2), 341–362 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003)CrossRefzbMATHGoogle Scholar
  28. 28.
    Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Tropp, J.A., Laska, J.N., Duarte, M.F., Romberg, J.K., Baraniuk, R.G.: Beyond Nyquist: Efficient sampling of sparse bandlimited signals. IEEE Trans. Inf. Theory 56(1), 520–544 (2009)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of HaifaHaifaIsrael

Personalised recommendations