Skip to main content
SpringerLink
Log in

A derandomization approach to recovering bandlimited signals across a wide range of random sampling rates

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

Reconstructing bandlimited functions from random sampling is an important problem in signal processing. Strohmer and Vershynin obtained good results for this problem by using a randomized version of the Kaczmarz algorithm (RK) and assigning to every equation a probability weight proportional to the average distance of the sample from its two nearest neighbors. However, their results are valid only for moderate to high sampling rates; in practice, it may not always be possible to obtain many samples. Experiments show that the number of projections required by RK and other Kaczmarz variants rises seemingly exponentially when the equations/variables ratio (EVR) falls below 5. CGMN, which is a CG acceleration of Kaczmarz, provides very good results for low values of EVR and it is much better than CGNR and CGNE. A derandomization method, based on an extension of the bit-reversal permutation, is combined with the weights and shown to improve the performance of CGMN and the regular (cyclic) Kaczmarz, which even outperforms RK. A byproduct of our results is the finding that signals composed mainly of high-frequency components are easier to recover.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Björck, Å, Elfving, T: Accelerated projection methods for computing pseudoinverse solutions of systems of linear equations. BIT, 19, 145–163 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  2. Blum, M., Floyd, R.W., Pratt, V., Rivest, R.L., Tarjan, R.E.: Linear time bounds for median computations Proc.4th Annual ACM Symp.on Theory of Computing, STOC ’72, pp. 119–124. ACM, New York (1972)

  3. Byrne, C.: A unified treatment of some iterative algorithms in signal processing and image reconstruction. Inverse Probl. 20, 103–120 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  4. Cenker, C., Feichtinger, H., Mayer, M., Steier, H., Strohmer, T.: New variants of the POCS method using affine subspaces of finite codimension, with applications to irregular sampling. In: Maragos, P. (ed.) Visual Communications and Image Processing ’92, pp. 299–310. SPIE (1992)

  5. Censor, Y., Elfving, T., Herman, G.T.: Averaging strings of sequential iterations for convex feasibility problems. In: Butnariu, D., Censor, Y., Reich, S. (eds.) Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications, volume 8 of Studies in Computational Mathematics, pp. 101–113. Elsevier, Amsterdam (2001)

    Google Scholar 

  6. Censor, Y., Herman, G.T., Jiang, M.: A note on the behavior of the randomized Kaczmarz algorithm of Strohmer and Vershynin. J. Fourier Anal. Appl. 15, 431–436 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  7. Cimmino, G.: Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari. La Ricerca Scientifica XVI, Series II Anno IX(1), 326–333 (1938)

    MATH  Google Scholar 

  8. Cooley, J.W., Tukey, J.W.: An algorithm for the machine calculation of complex Fourier series. Math. Comput. 19, 297–301 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  9. Demmel, J.: The probability that a numerical analysis problem is difficult. Math. Comput. 50(182), 449–480 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  10. Eldar, Y.C., Needell, D.: Acceleration of randomized Kaczmarz method via the Johnson-Lindenstrauss lemma. Numer. Algor. 58, 163–177 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Feichtinger, H., Gröchenig, K.: Theory and practice of irregular sampling. In: Frazier, M. (ed.) Wavelets: Mathematics and Applications, pp. 305–363. CRC Press, Boca Raton (1994)

    Google Scholar 

  12. Feichtinger, H.G., Gröchenig, K., Strohmer, T.: Efficient numerical methods in non-uniform sampling theory. Numer. Math. 69, 423–440 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  13. Gordon, D., Gordon, R.: Component-averaged row projections: A robust, block-parallel scheme for sparse linear systems. SIAM J. Sci. Comput. 27, 1092–1117 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  14. Gordon, D., Gordon, R.: CGMN revisited: robust and efficient solution of stiff linear systems derived from elliptic partial differential equations. ACM Trans. Math. Softw. 35(3), 18,1–18,27 (2008)

    Article  MathSciNet  Google Scholar 

  15. Gordon, D., Gordon, R.: Solution methods for linear systems with large off-diagonal elements and discontinuous coefficients. Comput. Model. Eng. Sci. 53 (1), 23–45 (2009)

    MathSciNet  MATH  Google Scholar 

  16. Gordon, D., Gordon, R.: CARP-CG: A robust and efficient parallel solver for linear systems, applied to strongly convection-dominated PDEs. Parallel Comput. 36(9), 495–515 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  17. Gordon, D., Gordon, R., Turkel, E.: Compact high order schemes with gradient-direction derivatives for absorbing boundary conditions. J. Comput. Phys. 297(9), 295–315 (Sept. 2015)

  18. Herman, G.T.: Fundamentals of Computerized Tomography: Image Reconstruction From Projections, 2nd edn. Springer (2009)

  19. Herman, G.T., Meyer, L.B.: Algebraic reconstruction techniques can be made computationally efficient. IEEE Trans. Med. Imaging MI-12, 600–609 (1993)

    Article  Google Scholar 

  20. Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49, 409–436 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  21. Kaczmarz, S.: Angenäherte Auflösung von Systemen linearer Gleichungen. Bulletin de l’Académie Polonaise des Sciences et Lettres A35, 355–357 (1937)

    MATH  Google Scholar 

  22. Liu, J., Wright, S.J.: An accelerated randomized Kaczmarz algorithm. Math. Comput. 85(297), 153–178 (Jan. 2016)

  23. Margolis, E., Eldar, Y.C.: Nonuniform sampling of periodic bandlimited signals. IEEE Trans. Signal Process. 56(7), 2728–2745 (Jul. 2008)

  24. Mayer, M.: POCS-Methoden. PhD thesis, University of Vienna, Austria. http://univie.ac.at/nuhag-php/bibtex/open_files/ma00_mayerPOCS.pdf (2000)

  25. Needell, D.: Randomized Kaczmarz solver for noisy linear systems. BIT – Numer. Math. 50, 395–403 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  26. Nesterov, Y.: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. Optim. 22(2), 341–362 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  27. Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003)

    Book  MATH  Google Scholar 

  28. Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  29. Tropp, J.A., Laska, J.N., Duarte, M.F., Romberg, J.K., Baraniuk, R.G.: Beyond Nyquist: Efficient sampling of sparse bandlimited signals. IEEE Trans. Inf. Theory 56(1), 520–544 (2009)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The author would like to thank the anonymous reviewers for their helpful comments. Section 4 was added in response to the issues raised by one of the reviewers.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Haifa, Haifa, 34988, Israel

    Dan Gordon

Authors
  1. Dan Gordon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dan Gordon.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gordon, D. A derandomization approach to recovering bandlimited signals across a wide range of random sampling rates. Numer Algor 77, 1141–1157 (2018). https://doi.org/10.1007/s11075-017-0356-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-017-0356-3

Keywords

Search

Navigation