Random Sampling in Bounded Orthonormal Systems

  • Simon Foucart
  • Holger Rauhut
Chapter
Part of the Applied and Numerical Harmonic Analysis book series (ANHA)

Abstract

This chapter considers the recovery of signals with a sparse expansion in a bounded orthonormal system. After an inventory of such bounded orthonormal systems including the Fourier systems, theoretical limitations specific to this situation are obtained for the minimal number of samples. Then, using this number of random samples, nonuniform sparse recovery is proved to be possible via ℓ1-minimization. Next, using a slightly higher number of random samples, uniform sparse recovery is also proved to be possible via a variety of algorithms. This is derived via establishing the restricted isometry property for the associated random sampling matrix—the random partial Fourier matrix is a particular case. Finally, a connection to the Λ 1-problem is pointed out.

Keywords

sampling matrix random matrices bounded orthonormal systems discrete bounded orthonormal system partial Fourier matrix partial Hadamard matrix uncertainty principles nonuniform recovery 1-minimization golfing scheme restricted isometry property Λ1-problem 

References

  1. 1.
    P. Abrial, Y. Moudden, J.-L. Starck, J. Fadili, J. Delabrouille, M. Nguyen, CMB data analysis and sparsity. Stat. Meth. 5, 289–298 (2008)MathSciNetMATHCrossRefGoogle Scholar
  2. 7.
    N. Ailon, B. Chazelle, The fast Johnson-Lindenstrauss transform and approximate nearest neighbors. SIAM J. Comput. 39(1), 302–322 (2009)MathSciNetMATHCrossRefGoogle Scholar
  3. 8.
    N. Ailon, E. Liberty, Fast dimension reduction using Rademacher series on dual BCH codes. Discrete Comput. Geom. 42(4), 615–630 (2009)MathSciNetMATHCrossRefGoogle Scholar
  4. 9.
    N. Ailon, E. Liberty, Almost optimal unrestricted fast Johnson-Lindenstrauss transform. In Proceedings of the 22nd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), San Francisco, USA, 22–25 January 2011Google Scholar
  5. 10.
    N. Ailon, H. Rauhut, Fast and RIP-optimal transforms. Preprint (2013)Google Scholar
  6. 17.
    G. Andrews, R. Askey, R. Roy, in Special Functions. Encyclopedia of Mathematics and its Applications, vol. 71 (Cambridge University Press, Cambridge, 1999)Google Scholar
  7. 26.
    W. Bajwa, J. Haupt, G. Raz, S. Wright, R. Nowak, Toeplitz-structured compressed sensing matrices. In Proc. IEEE Stat. Sig. Proc. Workshop, pp. 294–298, 2007Google Scholar
  8. 62.
    J. Bourgain, Bounded orthogonal systems and the Λ(p)-set problem. Acta Math. 162(3–4), 227–245 (1989)MathSciNetMATHCrossRefGoogle Scholar
  9. 63.
    J. Bourgain, \(\Lambda _{p}\)-sets in analysis: results, problems and related aspects. In Handbook of the Geometry of Banach Spaces, vol. 1, ed. by W. B. Johnson, J. Lindenstrauss (North-Holland, Amsterdam, 2001), pp. 195–232Google Scholar
  10. 80.
    N. Burq, S. Dyatlov, R. Ward, M. Zworski, Weighted eigenfunction estimates with applications to compressed sensing. SIAM J. Math. Anal. 44(5), 3481–3501 (2012)MathSciNetMATHCrossRefGoogle Scholar
  11. 88.
    E.J. Candès, Y. Plan, A probabilistic and RIPless theory of compressed sensing. IEEE Trans. Inform. Theor. 57(11), 7235–7254 (2011)CrossRefGoogle Scholar
  12. 92.
    E.J. Candès, J. Romberg, Quantitative robust uncertainty principles and optimally sparse decompositions. Found. Comput. Math. 6(2), 227–254 (2006)MathSciNetMATHCrossRefGoogle Scholar
  13. 93.
    E.J. Candès, J. Romberg, Sparsity and incoherence in compressive sampling. Inverse Probl. 23(3), 969–985 (2007)MATHCrossRefGoogle Scholar
  14. 94.
    E.J. Candès, J. Romberg, T. Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theor. 52(2), 489–509 (2006)MATHCrossRefGoogle Scholar
  15. 97.
    E.J. Candès, T. Tao, Near optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inform. Theor. 52(12), 5406–5425 (2006)CrossRefGoogle Scholar
  16. 103.
    D. Chafaï, O. Guédon, G. Lecué, A. Pajor, Interactions Between Compressed Sensing, Random Matrices and High-dimensional Geometry, Panoramas et Synthèses, vol. 37 (Société Mathématique de France, to appear)Google Scholar
  17. 115.
    M. Cheraghchi, V. Guruswami, A. Velingker, Restricted isometry of Fourier matrices and list decodability of random linear codes. Preprint (2012)Google Scholar
  18. 117.
    T. Chihara, An Introduction to Orthogonal Polynomials (Gordon and Breach Science Publishers, New York 1978)MATHGoogle Scholar
  19. 119.
    O. Christensen, An Introduction to Frames and Riesz Bases. Applied and Numerical Harmonic Analysis (Birkhäuser, Boston, 2003)MATHGoogle Scholar
  20. 122.
    A. Cohen, Numerical Analysis of Wavelet Methods (North-Holland, Amsterdam, 2003)MATHGoogle Scholar
  21. 126.
    R. Coifman, F. Geshwind, Y. Meyer, Noiselets. Appl. Comput. Harmon. Anal. 10(1), 27–44 (2001)MathSciNetMATHCrossRefGoogle Scholar
  22. 137.
    I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 61 (SIAM, Philadelphia, 1992)Google Scholar
  23. 158.
    D.L. Donoho, X. Huo, Uncertainty principles and ideal atomic decompositions. IEEE Trans. Inform. Theor. 47(7), 2845–2862 (2001)MathSciNetMATHCrossRefGoogle Scholar
  24. 164.
    D.L. Donoho, P. Stark, Uncertainty principles and signal recovery. SIAM J. Appl. Math. 48(3), 906–931 (1989)MathSciNetCrossRefGoogle Scholar
  25. 181.
    M. Elad, A.M. Bruckstein, A generalized uncertainty principle and sparse representation in pairs of bases. IEEE Trans. Inform. Theor. 48(9), 2558–2567 (2002)MathSciNetMATHCrossRefGoogle Scholar
  26. 191.
    H.G. Feichtinger, F. Luef, T. Werther, A guided tour from linear algebra to the foundations of Gabor analysis. In Gabor and Wavelet Frames. Lecture Notes Series, Institute for Mathematical Sciences National University of Singapore, vol. 10 (World Sci. Publ., Hackensack, 2007), pp. 1–49Google Scholar
  27. 192.
    H.G. Feichtinger, T. Strohmer, Gabor Analysis and Algorithms: Theory and Applications (Birkhäuser, Boston, 1998)MATHCrossRefGoogle Scholar
  28. 198.
    G.B. Folland, Fourier Analysis and Its Applications (Wadsworth and Brooks, Pacific Grove, 1992)MATHGoogle Scholar
  29. 199.
    G.B. Folland, A Course in Abstract Harmonic Analysis (CRC Press, Boca Raton, 1995)MATHGoogle Scholar
  30. 200.
    G.B. Folland, A. Sitaram, The uncertainty principle: A mathematical survey. J. Fourier Anal. Appl. 3(3), 207–238 (1997)MathSciNetMATHCrossRefGoogle Scholar
  31. 223.
    A.C. Gilbert, S. Muthukrishnan, S. Guha, P. Indyk, M. Strauss, Near-Optimal Sparse Fourier Representations via Sampling. In Proceedings of the Thiry-fourth Annual ACM Symposium on Theory of Computing, STOC ’02, pp. 152–161, ACM, New York, NY, USA, 2002Google Scholar
  32. 231.
    G. Golub, C.F. van Loan, Matrix Computations, 3rd edn. (The Johns Hopkins University Press, Baltimore, MD, 1996)MATHGoogle Scholar
  33. 236.
    L. Grafakos, Modern Fourier Analysis, 2nd edn. Graduate Texts in Mathematics, vol. 250 (Springer, New York, 2009)Google Scholar
  34. 244.
    K. Gröchenig, Foundations of Time-Frequency Analysis. Applied and Numerical Harmonic Analysis (Birkhäuser, Boston, MA, 2001)Google Scholar
  35. 245.
    D. Gross, Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inform. Theor. 57(3), 1548–1566 (2011)CrossRefGoogle Scholar
  36. 248.
    O. Guédon, S. Mendelson, A. Pajor, N. Tomczak-Jaegermann, Majorizing measures and proportional subsets of bounded orthonormal systems. Rev. Mat. Iberoam. 24(3), 1075–1095 (2008)MathSciNetMATHCrossRefGoogle Scholar
  37. 261.
    H. Hassanieh, P. Indyk, D. Katabi, E. Price, Nearly optimal sparse Fourier transform. In Proceedings of the 44th Symposium on Theory of Computing, STOC ’12, pp. 563–578, ACM, New York, NY, USA, 2012Google Scholar
  38. 263.
    J. Haupt, W. Bajwa, G. Raz, R. Nowak, Toeplitz compressed sensing matrices with applications to sparse channel estimation. IEEE Trans. Inform. Theor. 56(11), 5862–5875 (2010)MathSciNetCrossRefGoogle Scholar
  39. 265.
    D. Healy Jr., D. Rockmore, P. Kostelec, S. Moore, FFTs for the 2-Sphere—Improvements and Variations. J. Fourier Anal. Appl. 9(4), 341–385 (2003)MathSciNetMATHCrossRefGoogle Scholar
  40. 268.
    M. Herman, T. Strohmer, High-resolution radar via compressed sensing. IEEE Trans. Signal Process. 57(6), 2275–2284 (2009)MathSciNetCrossRefGoogle Scholar
  41. 274.
    A. Hinrichs, J. Vybiral, Johnson-Lindenstrauss lemma for circulant matrices. Random Struct. Algorithm. 39(3), 391–398 (2011)MathSciNetMATHGoogle Scholar
  42. 283.
    M. Hügel, H. Rauhut, T. Strohmer, Remote sensing via 1-minimization. Found. Comput. Math., to appear. (2012)Google Scholar
  43. 287.
    M. Iwen, Combinatorial sublinear-time Fourier algorithms. Found. Comput. Math. 10(3), 303–338 (2010)MathSciNetMATHCrossRefGoogle Scholar
  44. 288.
    M. Iwen, Improved approximation guarantees for sublinear-time Fourier algorithms. Appl. Comput. Harmon. Anal. 34(1), 57–82 (2013)MathSciNetMATHCrossRefGoogle Scholar
  45. 289.
    M. Iwen, A. Gilbert, M. Strauss, Empirical evaluation of a sub-linear time sparse DFT algorithm. Commun. Math. Sci. 5(4), 981–998 (2007)MathSciNetMATHGoogle Scholar
  46. 292.
    R. James, M. Dennis, N. Daniel, Fast discrete polynomial transforms with applications to data analysis for distance transitive graphs. SIAM J. Comput. 26(4), 1066–1099 (1997)MathSciNetCrossRefGoogle Scholar
  47. 307.
    F. Krahmer, S. Mendelson, H. Rauhut, Suprema of chaos processes and the restricted isometry property. Comm. Pure Appl. Math. (to appear)Google Scholar
  48. 308.
    F. Krahmer, G.E. Pfander, P. Rashkov, Uncertainty principles for time–frequency representations on finite abelian groups. Appl. Comput. Harmon. Anal. 25(2), 209–225 (2008)MathSciNetMATHCrossRefGoogle Scholar
  49. 309.
    F. Krahmer, R. Ward, New and improved Johnson-Lindenstrauss embeddings via the Restricted Isometry Property. SIAM J. Math. Anal. 43(3), 1269–1281 (2011)MathSciNetMATHCrossRefGoogle Scholar
  50. 310.
    F. Krahmer, R. Ward, Beyond incoherence: stable and robust sampling strategies for compressive imaging. Preprint (2012)Google Scholar
  51. 311.
    I. Krasikov, On the Erdelyi-Magnus-Nevai conjecture for Jacobi polynomials. Constr. Approx. 28(2), 113–125 (2008)MathSciNetMATHCrossRefGoogle Scholar
  52. 314.
    P. Kuppinger, G. Durisi, H. Bölcskei, Uncertainty relations and sparse signal recovery for pairs of general signal sets. IEEE Trans. Inform. Theor. 58(1), 263–277 (2012)CrossRefGoogle Scholar
  53. 317.
    J. Lawrence, G.E. Pfander, D. Walnut, Linear independence of Gabor systems in finite dimensional vector spaces. J. Fourier Anal. Appl. 11(6), 715–726 (2005)MathSciNetMATHCrossRefGoogle Scholar
  54. 341.
    S. Mallat, A Wavelet Tour of Signal Processing. Middleton Academic Press, San Diego, 1998MATHGoogle Scholar
  55. 352.
    D. Middleton, Channel Modeling and Threshold Signal Processing in Underwater Acoustics: An Analytical Overview. IEEE J. Oceanic Eng. 12(1), 4–28 (1987)MathSciNetCrossRefGoogle Scholar
  56. 384.
    G. Pfander, H. Rauhut, J. Tanner, Identification of matrices having a sparse representation. IEEE Trans. Signal Process. 56(11), 5376–5388 (2008)MathSciNetCrossRefGoogle Scholar
  57. 385.
    G. Pfander, H. Rauhut, J. Tropp, The restricted isometry property for time-frequency structured random matrices. Prob. Theor. Relat. Field. to appearGoogle Scholar
  58. 390.
    M. Pinsky, Introduction to Fourier Analysis and Wavelets. Graduate Studies in Mathematics, vol. 102 (American Mathematical Society, Providence, RI, 2009)Google Scholar
  59. 398.
    D. Potts, Fast algorithms for discrete polynomial transforms on arbitrary grids. Lin. Algebra Appl. 366, 353–370 (2003)MathSciNetMATHCrossRefGoogle Scholar
  60. 399.
    D. Potts, G. Steidl, M. Tasche, Fast algorithms for discrete polynomial transforms. Math. Comp. 67, 1577–1590 (1998)MathSciNetMATHCrossRefGoogle Scholar
  61. 400.
    D. Potts, G. Steidl, M. Tasche, Fast fourier transforms for nonequispaced data: a tutorial. In Modern Sampling Theory: Mathematics and Applications ed. by J. Benedetto, P. Ferreira, Chap. 12 (Birkhäuser, Boston, 2001), pp. 247–270Google Scholar
  62. 408.
    H. Rauhut, Random sampling of sparse trigonometric polynomials. Appl. Comput. Harmon. Anal. 22(1), 16–42 (2007)MathSciNetMATHCrossRefGoogle Scholar
  63. 409.
    H. Rauhut, On the impossibility of uniform sparse reconstruction using greedy methods. Sampl. Theor. Signal Image Process. 7(2), 197–215 (2008)MathSciNetMATHGoogle Scholar
  64. 410.
    H. Rauhut, Circulant and Toeplitz matrices in compressed sensing. In Proc. SPARS’09 (Saint-Malo, France, 2009)Google Scholar
  65. 411.
    H. Rauhut, Compressive sensing and structured random matrices. In Theoretical Foundations and Numerical Methods for Sparse Recovery, ed. by M. Fornasier. Radon Series on Computational and Applied Mathematics, vol. 9 (de Gruyter, Berlin, 2010), pp. 1–92Google Scholar
  66. 412.
    H. Rauhut, G.E. Pfander, Sparsity in time-frequency representations. J. Fourier Anal. Appl. 16(2), 233–260 (2010)MathSciNetMATHCrossRefGoogle Scholar
  67. 413.
    H. Rauhut, J.K. Romberg, J.A. Tropp, Restricted isometries for partial random circulant matrices. Appl. Comput. Harmon. Anal. 32(2), 242–254 (2012)MathSciNetMATHCrossRefGoogle Scholar
  68. 415.
    H. Rauhut, R. Ward, Sparse recovery for spherical harmonic expansions. In Proc. SampTA 2011, Singapore, 2011Google Scholar
  69. 416.
    H. Rauhut, R. Ward, Sparse Legendre expansions via 1-minimization. J. Approx. Theor. 164(5), 517–533 (2012)MathSciNetMATHCrossRefGoogle Scholar
  70. 417.
    B. Recht, A simpler approach to matrix completion. J. Mach. Learn. Res. 12, 3413–3430 (2011)MathSciNetGoogle Scholar
  71. 433.
    M. Rudelson, R. Vershynin, On sparse reconstruction from Fourier and Gaussian measurements. Comm. Pure Appl. Math. 61, 1025–1045 (2008)MathSciNetMATHCrossRefGoogle Scholar
  72. 437.
    W. Rudin, Fourier Analysis on Groups (Interscience Publishers, New York, 1962)MATHGoogle Scholar
  73. 445.
    I. Segal, M. Iwen, Improved sparse Fourier approximation results: Faster implementations and stronger guarantees. Numer. Algorithm. 63(2), 239–263 (2013)MathSciNetMATHCrossRefGoogle Scholar
  74. 452.
    E.M. Stein, R. Shakarchi, Functional Analysis: Introduction to Further Topics in Analysis. Princeton Lectures in Analysis, vol. 4 (Princeton University Press, Princeton, NJ, 2011)Google Scholar
  75. 453.
    M. Stojanovic, Underwater Acoustic Communications. In Encyclopedia of Electrical and Electronics Engineering, vol. 22, ed. by M. Stojanovic, J. G. Webster (Wiley, New York, 1999), pp. 688–698Google Scholar
  76. 458.
    G. Szegő, Orthogonal Polynomials, 4th edn. (American Mathematical Society, Providence, 1975)Google Scholar
  77. 464.
    M. Talagrand, Selecting a proportion of characters. Israel J. Math. 108, 173–191 (1998)MathSciNetMATHCrossRefGoogle Scholar
  78. 481.
    J.A. Tropp, On the conditioning of random subdictionaries. Appl. Comput. Harmon. Anal. 25, 1–24 (2008)MathSciNetMATHCrossRefGoogle Scholar
  79. 488.
    J.A. Tropp, J.N. Laska, M.F. Duarte, J.K. Romberg, R.G. Baraniuk, Beyond Nyquist: Efficient sampling of sparse bandlimited signals. IEEE Trans. Inform. Theor. 56(1), 520–544 (2010)MathSciNetCrossRefGoogle Scholar
  80. 489.
    J.A. Tropp, M. Wakin, M. Duarte, D. Baron, R. Baraniuk, Random filters for compressive sampling and reconstruction. Proc. 2006 IEEE International Conference Acoustics, Speech, and Signal Processing, vol. 3, pp. 872–875, 2006Google Scholar
  81. 491.
    M. Tygert, Fast algorithms for spherical harmonic expansions, II. J. Comput. Phys. 227(8), 4260–4279 (2008)MathSciNetMATHCrossRefGoogle Scholar
  82. 502.
    J. Vybiral, A variant of the Johnson-Lindenstrauss lemma for circulant matrices. J. Funct. Anal. 260(4), 1096–1105 (2011)MathSciNetMATHCrossRefGoogle Scholar
  83. 505.
    J. Walker, Fourier analysis and wavelet analysis. Notices Am. Math. Soc. 44(6), 658–670 (1997)MATHGoogle Scholar
  84. 508.
    P. Wojtaszczyk, A Mathematical Introduction to Wavelets (Cambridge University Press, Cambridge, 1997)MATHCrossRefGoogle Scholar
  85. 519.
    J. Zou, A.C. Gilbert, M. Strauss, I. Daubechies, Theoretical and experimental analysis of a randomized algorithm for sparse Fourier transform analysis. J. Comput. Phys. 211, 572–595 (2005)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Simon Foucart
    • 1
  • Holger Rauhut
    • 2
  1. 1.Department of MathematicsDrexel UniversityPhiladelphiaUSA
  2. 2.Lehrstuhl C für Mathematik (Analysis)RWTH Aachen UniversityAachenGermany

Personalised recommendations