Random Sampling in Bounded Orthonormal Systems

  • Simon Foucart
  • Holger Rauhut
Part of the Applied and Numerical Harmonic Analysis book series (ANHA)


This chapter considers the recovery of signals with a sparse expansion in a bounded orthonormal system. After an inventory of such bounded orthonormal systems including the Fourier systems, theoretical limitations specific to this situation are obtained for the minimal number of samples. Then, using this number of random samples, nonuniform sparse recovery is proved to be possible via ℓ1-minimization. Next, using a slightly higher number of random samples, uniform sparse recovery is also proved to be possible via a variety of algorithms. This is derived via establishing the restricted isometry property for the associated random sampling matrix—the random partial Fourier matrix is a particular case. Finally, a connection to the Λ 1-problem is pointed out.


sampling matrix random matrices bounded orthonormal systems discrete bounded orthonormal system partial Fourier matrix partial Hadamard matrix uncertainty principles nonuniform recovery 1-minimization golfing scheme restricted isometry property Λ1-problem 


  1. 1.
    P. Abrial, Y. Moudden, J.-L. Starck, J. Fadili, J. Delabrouille, M. Nguyen, CMB data analysis and sparsity. Stat. Meth. 5, 289–298 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 7.
    N. Ailon, B. Chazelle, The fast Johnson-Lindenstrauss transform and approximate nearest neighbors. SIAM J. Comput. 39(1), 302–322 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 8.
    N. Ailon, E. Liberty, Fast dimension reduction using Rademacher series on dual BCH codes. Discrete Comput. Geom. 42(4), 615–630 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 9.
    N. Ailon, E. Liberty, Almost optimal unrestricted fast Johnson-Lindenstrauss transform. In Proceedings of the 22nd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), San Francisco, USA, 22–25 January 2011Google Scholar
  5. 10.
    N. Ailon, H. Rauhut, Fast and RIP-optimal transforms. Preprint (2013)Google Scholar
  6. 17.
    G. Andrews, R. Askey, R. Roy, in Special Functions. Encyclopedia of Mathematics and its Applications, vol. 71 (Cambridge University Press, Cambridge, 1999)Google Scholar
  7. 26.
    W. Bajwa, J. Haupt, G. Raz, S. Wright, R. Nowak, Toeplitz-structured compressed sensing matrices. In Proc. IEEE Stat. Sig. Proc. Workshop, pp. 294–298, 2007Google Scholar
  8. 62.
    J. Bourgain, Bounded orthogonal systems and the Λ(p)-set problem. Acta Math. 162(3–4), 227–245 (1989)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 63.
    J. Bourgain, \(\Lambda _{p}\)-sets in analysis: results, problems and related aspects. In Handbook of the Geometry of Banach Spaces, vol. 1, ed. by W. B. Johnson, J. Lindenstrauss (North-Holland, Amsterdam, 2001), pp. 195–232Google Scholar
  10. 80.
    N. Burq, S. Dyatlov, R. Ward, M. Zworski, Weighted eigenfunction estimates with applications to compressed sensing. SIAM J. Math. Anal. 44(5), 3481–3501 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 88.
    E.J. Candès, Y. Plan, A probabilistic and RIPless theory of compressed sensing. IEEE Trans. Inform. Theor. 57(11), 7235–7254 (2011)CrossRefGoogle Scholar
  12. 92.
    E.J. Candès, J. Romberg, Quantitative robust uncertainty principles and optimally sparse decompositions. Found. Comput. Math. 6(2), 227–254 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 93.
    E.J. Candès, J. Romberg, Sparsity and incoherence in compressive sampling. Inverse Probl. 23(3), 969–985 (2007)zbMATHCrossRefGoogle Scholar
  14. 94.
    E.J. Candès, J. Romberg, T. Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theor. 52(2), 489–509 (2006)zbMATHCrossRefGoogle Scholar
  15. 97.
    E.J. Candès, T. Tao, Near optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inform. Theor. 52(12), 5406–5425 (2006)CrossRefGoogle Scholar
  16. 103.
    D. Chafaï, O. Guédon, G. Lecué, A. Pajor, Interactions Between Compressed Sensing, Random Matrices and High-dimensional Geometry, Panoramas et Synthèses, vol. 37 (Société Mathématique de France, to appear)Google Scholar
  17. 115.
    M. Cheraghchi, V. Guruswami, A. Velingker, Restricted isometry of Fourier matrices and list decodability of random linear codes. Preprint (2012)Google Scholar
  18. 117.
    T. Chihara, An Introduction to Orthogonal Polynomials (Gordon and Breach Science Publishers, New York 1978)zbMATHGoogle Scholar
  19. 119.
    O. Christensen, An Introduction to Frames and Riesz Bases. Applied and Numerical Harmonic Analysis (Birkhäuser, Boston, 2003)zbMATHGoogle Scholar
  20. 122.
    A. Cohen, Numerical Analysis of Wavelet Methods (North-Holland, Amsterdam, 2003)zbMATHGoogle Scholar
  21. 126.
    R. Coifman, F. Geshwind, Y. Meyer, Noiselets. Appl. Comput. Harmon. Anal. 10(1), 27–44 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 137.
    I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 61 (SIAM, Philadelphia, 1992)Google Scholar
  23. 158.
    D.L. Donoho, X. Huo, Uncertainty principles and ideal atomic decompositions. IEEE Trans. Inform. Theor. 47(7), 2845–2862 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 164.
    D.L. Donoho, P. Stark, Uncertainty principles and signal recovery. SIAM J. Appl. Math. 48(3), 906–931 (1989)MathSciNetCrossRefGoogle Scholar
  25. 181.
    M. Elad, A.M. Bruckstein, A generalized uncertainty principle and sparse representation in pairs of bases. IEEE Trans. Inform. Theor. 48(9), 2558–2567 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 191.
    H.G. Feichtinger, F. Luef, T. Werther, A guided tour from linear algebra to the foundations of Gabor analysis. In Gabor and Wavelet Frames. Lecture Notes Series, Institute for Mathematical Sciences National University of Singapore, vol. 10 (World Sci. Publ., Hackensack, 2007), pp. 1–49Google Scholar
  27. 192.
    H.G. Feichtinger, T. Strohmer, Gabor Analysis and Algorithms: Theory and Applications (Birkhäuser, Boston, 1998)zbMATHCrossRefGoogle Scholar
  28. 198.
    G.B. Folland, Fourier Analysis and Its Applications (Wadsworth and Brooks, Pacific Grove, 1992)zbMATHGoogle Scholar
  29. 199.
    G.B. Folland, A Course in Abstract Harmonic Analysis (CRC Press, Boca Raton, 1995)zbMATHGoogle Scholar
  30. 200.
    G.B. Folland, A. Sitaram, The uncertainty principle: A mathematical survey. J. Fourier Anal. Appl. 3(3), 207–238 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 223.
    A.C. Gilbert, S. Muthukrishnan, S. Guha, P. Indyk, M. Strauss, Near-Optimal Sparse Fourier Representations via Sampling. In Proceedings of the Thiry-fourth Annual ACM Symposium on Theory of Computing, STOC ’02, pp. 152–161, ACM, New York, NY, USA, 2002Google Scholar
  32. 231.
    G. Golub, C.F. van Loan, Matrix Computations, 3rd edn. (The Johns Hopkins University Press, Baltimore, MD, 1996)zbMATHGoogle Scholar
  33. 236.
    L. Grafakos, Modern Fourier Analysis, 2nd edn. Graduate Texts in Mathematics, vol. 250 (Springer, New York, 2009)Google Scholar
  34. 244.
    K. Gröchenig, Foundations of Time-Frequency Analysis. Applied and Numerical Harmonic Analysis (Birkhäuser, Boston, MA, 2001)Google Scholar
  35. 245.
    D. Gross, Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inform. Theor. 57(3), 1548–1566 (2011)CrossRefGoogle Scholar
  36. 248.
    O. Guédon, S. Mendelson, A. Pajor, N. Tomczak-Jaegermann, Majorizing measures and proportional subsets of bounded orthonormal systems. Rev. Mat. Iberoam. 24(3), 1075–1095 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  37. 261.
    H. Hassanieh, P. Indyk, D. Katabi, E. Price, Nearly optimal sparse Fourier transform. In Proceedings of the 44th Symposium on Theory of Computing, STOC ’12, pp. 563–578, ACM, New York, NY, USA, 2012Google Scholar
  38. 263.
    J. Haupt, W. Bajwa, G. Raz, R. Nowak, Toeplitz compressed sensing matrices with applications to sparse channel estimation. IEEE Trans. Inform. Theor. 56(11), 5862–5875 (2010)MathSciNetCrossRefGoogle Scholar
  39. 265.
    D. Healy Jr., D. Rockmore, P. Kostelec, S. Moore, FFTs for the 2-Sphere—Improvements and Variations. J. Fourier Anal. Appl. 9(4), 341–385 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  40. 268.
    M. Herman, T. Strohmer, High-resolution radar via compressed sensing. IEEE Trans. Signal Process. 57(6), 2275–2284 (2009)MathSciNetCrossRefGoogle Scholar
  41. 274.
    A. Hinrichs, J. Vybiral, Johnson-Lindenstrauss lemma for circulant matrices. Random Struct. Algorithm. 39(3), 391–398 (2011)MathSciNetzbMATHGoogle Scholar
  42. 283.
    M. Hügel, H. Rauhut, T. Strohmer, Remote sensing via 1-minimization. Found. Comput. Math., to appear. (2012)Google Scholar
  43. 287.
    M. Iwen, Combinatorial sublinear-time Fourier algorithms. Found. Comput. Math. 10(3), 303–338 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  44. 288.
    M. Iwen, Improved approximation guarantees for sublinear-time Fourier algorithms. Appl. Comput. Harmon. Anal. 34(1), 57–82 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  45. 289.
    M. Iwen, A. Gilbert, M. Strauss, Empirical evaluation of a sub-linear time sparse DFT algorithm. Commun. Math. Sci. 5(4), 981–998 (2007)MathSciNetzbMATHGoogle Scholar
  46. 292.
    R. James, M. Dennis, N. Daniel, Fast discrete polynomial transforms with applications to data analysis for distance transitive graphs. SIAM J. Comput. 26(4), 1066–1099 (1997)MathSciNetCrossRefGoogle Scholar
  47. 307.
    F. Krahmer, S. Mendelson, H. Rauhut, Suprema of chaos processes and the restricted isometry property. Comm. Pure Appl. Math. (to appear)Google Scholar
  48. 308.
    F. Krahmer, G.E. Pfander, P. Rashkov, Uncertainty principles for time–frequency representations on finite abelian groups. Appl. Comput. Harmon. Anal. 25(2), 209–225 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  49. 309.
    F. Krahmer, R. Ward, New and improved Johnson-Lindenstrauss embeddings via the Restricted Isometry Property. SIAM J. Math. Anal. 43(3), 1269–1281 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  50. 310.
    F. Krahmer, R. Ward, Beyond incoherence: stable and robust sampling strategies for compressive imaging. Preprint (2012)Google Scholar
  51. 311.
    I. Krasikov, On the Erdelyi-Magnus-Nevai conjecture for Jacobi polynomials. Constr. Approx. 28(2), 113–125 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  52. 314.
    P. Kuppinger, G. Durisi, H. Bölcskei, Uncertainty relations and sparse signal recovery for pairs of general signal sets. IEEE Trans. Inform. Theor. 58(1), 263–277 (2012)CrossRefGoogle Scholar
  53. 317.
    J. Lawrence, G.E. Pfander, D. Walnut, Linear independence of Gabor systems in finite dimensional vector spaces. J. Fourier Anal. Appl. 11(6), 715–726 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  54. 341.
    S. Mallat, A Wavelet Tour of Signal Processing. Middleton Academic Press, San Diego, 1998zbMATHGoogle Scholar
  55. 352.
    D. Middleton, Channel Modeling and Threshold Signal Processing in Underwater Acoustics: An Analytical Overview. IEEE J. Oceanic Eng. 12(1), 4–28 (1987)MathSciNetCrossRefGoogle Scholar
  56. 384.
    G. Pfander, H. Rauhut, J. Tanner, Identification of matrices having a sparse representation. IEEE Trans. Signal Process. 56(11), 5376–5388 (2008)MathSciNetCrossRefGoogle Scholar
  57. 385.
    G. Pfander, H. Rauhut, J. Tropp, The restricted isometry property for time-frequency structured random matrices. Prob. Theor. Relat. Field. to appearGoogle Scholar
  58. 390.
    M. Pinsky, Introduction to Fourier Analysis and Wavelets. Graduate Studies in Mathematics, vol. 102 (American Mathematical Society, Providence, RI, 2009)Google Scholar
  59. 398.
    D. Potts, Fast algorithms for discrete polynomial transforms on arbitrary grids. Lin. Algebra Appl. 366, 353–370 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  60. 399.
    D. Potts, G. Steidl, M. Tasche, Fast algorithms for discrete polynomial transforms. Math. Comp. 67, 1577–1590 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  61. 400.
    D. Potts, G. Steidl, M. Tasche, Fast fourier transforms for nonequispaced data: a tutorial. In Modern Sampling Theory: Mathematics and Applications ed. by J. Benedetto, P. Ferreira, Chap. 12 (Birkhäuser, Boston, 2001), pp. 247–270Google Scholar
  62. 408.
    H. Rauhut, Random sampling of sparse trigonometric polynomials. Appl. Comput. Harmon. Anal. 22(1), 16–42 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  63. 409.
    H. Rauhut, On the impossibility of uniform sparse reconstruction using greedy methods. Sampl. Theor. Signal Image Process. 7(2), 197–215 (2008)MathSciNetzbMATHGoogle Scholar
  64. 410.
    H. Rauhut, Circulant and Toeplitz matrices in compressed sensing. In Proc. SPARS’09 (Saint-Malo, France, 2009)Google Scholar
  65. 411.
    H. Rauhut, Compressive sensing and structured random matrices. In Theoretical Foundations and Numerical Methods for Sparse Recovery, ed. by M. Fornasier. Radon Series on Computational and Applied Mathematics, vol. 9 (de Gruyter, Berlin, 2010), pp. 1–92Google Scholar
  66. 412.
    H. Rauhut, G.E. Pfander, Sparsity in time-frequency representations. J. Fourier Anal. Appl. 16(2), 233–260 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  67. 413.
    H. Rauhut, J.K. Romberg, J.A. Tropp, Restricted isometries for partial random circulant matrices. Appl. Comput. Harmon. Anal. 32(2), 242–254 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  68. 415.
    H. Rauhut, R. Ward, Sparse recovery for spherical harmonic expansions. In Proc. SampTA 2011, Singapore, 2011Google Scholar
  69. 416.
    H. Rauhut, R. Ward, Sparse Legendre expansions via 1-minimization. J. Approx. Theor. 164(5), 517–533 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  70. 417.
    B. Recht, A simpler approach to matrix completion. J. Mach. Learn. Res. 12, 3413–3430 (2011)MathSciNetGoogle Scholar
  71. 433.
    M. Rudelson, R. Vershynin, On sparse reconstruction from Fourier and Gaussian measurements. Comm. Pure Appl. Math. 61, 1025–1045 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  72. 437.
    W. Rudin, Fourier Analysis on Groups (Interscience Publishers, New York, 1962)zbMATHGoogle Scholar
  73. 445.
    I. Segal, M. Iwen, Improved sparse Fourier approximation results: Faster implementations and stronger guarantees. Numer. Algorithm. 63(2), 239–263 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  74. 452.
    E.M. Stein, R. Shakarchi, Functional Analysis: Introduction to Further Topics in Analysis. Princeton Lectures in Analysis, vol. 4 (Princeton University Press, Princeton, NJ, 2011)Google Scholar
  75. 453.
    M. Stojanovic, Underwater Acoustic Communications. In Encyclopedia of Electrical and Electronics Engineering, vol. 22, ed. by M. Stojanovic, J. G. Webster (Wiley, New York, 1999), pp. 688–698Google Scholar
  76. 458.
    G. Szegő, Orthogonal Polynomials, 4th edn. (American Mathematical Society, Providence, 1975)Google Scholar
  77. 464.
    M. Talagrand, Selecting a proportion of characters. Israel J. Math. 108, 173–191 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  78. 481.
    J.A. Tropp, On the conditioning of random subdictionaries. Appl. Comput. Harmon. Anal. 25, 1–24 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  79. 488.
    J.A. Tropp, J.N. Laska, M.F. Duarte, J.K. Romberg, R.G. Baraniuk, Beyond Nyquist: Efficient sampling of sparse bandlimited signals. IEEE Trans. Inform. Theor. 56(1), 520–544 (2010)MathSciNetCrossRefGoogle Scholar
  80. 489.
    J.A. Tropp, M. Wakin, M. Duarte, D. Baron, R. Baraniuk, Random filters for compressive sampling and reconstruction. Proc. 2006 IEEE International Conference Acoustics, Speech, and Signal Processing, vol. 3, pp. 872–875, 2006Google Scholar
  81. 491.
    M. Tygert, Fast algorithms for spherical harmonic expansions, II. J. Comput. Phys. 227(8), 4260–4279 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  82. 502.
    J. Vybiral, A variant of the Johnson-Lindenstrauss lemma for circulant matrices. J. Funct. Anal. 260(4), 1096–1105 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  83. 505.
    J. Walker, Fourier analysis and wavelet analysis. Notices Am. Math. Soc. 44(6), 658–670 (1997)zbMATHGoogle Scholar
  84. 508.
    P. Wojtaszczyk, A Mathematical Introduction to Wavelets (Cambridge University Press, Cambridge, 1997)zbMATHCrossRefGoogle Scholar
  85. 519.
    J. Zou, A.C. Gilbert, M. Strauss, I. Daubechies, Theoretical and experimental analysis of a randomized algorithm for sparse Fourier transform analysis. J. Comput. Phys. 211, 572–595 (2005)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Simon Foucart
    • 1
  • Holger Rauhut
    • 2
  1. 1.Department of MathematicsDrexel UniversityPhiladelphiaUSA
  2. 2.Lehrstuhl C für Mathematik (Analysis)RWTH Aachen UniversityAachenGermany

Personalised recommendations