Journal of Fourier Analysis and Applications

, Volume 24, Issue 4, pp 935–956 | Cite as

Discrete Uncertainty Principles and Sparse Signal Processing

  • Afonso S. Bandeira
  • Megan E. Lewis
  • Dustin G. MixonEmail author


We develop new discrete uncertainty principles in terms of numerical sparsity, which is a continuous proxy for the 0-norm. Unlike traditional sparsity, the continuity of numerical sparsity naturally accommodates functions which are nearly sparse. After studying these principles and the functions that achieve exact or near equality in them, we identify certain consequences in a number of sparse signal processing applications.


Uncertainty principle Sparsity Compressed sensing 



The authors thank Laurent Duval, Joel Tropp, and the anonymous referees for multiple suggestions that significantly improved the presentation of our results and our discussion of the relevant literature. ASB was supported by AFOSR Grant No. FA9550-12-1-0317. DGM was supported by an AFOSR Young Investigator Research Program award, NSF Grant No. DMS-1321779, and AFOSR Grant No. F4FGA05076J002. The views expressed in this article are those of the authors and do not reflect the official policy or position of the United States Air Force, Department of Defense, or the U.S. Government.


  1. 1.
    Alon, N.: Problems and results in extremal combinatorics. Discret. Math. 273, 31–53 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Amrein, W.O., Berthier, A.M.: On support properties of \(L^p\)-functions and their Fourier transforms. J. Funct. Anal. 24, 258–267 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Bandeira, A.S., Fickus, M., Mixon, D.G., Moreira, J.: Derandomizing restricted isometries via the Legendre symbol. arXiv:1406.4089
  4. 4.
    Bandeira, A.S., Fickus, M., Mixon, D.G., Wong, P.: The road to deterministic matrices with the restricted isometry property. J. Fourier Anal. Appl. 19, 1123–1149 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28, 253–263 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Beckner, W.: Inequalities in Fourier analysis. Ann. Math. 102, 159–182 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Blanchard, J.D., Cartis, C., Tanner, J.: Compressed sensing: How sharp is the restricted isometry property? SIAM Rev. 53, 105–125 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Bourgain, J.: An improved estimate in the restricted isometry problem. Lect. Notes Math. 2116, 65–70 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Cahill, J., Mixon, D.G.: Robust width: A characterization of uniformly stable and robust compressed sensing. arXiv:1408.4409
  10. 10.
    Cai, T.T., Zhang, A.: Sharp RIP bound for sparse signal and low-rank matrix recovery. Appl. Comput. Harmon. Anal. 35, 74–93 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Candès, E.J.: The restricted isometry property and its implications for compressed sensing. C. R. Acad. Sci. Paris Ser. I 346, 589–592 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Candès, E.J., Tao, T.: Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52, 5406–5425 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Csörgő, S.: A rate of convergence for coupon collectors. Acta Sci. Math. (Szeged) 57, 337–351 (1993)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Demanet, L., Hand, P.: Scaling law for recovering the sparsest element in a subspace. arXiv:1310.1654
  15. 15.
    Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via \(\ell ^1\) minimization. Proc. Natl. Acad. Sci. USA 100, 2197–2202 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Donoho, D.L., Huo, X.: Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 47, 2845–2862 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Donoho, D.L., Stark, P.B.: Uncertainty principles and signal recovery. SIAM J. Appl. Math. 49, 906–931 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Erdős, P., Rényi, A.: On a classical problem of probability theory. Magy. Tud Akad. Mat. Kutató Int. Közl. 6, 215–220 (1961)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Foucart, A., Rauhut, H.: A Mathematical Introduction to Compressive Sensing. Applied and Numerical Harmonic Analysis. Birkhäuser, Basel (2013)zbMATHGoogle Scholar
  20. 20.
    Gray, W.C.: Variable norm deconvolution, Tech. Rep. SEP-14, Stanford University (1978)Google Scholar
  21. 21.
    Gribonval, R., Nielsen, M.: Highly sparse representations from dictionaries are unique and independent of the sparseness measure. Appl. Comput. Harmon. Anal. 22, 335–355 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Guédon, O., Mendelson, S., Pajor, A., Tomczak-Jaegermann, N.: Majorizing measures and proportional subsets of bounded orthonormal systems. Rev. Mat. Iberoam. 24, 1075–1095 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Hardy, G.H.: A theorem concerning Fourier transforms. J. Lond. Math. Soc. 8, 227–231 (1933)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Heisenberg, W.: Über den anschaulichen Inhalt der quantentheoretischen Kinematik und Mechanik. Z. Phys. (in German) 43, 172–198 (1927)CrossRefzbMATHGoogle Scholar
  25. 25.
    Hurley, N., Rickard, S.: Comparing measures of sparsity. IEEE Trans. Inf. Theory 55, 4723–4741 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Indyk P, Kapralov M.: Sample-optimal Fourier sampling in any constant dimension. In: foundations of computer science (FOCS), 2014 IEEE 55th Annual Symposium on 2014, pp. 514–523Google Scholar
  27. 27.
    Kashin, B.S., Temlyakov, V.N.: A remark on compressed sensing. Math. Notes 82, 748–755 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Krahmer, F., Mendelson, S., Rauhut, H.: Suprema of chaos processes and the restricted isometry property. Commun. Pure Appl. Math. 67, 1877–1904 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Kraus, K.: Complementary observables and uncertainty relations. Phys. Rev. D 35, 3070–3075 (1987)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Laurent, B., Massart, P.: Adaptive estimation of a quadratic functional by model selection. Ann. Stat. 28, 1302–1338 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Lopes, M.E.: Estimating unknown sparsity in compressed sensing. JMLR W&CP 28, 217–225 (2013)Google Scholar
  32. 32.
    McCoy, M.B., Cevher, V., Dinh, Q.T., Asaei, A., Baldassarre, L.: Convexity in source separation: models, geometry, and algorithms. Signal Proc. Mag. 31, 87–95 (2014)CrossRefGoogle Scholar
  33. 33.
    McCoy, M.B., Tropp, J.A.: Sharp recovery bounds for convex demixing, with applications. Found. Comput. Math. 14, 503–567 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Nelson, J., Price, E., Wootters, M.: New constructions of RIP matrices with fast multiplication and fewer rows. In: SODA, pp. 1515–1528 (2014)Google Scholar
  35. 35.
    Rauhut, H.: Compressive sensing and structured random matrices. Theor. Found. Numer. Methods Sparse Recover. 9, 1–92 (2010)MathSciNetzbMATHGoogle Scholar
  36. 36.
    Repetti, A., Pham, M.Q., Duval, L., Chouzenoux, É., Pesquet, J.-C.: Euclid in a taxicab: sparse blind deconvolution with smoothed \(\ell _1/\ell _2\) regularization. IEEE Signal Process. Lett. 22, 539–543 (2014)CrossRefGoogle Scholar
  37. 37.
    Rudelson, M., Vershynin, R.: On sparse reconstruction from Fourier and Gaussian measurements. Comm. Pure Appl. Math. 61, 1025–1045 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Santosa, F., Symes, W.W.: Linear inversion of band-limited reflection seismograms. SIAM J. Sci. Stat. Comput. 7, 1307–1330 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Stevenhagen, P., Lenstra, H.W.: Chebotarëv and his density theorem. Math. Intell. 18, 26–37 (1996)CrossRefzbMATHGoogle Scholar
  40. 40.
    Studer, C.: Recovery of signals with low density. arXiv:1507.02821
  41. 41.
    Talagrand, M.: Selecting a proportion of characters. Israel J. Math. 108, 173–191 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Tao, T.: An uncertainty principle for cyclic groups of prime order. Math. Res. Lett. 12, 121–127 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Terras, A.: Fourier Analysis on Finite Groups and Applications. Cambridge University Press, Cambridge (1999)CrossRefzbMATHGoogle Scholar
  44. 44.
    Tropp, J.A.: The sparsity gap: uncertainty principles proportional to dimension. In: CISS, pp. 1–6 (2010)Google Scholar
  45. 45.
    Tropp, J.A.: On the linear independence of spikes and sines. J. Fourier Anal. Appl. 14, 838–858 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Tropp, J.A.: On the conditioning of random subdictionaries. Appl. Comput. Harmon. Anal. 25, 1–24 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Vershynin, R.: Introduction to the non-asymptotic analysis of random matrices. Theory and Applications, Cambridge University Press, Compressed Sensing (2012)Google Scholar

Copyright information

© Springer Science+Business Media, Inc. (outside the US) 2017

Authors and Affiliations

  • Afonso S. Bandeira
    • 1
  • Megan E. Lewis
    • 2
  • Dustin G. Mixon
    • 3
    Email author
  1. 1.Department of Mathematics, Courant Institute of Mathematical SciencesNew York UniversityNew YorkUSA
  2. 2.Detachment 5, Air Force Operational Test and Evaluation CenterEdwards AFBUSA
  3. 3.Department of Mathematics and StatisticsAir Force Institute of TechnologyWright-Patterson AFBUSA

Personalised recommendations