Discrete Uncertainty Principles and Sparse Signal Processing

  • Afonso S. Bandeira
  • Megan E. Lewis
  • Dustin G. Mixon


We develop new discrete uncertainty principles in terms of numerical sparsity, which is a continuous proxy for the 0-norm. Unlike traditional sparsity, the continuity of numerical sparsity naturally accommodates functions which are nearly sparse. After studying these principles and the functions that achieve exact or near equality in them, we identify certain consequences in a number of sparse signal processing applications.


Uncertainty principle Sparsity Compressed sensing 



The authors thank Laurent Duval, Joel Tropp, and the anonymous referees for multiple suggestions that significantly improved the presentation of our results and our discussion of the relevant literature. ASB was supported by AFOSR Grant No. FA9550-12-1-0317. DGM was supported by an AFOSR Young Investigator Research Program award, NSF Grant No. DMS-1321779, and AFOSR Grant No. F4FGA05076J002. The views expressed in this article are those of the authors and do not reflect the official policy or position of the United States Air Force, Department of Defense, or the U.S. Government.


  1. 1.
    Alon, N.: Problems and results in extremal combinatorics. Discret. Math. 273, 31–53 (2003)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Amrein, W.O., Berthier, A.M.: On support properties of \(L^p\)-functions and their Fourier transforms. J. Funct. Anal. 24, 258–267 (1977)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Bandeira, A.S., Fickus, M., Mixon, D.G., Moreira, J.: Derandomizing restricted isometries via the Legendre symbol. arXiv:1406.4089
  4. 4.
    Bandeira, A.S., Fickus, M., Mixon, D.G., Wong, P.: The road to deterministic matrices with the restricted isometry property. J. Fourier Anal. Appl. 19, 1123–1149 (2013)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28, 253–263 (2008)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Beckner, W.: Inequalities in Fourier analysis. Ann. Math. 102, 159–182 (1975)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Blanchard, J.D., Cartis, C., Tanner, J.: Compressed sensing: How sharp is the restricted isometry property? SIAM Rev. 53, 105–125 (2011)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Bourgain, J.: An improved estimate in the restricted isometry problem. Lect. Notes Math. 2116, 65–70 (2014)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Cahill, J., Mixon, D.G.: Robust width: A characterization of uniformly stable and robust compressed sensing. arXiv:1408.4409
  10. 10.
    Cai, T.T., Zhang, A.: Sharp RIP bound for sparse signal and low-rank matrix recovery. Appl. Comput. Harmon. Anal. 35, 74–93 (2013)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Candès, E.J.: The restricted isometry property and its implications for compressed sensing. C. R. Acad. Sci. Paris Ser. I 346, 589–592 (2008)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Candès, E.J., Tao, T.: Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52, 5406–5425 (2006)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Csörgő, S.: A rate of convergence for coupon collectors. Acta Sci. Math. (Szeged) 57, 337–351 (1993)MathSciNetMATHGoogle Scholar
  14. 14.
    Demanet, L., Hand, P.: Scaling law for recovering the sparsest element in a subspace. arXiv:1310.1654
  15. 15.
    Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via \(\ell ^1\) minimization. Proc. Natl. Acad. Sci. USA 100, 2197–2202 (2003)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Donoho, D.L., Huo, X.: Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 47, 2845–2862 (2001)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Donoho, D.L., Stark, P.B.: Uncertainty principles and signal recovery. SIAM J. Appl. Math. 49, 906–931 (1989)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Erdős, P., Rényi, A.: On a classical problem of probability theory. Magy. Tud Akad. Mat. Kutató Int. Közl. 6, 215–220 (1961)MathSciNetMATHGoogle Scholar
  19. 19.
    Foucart, A., Rauhut, H.: A Mathematical Introduction to Compressive Sensing. Applied and Numerical Harmonic Analysis. Birkhäuser, Basel (2013)MATHGoogle Scholar
  20. 20.
    Gray, W.C.: Variable norm deconvolution, Tech. Rep. SEP-14, Stanford University (1978)Google Scholar
  21. 21.
    Gribonval, R., Nielsen, M.: Highly sparse representations from dictionaries are unique and independent of the sparseness measure. Appl. Comput. Harmon. Anal. 22, 335–355 (2007)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Guédon, O., Mendelson, S., Pajor, A., Tomczak-Jaegermann, N.: Majorizing measures and proportional subsets of bounded orthonormal systems. Rev. Mat. Iberoam. 24, 1075–1095 (2008)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Hardy, G.H.: A theorem concerning Fourier transforms. J. Lond. Math. Soc. 8, 227–231 (1933)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Heisenberg, W.: Über den anschaulichen Inhalt der quantentheoretischen Kinematik und Mechanik. Z. Phys. (in German) 43, 172–198 (1927)CrossRefMATHGoogle Scholar
  25. 25.
    Hurley, N., Rickard, S.: Comparing measures of sparsity. IEEE Trans. Inf. Theory 55, 4723–4741 (2009)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Indyk P, Kapralov M.: Sample-optimal Fourier sampling in any constant dimension. In: foundations of computer science (FOCS), 2014 IEEE 55th Annual Symposium on 2014, pp. 514–523Google Scholar
  27. 27.
    Kashin, B.S., Temlyakov, V.N.: A remark on compressed sensing. Math. Notes 82, 748–755 (2007)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Krahmer, F., Mendelson, S., Rauhut, H.: Suprema of chaos processes and the restricted isometry property. Commun. Pure Appl. Math. 67, 1877–1904 (2014)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Kraus, K.: Complementary observables and uncertainty relations. Phys. Rev. D 35, 3070–3075 (1987)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Laurent, B., Massart, P.: Adaptive estimation of a quadratic functional by model selection. Ann. Stat. 28, 1302–1338 (2000)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Lopes, M.E.: Estimating unknown sparsity in compressed sensing. JMLR W&CP 28, 217–225 (2013)Google Scholar
  32. 32.
    McCoy, M.B., Cevher, V., Dinh, Q.T., Asaei, A., Baldassarre, L.: Convexity in source separation: models, geometry, and algorithms. Signal Proc. Mag. 31, 87–95 (2014)CrossRefGoogle Scholar
  33. 33.
    McCoy, M.B., Tropp, J.A.: Sharp recovery bounds for convex demixing, with applications. Found. Comput. Math. 14, 503–567 (2014)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Nelson, J., Price, E., Wootters, M.: New constructions of RIP matrices with fast multiplication and fewer rows. In: SODA, pp. 1515–1528 (2014)Google Scholar
  35. 35.
    Rauhut, H.: Compressive sensing and structured random matrices. Theor. Found. Numer. Methods Sparse Recover. 9, 1–92 (2010)MathSciNetMATHGoogle Scholar
  36. 36.
    Repetti, A., Pham, M.Q., Duval, L., Chouzenoux, É., Pesquet, J.-C.: Euclid in a taxicab: sparse blind deconvolution with smoothed \(\ell _1/\ell _2\) regularization. IEEE Signal Process. Lett. 22, 539–543 (2014)CrossRefGoogle Scholar
  37. 37.
    Rudelson, M., Vershynin, R.: On sparse reconstruction from Fourier and Gaussian measurements. Comm. Pure Appl. Math. 61, 1025–1045 (2008)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Santosa, F., Symes, W.W.: Linear inversion of band-limited reflection seismograms. SIAM J. Sci. Stat. Comput. 7, 1307–1330 (1986)MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    Stevenhagen, P., Lenstra, H.W.: Chebotarëv and his density theorem. Math. Intell. 18, 26–37 (1996)CrossRefMATHGoogle Scholar
  40. 40.
    Studer, C.: Recovery of signals with low density. arXiv:1507.02821
  41. 41.
    Talagrand, M.: Selecting a proportion of characters. Israel J. Math. 108, 173–191 (1998)MathSciNetCrossRefMATHGoogle Scholar
  42. 42.
    Tao, T.: An uncertainty principle for cyclic groups of prime order. Math. Res. Lett. 12, 121–127 (2005)MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    Terras, A.: Fourier Analysis on Finite Groups and Applications. Cambridge University Press, Cambridge (1999)CrossRefMATHGoogle Scholar
  44. 44.
    Tropp, J.A.: The sparsity gap: uncertainty principles proportional to dimension. In: CISS, pp. 1–6 (2010)Google Scholar
  45. 45.
    Tropp, J.A.: On the linear independence of spikes and sines. J. Fourier Anal. Appl. 14, 838–858 (2008)MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Tropp, J.A.: On the conditioning of random subdictionaries. Appl. Comput. Harmon. Anal. 25, 1–24 (2008)MathSciNetCrossRefMATHGoogle Scholar
  47. 47.
    Vershynin, R.: Introduction to the non-asymptotic analysis of random matrices. Theory and Applications, Cambridge University Press, Compressed Sensing (2012)Google Scholar

Copyright information

© Springer Science+Business Media, Inc. (outside the US) 2017

Authors and Affiliations

  • Afonso S. Bandeira
    • 1
  • Megan E. Lewis
    • 2
  • Dustin G. Mixon
    • 3
  1. 1.Department of Mathematics, Courant Institute of Mathematical SciencesNew York UniversityNew YorkUSA
  2. 2.Detachment 5, Air Force Operational Test and Evaluation CenterEdwards AFBUSA
  3. 3.Department of Mathematics and StatisticsAir Force Institute of TechnologyWright-Patterson AFBUSA

Personalised recommendations