Discrete Uncertainty Principles and Sparse Signal Processing

Abstract

We develop new discrete uncertainty principles in terms of numerical sparsity, which is a continuous proxy for the 0-norm. Unlike traditional sparsity, the continuity of numerical sparsity naturally accommodates functions which are nearly sparse. After studying these principles and the functions that achieve exact or near equality in them, we identify certain consequences in a number of sparse signal processing applications.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

Notes

  1. 1.

    Recall that \(f(n)=O(g(n))\) if there exists \(C,n_0>0\) such that \(f(n)\le Cg(n)\) for all \(n>n_0\). We write \(f(n)=O_\delta (g(n))\) if the constant C is a function of \(\delta \). Also, \(f(n)=\Omega (g(n))\) if \(g(n)=O(f(n))\), and \(f(n)=o(g(n))\) if \(f(n)/g(n)\rightarrow 0\) as \(n\rightarrow \infty \).

References

  1. 1.

    Alon, N.: Problems and results in extremal combinatorics. Discret. Math. 273, 31–53 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  2. 2.

    Amrein, W.O., Berthier, A.M.: On support properties of \(L^p\)-functions and their Fourier transforms. J. Funct. Anal. 24, 258–267 (1977)

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Bandeira, A.S., Fickus, M., Mixon, D.G., Moreira, J.: Derandomizing restricted isometries via the Legendre symbol. arXiv:1406.4089

  4. 4.

    Bandeira, A.S., Fickus, M., Mixon, D.G., Wong, P.: The road to deterministic matrices with the restricted isometry property. J. Fourier Anal. Appl. 19, 1123–1149 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  5. 5.

    Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28, 253–263 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  6. 6.

    Beckner, W.: Inequalities in Fourier analysis. Ann. Math. 102, 159–182 (1975)

    MathSciNet  Article  MATH  Google Scholar 

  7. 7.

    Blanchard, J.D., Cartis, C., Tanner, J.: Compressed sensing: How sharp is the restricted isometry property? SIAM Rev. 53, 105–125 (2011)

    MathSciNet  Article  MATH  Google Scholar 

  8. 8.

    Bourgain, J.: An improved estimate in the restricted isometry problem. Lect. Notes Math. 2116, 65–70 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Cahill, J., Mixon, D.G.: Robust width: A characterization of uniformly stable and robust compressed sensing. arXiv:1408.4409

  10. 10.

    Cai, T.T., Zhang, A.: Sharp RIP bound for sparse signal and low-rank matrix recovery. Appl. Comput. Harmon. Anal. 35, 74–93 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    Candès, E.J.: The restricted isometry property and its implications for compressed sensing. C. R. Acad. Sci. Paris Ser. I 346, 589–592 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    Candès, E.J., Tao, T.: Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52, 5406–5425 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  13. 13.

    Csörgő, S.: A rate of convergence for coupon collectors. Acta Sci. Math. (Szeged) 57, 337–351 (1993)

    MathSciNet  MATH  Google Scholar 

  14. 14.

    Demanet, L., Hand, P.: Scaling law for recovering the sparsest element in a subspace. arXiv:1310.1654

  15. 15.

    Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via \(\ell ^1\) minimization. Proc. Natl. Acad. Sci. USA 100, 2197–2202 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Donoho, D.L., Huo, X.: Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 47, 2845–2862 (2001)

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    Donoho, D.L., Stark, P.B.: Uncertainty principles and signal recovery. SIAM J. Appl. Math. 49, 906–931 (1989)

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    Erdős, P., Rényi, A.: On a classical problem of probability theory. Magy. Tud Akad. Mat. Kutató Int. Közl. 6, 215–220 (1961)

    MathSciNet  MATH  Google Scholar 

  19. 19.

    Foucart, A., Rauhut, H.: A Mathematical Introduction to Compressive Sensing. Applied and Numerical Harmonic Analysis. Birkhäuser, Basel (2013)

    Google Scholar 

  20. 20.

    Gray, W.C.: Variable norm deconvolution, Tech. Rep. SEP-14, Stanford University (1978)

  21. 21.

    Gribonval, R., Nielsen, M.: Highly sparse representations from dictionaries are unique and independent of the sparseness measure. Appl. Comput. Harmon. Anal. 22, 335–355 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Guédon, O., Mendelson, S., Pajor, A., Tomczak-Jaegermann, N.: Majorizing measures and proportional subsets of bounded orthonormal systems. Rev. Mat. Iberoam. 24, 1075–1095 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  23. 23.

    Hardy, G.H.: A theorem concerning Fourier transforms. J. Lond. Math. Soc. 8, 227–231 (1933)

    MathSciNet  Article  MATH  Google Scholar 

  24. 24.

    Heisenberg, W.: Über den anschaulichen Inhalt der quantentheoretischen Kinematik und Mechanik. Z. Phys. (in German) 43, 172–198 (1927)

    Article  MATH  Google Scholar 

  25. 25.

    Hurley, N., Rickard, S.: Comparing measures of sparsity. IEEE Trans. Inf. Theory 55, 4723–4741 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Indyk P, Kapralov M.: Sample-optimal Fourier sampling in any constant dimension. In: foundations of computer science (FOCS), 2014 IEEE 55th Annual Symposium on 2014, pp. 514–523

  27. 27.

    Kashin, B.S., Temlyakov, V.N.: A remark on compressed sensing. Math. Notes 82, 748–755 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  28. 28.

    Krahmer, F., Mendelson, S., Rauhut, H.: Suprema of chaos processes and the restricted isometry property. Commun. Pure Appl. Math. 67, 1877–1904 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  29. 29.

    Kraus, K.: Complementary observables and uncertainty relations. Phys. Rev. D 35, 3070–3075 (1987)

    MathSciNet  Article  Google Scholar 

  30. 30.

    Laurent, B., Massart, P.: Adaptive estimation of a quadratic functional by model selection. Ann. Stat. 28, 1302–1338 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  31. 31.

    Lopes, M.E.: Estimating unknown sparsity in compressed sensing. JMLR W&CP 28, 217–225 (2013)

    Google Scholar 

  32. 32.

    McCoy, M.B., Cevher, V., Dinh, Q.T., Asaei, A., Baldassarre, L.: Convexity in source separation: models, geometry, and algorithms. Signal Proc. Mag. 31, 87–95 (2014)

    Article  Google Scholar 

  33. 33.

    McCoy, M.B., Tropp, J.A.: Sharp recovery bounds for convex demixing, with applications. Found. Comput. Math. 14, 503–567 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  34. 34.

    Nelson, J., Price, E., Wootters, M.: New constructions of RIP matrices with fast multiplication and fewer rows. In: SODA, pp. 1515–1528 (2014)

  35. 35.

    Rauhut, H.: Compressive sensing and structured random matrices. Theor. Found. Numer. Methods Sparse Recover. 9, 1–92 (2010)

    MathSciNet  MATH  Google Scholar 

  36. 36.

    Repetti, A., Pham, M.Q., Duval, L., Chouzenoux, É., Pesquet, J.-C.: Euclid in a taxicab: sparse blind deconvolution with smoothed \(\ell _1/\ell _2\) regularization. IEEE Signal Process. Lett. 22, 539–543 (2014)

    Article  Google Scholar 

  37. 37.

    Rudelson, M., Vershynin, R.: On sparse reconstruction from Fourier and Gaussian measurements. Comm. Pure Appl. Math. 61, 1025–1045 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  38. 38.

    Santosa, F., Symes, W.W.: Linear inversion of band-limited reflection seismograms. SIAM J. Sci. Stat. Comput. 7, 1307–1330 (1986)

    MathSciNet  Article  MATH  Google Scholar 

  39. 39.

    Stevenhagen, P., Lenstra, H.W.: Chebotarëv and his density theorem. Math. Intell. 18, 26–37 (1996)

    Article  MATH  Google Scholar 

  40. 40.

    Studer, C.: Recovery of signals with low density. arXiv:1507.02821

  41. 41.

    Talagrand, M.: Selecting a proportion of characters. Israel J. Math. 108, 173–191 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  42. 42.

    Tao, T.: An uncertainty principle for cyclic groups of prime order. Math. Res. Lett. 12, 121–127 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  43. 43.

    Terras, A.: Fourier Analysis on Finite Groups and Applications. Cambridge University Press, Cambridge (1999)

    Google Scholar 

  44. 44.

    Tropp, J.A.: The sparsity gap: uncertainty principles proportional to dimension. In: CISS, pp. 1–6 (2010)

  45. 45.

    Tropp, J.A.: On the linear independence of spikes and sines. J. Fourier Anal. Appl. 14, 838–858 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  46. 46.

    Tropp, J.A.: On the conditioning of random subdictionaries. Appl. Comput. Harmon. Anal. 25, 1–24 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  47. 47.

    Vershynin, R.: Introduction to the non-asymptotic analysis of random matrices. Theory and Applications, Cambridge University Press, Compressed Sensing (2012)

Download references

Acknowledgements

The authors thank Laurent Duval, Joel Tropp, and the anonymous referees for multiple suggestions that significantly improved the presentation of our results and our discussion of the relevant literature. ASB was supported by AFOSR Grant No. FA9550-12-1-0317. DGM was supported by an AFOSR Young Investigator Research Program award, NSF Grant No. DMS-1321779, and AFOSR Grant No. F4FGA05076J002. The views expressed in this article are those of the authors and do not reflect the official policy or position of the United States Air Force, Department of Defense, or the U.S. Government.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Dustin G. Mixon.

Additional information

Communicated by Roman Vershynin.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bandeira, A.S., Lewis, M.E. & Mixon, D.G. Discrete Uncertainty Principles and Sparse Signal Processing. J Fourier Anal Appl 24, 935–956 (2018). https://doi.org/10.1007/s00041-017-9550-x

Download citation

Keywords

  • Uncertainty principle
  • Sparsity
  • Compressed sensing