Skip to main content
SpringerLink
Log in

Discrete Uncertainty Principles and Sparse Signal Processing

  • Published:
Journal of Fourier Analysis and Applications Aims and scope Submit manuscript

Abstract

We develop new discrete uncertainty principles in terms of numerical sparsity, which is a continuous proxy for the 0-norm. Unlike traditional sparsity, the continuity of numerical sparsity naturally accommodates functions which are nearly sparse. After studying these principles and the functions that achieve exact or near equality in them, we identify certain consequences in a number of sparse signal processing applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Recall that \(f(n)=O(g(n))\) if there exists \(C,n_0>0\) such that \(f(n)\le Cg(n)\) for all \(n>n_0\). We write \(f(n)=O_\delta (g(n))\) if the constant C is a function of \(\delta \). Also, \(f(n)=\Omega (g(n))\) if \(g(n)=O(f(n))\), and \(f(n)=o(g(n))\) if \(f(n)/g(n)\rightarrow 0\) as \(n\rightarrow \infty \).

References

  1. Alon, N.: Problems and results in extremal combinatorics. Discret. Math. 273, 31–53 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  2. Amrein, W.O., Berthier, A.M.: On support properties of \(L^p\)-functions and their Fourier transforms. J. Funct. Anal. 24, 258–267 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bandeira, A.S., Fickus, M., Mixon, D.G., Moreira, J.: Derandomizing restricted isometries via the Legendre symbol. arXiv:1406.4089

  4. Bandeira, A.S., Fickus, M., Mixon, D.G., Wong, P.: The road to deterministic matrices with the restricted isometry property. J. Fourier Anal. Appl. 19, 1123–1149 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  5. Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28, 253–263 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Beckner, W.: Inequalities in Fourier analysis. Ann. Math. 102, 159–182 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  7. Blanchard, J.D., Cartis, C., Tanner, J.: Compressed sensing: How sharp is the restricted isometry property? SIAM Rev. 53, 105–125 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bourgain, J.: An improved estimate in the restricted isometry problem. Lect. Notes Math. 2116, 65–70 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  9. Cahill, J., Mixon, D.G.: Robust width: A characterization of uniformly stable and robust compressed sensing. arXiv:1408.4409

  10. Cai, T.T., Zhang, A.: Sharp RIP bound for sparse signal and low-rank matrix recovery. Appl. Comput. Harmon. Anal. 35, 74–93 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  11. Candès, E.J.: The restricted isometry property and its implications for compressed sensing. C. R. Acad. Sci. Paris Ser. I 346, 589–592 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  12. Candès, E.J., Tao, T.: Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52, 5406–5425 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  13. Csörgő, S.: A rate of convergence for coupon collectors. Acta Sci. Math. (Szeged) 57, 337–351 (1993)

    MathSciNet  MATH  Google Scholar 

  14. Demanet, L., Hand, P.: Scaling law for recovering the sparsest element in a subspace. arXiv:1310.1654

  15. Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via \(\ell ^1\) minimization. Proc. Natl. Acad. Sci. USA 100, 2197–2202 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  16. Donoho, D.L., Huo, X.: Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 47, 2845–2862 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  17. Donoho, D.L., Stark, P.B.: Uncertainty principles and signal recovery. SIAM J. Appl. Math. 49, 906–931 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  18. Erdős, P., Rényi, A.: On a classical problem of probability theory. Magy. Tud Akad. Mat. Kutató Int. Közl. 6, 215–220 (1961)

    MathSciNet  MATH  Google Scholar 

  19. Foucart, A., Rauhut, H.: A Mathematical Introduction to Compressive Sensing. Applied and Numerical Harmonic Analysis. Birkhäuser, Basel (2013)

    MATH  Google Scholar 

  20. Gray, W.C.: Variable norm deconvolution, Tech. Rep. SEP-14, Stanford University (1978)

  21. Gribonval, R., Nielsen, M.: Highly sparse representations from dictionaries are unique and independent of the sparseness measure. Appl. Comput. Harmon. Anal. 22, 335–355 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  22. Guédon, O., Mendelson, S., Pajor, A., Tomczak-Jaegermann, N.: Majorizing measures and proportional subsets of bounded orthonormal systems. Rev. Mat. Iberoam. 24, 1075–1095 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hardy, G.H.: A theorem concerning Fourier transforms. J. Lond. Math. Soc. 8, 227–231 (1933)

    Article  MathSciNet  MATH  Google Scholar 

  24. Heisenberg, W.: Über den anschaulichen Inhalt der quantentheoretischen Kinematik und Mechanik. Z. Phys. (in German) 43, 172–198 (1927)

    Article  MATH  Google Scholar 

  25. Hurley, N., Rickard, S.: Comparing measures of sparsity. IEEE Trans. Inf. Theory 55, 4723–4741 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  26. Indyk P, Kapralov M.: Sample-optimal Fourier sampling in any constant dimension. In: foundations of computer science (FOCS), 2014 IEEE 55th Annual Symposium on 2014, pp. 514–523

  27. Kashin, B.S., Temlyakov, V.N.: A remark on compressed sensing. Math. Notes 82, 748–755 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  28. Krahmer, F., Mendelson, S., Rauhut, H.: Suprema of chaos processes and the restricted isometry property. Commun. Pure Appl. Math. 67, 1877–1904 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  29. Kraus, K.: Complementary observables and uncertainty relations. Phys. Rev. D 35, 3070–3075 (1987)

    Article  MathSciNet  Google Scholar 

  30. Laurent, B., Massart, P.: Adaptive estimation of a quadratic functional by model selection. Ann. Stat. 28, 1302–1338 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  31. Lopes, M.E.: Estimating unknown sparsity in compressed sensing. JMLR W&CP 28, 217–225 (2013)

    Google Scholar 

  32. McCoy, M.B., Cevher, V., Dinh, Q.T., Asaei, A., Baldassarre, L.: Convexity in source separation: models, geometry, and algorithms. Signal Proc. Mag. 31, 87–95 (2014)

    Article  Google Scholar 

  33. McCoy, M.B., Tropp, J.A.: Sharp recovery bounds for convex demixing, with applications. Found. Comput. Math. 14, 503–567 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  34. Nelson, J., Price, E., Wootters, M.: New constructions of RIP matrices with fast multiplication and fewer rows. In: SODA, pp. 1515–1528 (2014)

  35. Rauhut, H.: Compressive sensing and structured random matrices. Theor. Found. Numer. Methods Sparse Recover. 9, 1–92 (2010)

    MathSciNet  MATH  Google Scholar 

  36. Repetti, A., Pham, M.Q., Duval, L., Chouzenoux, É., Pesquet, J.-C.: Euclid in a taxicab: sparse blind deconvolution with smoothed \(\ell _1/\ell _2\) regularization. IEEE Signal Process. Lett. 22, 539–543 (2014)

    Article  Google Scholar 

  37. Rudelson, M., Vershynin, R.: On sparse reconstruction from Fourier and Gaussian measurements. Comm. Pure Appl. Math. 61, 1025–1045 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  38. Santosa, F., Symes, W.W.: Linear inversion of band-limited reflection seismograms. SIAM J. Sci. Stat. Comput. 7, 1307–1330 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  39. Stevenhagen, P., Lenstra, H.W.: Chebotarëv and his density theorem. Math. Intell. 18, 26–37 (1996)

    Article  MATH  Google Scholar 

  40. Studer, C.: Recovery of signals with low density. arXiv:1507.02821

  41. Talagrand, M.: Selecting a proportion of characters. Israel J. Math. 108, 173–191 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  42. Tao, T.: An uncertainty principle for cyclic groups of prime order. Math. Res. Lett. 12, 121–127 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  43. Terras, A.: Fourier Analysis on Finite Groups and Applications. Cambridge University Press, Cambridge (1999)

    Book  MATH  Google Scholar 

  44. Tropp, J.A.: The sparsity gap: uncertainty principles proportional to dimension. In: CISS, pp. 1–6 (2010)

  45. Tropp, J.A.: On the linear independence of spikes and sines. J. Fourier Anal. Appl. 14, 838–858 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  46. Tropp, J.A.: On the conditioning of random subdictionaries. Appl. Comput. Harmon. Anal. 25, 1–24 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  47. Vershynin, R.: Introduction to the non-asymptotic analysis of random matrices. Theory and Applications, Cambridge University Press, Compressed Sensing (2012)

Download references

Acknowledgements

The authors thank Laurent Duval, Joel Tropp, and the anonymous referees for multiple suggestions that significantly improved the presentation of our results and our discussion of the relevant literature. ASB was supported by AFOSR Grant No. FA9550-12-1-0317. DGM was supported by an AFOSR Young Investigator Research Program award, NSF Grant No. DMS-1321779, and AFOSR Grant No. F4FGA05076J002. The views expressed in this article are those of the authors and do not reflect the official policy or position of the United States Air Force, Department of Defense, or the U.S. Government.

Author information

Authors and Affiliations

  1. Department of Mathematics, Courant Institute of Mathematical Sciences, New York University, New York, NY, USA

    Afonso S. Bandeira

  2. Detachment 5, Air Force Operational Test and Evaluation Center, Edwards AFB, CA, USA

    Megan E. Lewis

  3. Department of Mathematics and Statistics, Air Force Institute of Technology, Wright-Patterson AFB, OH, USA

    Dustin G. Mixon

Authors
  1. Afonso S. Bandeira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Megan E. Lewis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dustin G. Mixon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dustin G. Mixon.

Additional information

Communicated by Roman Vershynin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bandeira, A.S., Lewis, M.E. & Mixon, D.G. Discrete Uncertainty Principles and Sparse Signal Processing. J Fourier Anal Appl 24, 935–956 (2018). https://doi.org/10.1007/s00041-017-9550-x

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00041-017-9550-x

Keywords

Search

Navigation