Skip to main content
SpringerLink
Log in

A Survey of Compressed Sensing

  • Chapter
Book cover Compressed Sensing and its Applications

Part of the book series: Applied and Numerical Harmonic Analysis ((ANHA))

Abstract

Compressed sensing was introduced some ten years ago as an effective way of acquiring signals, which possess a sparse or nearly sparse representation in a suitable basis or dictionary. Due to its solid mathematical backgrounds, it quickly attracted the attention of mathematicians from several different areas, so that the most important aspects of the theory are nowadays very well understood. In recent years, its applications started to spread out through applied mathematics, signal processing, and electrical engineering. The aim of this chapter is to provide an introduction into the basic concepts of compressed sensing. In the first part of this chapter, we present the basic mathematical concepts of compressed sensing, including the Null Space Property, Restricted Isometry Property, their connection to basis pursuit and sparse recovery, and construction of matrices with small restricted isometry constants. This presentation is easily accessible, largely self-contained, and includes proofs of the most important theorems. The second part gives an overview of the most important extensions of these ideas, including recovery of vectors with sparse representation in frames and dictionaries, discussion of (in)coherence and its implications for compressed sensing, and presentation of other algorithms of sparse recovery.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Achlioptas, D.: Database-friendly random projections: Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66, 671–687 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  2. Arora, S., Barak, B.: Computational Complexity: A Modern Approach. Cambridge University Press, Cambridge (2009)

    Book  Google Scholar 

  3. Baraniuk, R., Cevher, V., Duarte, M.F., Hegde, C.: Model-based compressive sensing, IEEE Trans. Inf. Theory 56, 1982–2001 (2010)

    Article  MathSciNet  Google Scholar 

  4. Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28, 253–263 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  5. Baraniuk, R., Steeghs, P.: Compressive radar imaging. In: Proc. IEEE Radar Conf., Boston, pp. 128–133 (2007)

    Google Scholar 

  6. Baron, D., Duarte, M.F., Sarvotham, S., Wakin, M.B., Baraniuk, R.: Distributed compressed sensing of jointly sparse signals. In: Proc. Asilomar Conf. Signals, Systems, and Computers, Pacic Grove (2005)

    Google Scholar 

  7. Becker, S., Candés, E.J., Grant, M.: Templates for convex cone problems with applications to sparse signal recovery. Math. Program. Comput. 3, 165–218 (2010)

    Article  Google Scholar 

  8. Blumensath, T., Davies, M.: Iterative hard thresholding for compressive sensing. Appl. Comput. Harmon. Anal. 27, 265–274 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  9. Cai, T., Wang, L., Xu, G.: New bounds for restricted isometry constants. IEEE Trans. Inf. Theory 56, 4388–4394 (2010)

    Article  MathSciNet  Google Scholar 

  10. Cai, T., Wang, L., Xu, G.: Shifting inequality and recovery of sparse vectors. IEEE Trans. Signal Process. 58, 1300–1308 (2010)

    Article  MathSciNet  Google Scholar 

  11. Candés, E.J.: The restricted isometry property and its implications for compressed sensing. C. R. Acad. Sci., Paris, Ser. I 346, 589–592 (2008)

    MATH  Google Scholar 

  12. Candés, E.J., Plan, Y.: A probabilistic and RIPless theory of compressed sensing. IEEE Trans. Inf. Theory 57, 7235–7254 (2011)

    Article  Google Scholar 

  13. Candés, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489–509 (2006)

    Article  MATH  Google Scholar 

  14. Candés, E.J., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59, 1207–1223 (2006)

    Article  MATH  Google Scholar 

  15. Candés, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51, 4203–4215 (2005)

    Article  MATH  Google Scholar 

  16. Candés, E.J., Tao, T.: Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory 52, 5406–5425 (2006)

    Article  Google Scholar 

  17. Candés, E.J., Tao, T.: The Dantzig selector: statistical estimation when p is much larger than n. Ann. Stat. 35, 2313–2351 (2007)

    Google Scholar 

  18. Cevher, V., Tyagi, H.: Active learning of multi-index function models. In: Proc. NIPS (The Neural Information Processing Systems), Lake Tahoe, Reno (2012)

    Google Scholar 

  19. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20, 33–61 (1998)

    Article  MathSciNet  Google Scholar 

  20. Cohen, A., Dahmen, W., DeVore, R.: Compressed sensing and best k-term approximation. J. Am. Math. Soc. 22, 211–231 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  21. Cohen, A., Daubechies, I., DeVore, R., Kerkyacharian, G., Picard, D.: Capturing ridge functions in high dimensions from point queries. Constr. Approx. 35, 225–243 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. Dasgupta, S., Gupta, A.: An elementary proof of a theorem of Johnson and Lindenstrauss. Random Struct. Algorithm. 22, 60–65 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  23. Davenport, M.A., Duarte, M.F., Eldar, Y.C., Kutyniok, G.: Introduction to compressed sensing. Compressed sensing, pp. 1–64. Cambridge University Press, Cambridge (2012)

    Google Scholar 

  24. DeVore, R., Petrova, G., Wojtaszczyk, P.: Approximation of functions of few variables in high dimensions. Constr. Approx. 33, 125–143 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  25. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  26. Donoho, D.L., Tsaig, Y., Drori, I., Starck, J.-L.: Sparse solution of underdetermined systems of linear equations by stagewise Orthogonal Matching Pursuit. IEEE Trans. Inf. Theory 58, 1094–1121 (2012)

    Article  MathSciNet  Google Scholar 

  27. Dorfman, R.: The detection of defective members of large populations. Ann. Math. Stat. 14, 436–440 (1943)

    Article  Google Scholar 

  28. Du, D., Hwang, F.: Combinatorial group testing and its applications. World Scientic, Singapore (2000)

    MATH  Google Scholar 

  29. Duarte, M., Davenport, M., Takhar, D., Laska, J., Ting, S., Kelly, K., Baraniuk R.: Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25, 83–91 (2008)

    Article  Google Scholar 

  30. Duarte, M., Wakin, M., Baraniuk, R.: Fast reconstruction of piecewise smooth signals from random projections. In: Proc. Work. Struc. Parc. Rep. Adap. Signaux (SPARS), Rennes (2005)

    Google Scholar 

  31. Duarte, M., Wakin, M., Baraniuk, R.: Wavelet-domain compressive signal reconstruction using a hidden Markov tree model. In: Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing (ICASSP), Las Vegas (2008)

    Google Scholar 

  32. Eldar, Y., Mishali, M.: Robust recovery of signals from a structured union of subspaces. IEEE Trans. Inf. Theory 55, 5302–5316 (2009)

    Article  MathSciNet  Google Scholar 

  33. Elad, M.: Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer, New York (2010)

    Book  Google Scholar 

  34. Figueiredo, M., Nowak, R., Wright, S.: Gradient projections for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Select. Top. Signal Process. 1, 586–597 (2007)

    Article  Google Scholar 

  35. Fornasier, M., Rauhut, H.: Compressive sensing. In: Scherzer, O. (ed.) Handbook of Mathematical Methods in Imaging, pp. 187–228. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  36. Fornasier, M., Rauhut, H.: Recovery algorithms for vector valued data with joint sparsity constraints. SIAM J. Numer. Anal. 46, 577–613 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  37. Fornasier, M., Schnass, K., Vybíral, J.: Learning functions of few arbitrary linear parameters in high dimensions. Found. Comput. Math. 12, 229–262 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  38. Foucart, S.: A note on guaranteed sparse recovery via 1-minimization. Appl. Comput. Harmon. Anal. 29, 97–103 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  39. Foucart, S., Lai M.: Sparsest solutions of underdetermined linear systems via l q -minimization for 0 < q ≤ 1. Appl. Comput. Harmon. Anal. 26, 395–407 (2009)

    Google Scholar 

  40. Foucart, S., Rauhut, H.: A Mathematical Introduction to Compressive Sensing. Birkhäuser/Springer, New York (2013)

    Book  MATH  Google Scholar 

  41. Friedman, J., Hastie, T., Tibshirani, R.: Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33, 1–22 (2010)

    Google Scholar 

  42. Gärtner, B., Matoušek, J.: Understanding and Using Linear Programming. Springer, Berlin (2006)

    Google Scholar 

  43. Gilbert, A., Li, Y., Porat, E., and Strauss, M.: Approximate sparse recovery: optimizaing time and measurements. In: Proc. ACM Symp. Theory of Comput., Cambridge (2010)

    Book  Google Scholar 

  44. Gilbert, A., Strauss, M., Tropp, J., Vershynin, R.: One sketch for all: fast algorithms for compressed sensing. In: Proc. ACM Symp. Theory of Comput., San Diego (2007)

    Book  Google Scholar 

  45. Jasper, J., Mixon, D.G., Fickus M.: Kirkman equiangular tight frames and codes. IEEE Trans. Inf. Theory 60, 170–181 (2014)

    Article  MathSciNet  Google Scholar 

  46. Johnson, W.B., Lindenstrauss, J.: Extensions of Lipschitz mappings into a Hilbert space. In: Conf. in Modern Analysis and Probability, pp. 189–206 (1984)

    Google Scholar 

  47. Krahmer, F., Ward, R.: New and improved Johnson-Lindenstrauss embeddings via the restricted isometry property. SIAM J. Math. Anal. 43, 1269–1281 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  48. La, C., Do, M.N.: Tree-based orthogonal matching pursuit algorithm for signal reconstruction. In: IEEE Int. Conf. Image Processing (ICIP), Atlanta (2006)

    Google Scholar 

  49. Ledoux, M.: The Concentration of Measure Phenomenon. American Mathematical Society, Providence (2001)

    MATH  Google Scholar 

  50. Ledoux, M., Talagrand, M.: Probability in Banach Spaces. Isoperimetry and Processes. Springer, Berlin (1991)

    Book  MATH  Google Scholar 

  51. Loris, I.: On the performance of algorithms for the minimization of ℓ 1-penalized functions. Inverse Prob. 25, 035008 (2009)

    Article  MathSciNet  Google Scholar 

  52. Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 58, 1182–1195 (2007)

    Article  Google Scholar 

  53. Mallat, S., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41, 3397–3415 (1993)

    Article  MATH  Google Scholar 

  54. Matoušek, J.: On variants of the Johnson-Lindenstrauss lemma. Rand. Struct. Algorithm. 33, 142–156 (2008)

    Article  MATH  Google Scholar 

  55. Milman, V.D., Schechtman, G.: Asymptotic theory of finite-dimensional normed spaces. Springer, Berlin (1986)

    MATH  Google Scholar 

  56. Mishali, M., Eldar, Y.: From theory to practice: Sub-nyquist sampling of sparse wideband analog signals. IEEE J. Sel. Top. Signal Process. 4, 375–391 (2010)

    Article  Google Scholar 

  57. Needell, D., Tropp, J.: CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26, 301–321 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  58. Pietsch, A.: Operator Ideals. North-Holland, Amsterdam (1980)

    MATH  Google Scholar 

  59. Rauhut, H.: Random sampling of sparse trigonometric polynomials. Appl. Comput. Harmon. Anal. 22, 16–42 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  60. Rauhut, H., Schnass, K., Vandergheynst, P.: Compressed sensing and redundant dictionaries. IEEE Trans. Inf. Theor. 54, 2210–2219 (2008)

    Article  MathSciNet  Google Scholar 

  61. Rauhut, H., Ward, R.: Sparse Legendre expansions via ℓ 1-minimization. J. Approx. Theory 164, 517–533 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  62. Schnass, K., Vybíral, J.: Compressed learning of high-dimensional sparse functions. In: IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 3924–3927 (2011)

    Google Scholar 

  63. Stojnic, M., Parvaresh, F., Hassibi, B.: On the reconstruction of block-sparse signals with an optimal number of measurements. IEEE Trans. Signal Process. 57, 3075–3085 (2009)

    Article  MathSciNet  Google Scholar 

  64. Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. B 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  65. Tropp, J.: Greed is good: algorithmic results for sparse approximation. IEEE Trans. Inf. Theor. 50, 2231–2242 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  66. Tropp, J.: Algorithms for simultaneous sparse approximation. Part II: Convex relaxation. Signal Process. 86, 589–602 (2006)

    MATH  Google Scholar 

  67. Tropp, J., Gilbert, A.: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory 53, 4655–4666 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  68. Tropp, J., Gilbert, A., Strauss, M.: Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit. Signal Process. 86, 572–588 (2006)

    MATH  Google Scholar 

  69. Tropp, J., Laska, J., Duarte, M., Romberg, J., Baraniuk, R.: Beyond Nyquist: efficient sampling of sparse bandlimited signals. IEEE Trans. Inf. Theor. 56, 520–544 (2010)

    Article  MathSciNet  Google Scholar 

  70. Wakin, M.B., Sarvotham, S., Duarte, M.F., Baron, D., Baraniuk, R.: Recovery of jointly sparse signals from few random projections. In: Proc. Workshop on Neural Info. Proc. Sys. (NIPS), Vancouver (2005)

    Google Scholar 

  71. Welch, L.: Lower bounds on the maximum cross correlation of signals. IEEE Trans. Inf. Theory 20, 397–399 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  72. Wojtaszczyk, P.: Complexity of approximation of functions of few variables in high dimensions. J. Complex. 27, 141–150 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  73. Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for ℓ 1-minimization with applications to compressed sensing. SIAM J. Imag. Sci. 1, 143–168 (2008)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We would like to thank Sandra Keiper and Holger Rauhut for careful reading of a previous version of this Introduction and for useful comments. The last author was supported by the ERC CZ grant LL1203 of the Czech Ministry of Education.

Author information

Authors and Affiliations

  1. Technische Universität München, Theresienstr. 90/IV, München, Germany

    Holger Boche

  2. Duke University, 317 Gross Hall, Durham, NC, USA

    Robert Calderbank

  3. Technische Universität Berlin, Straße des 17. Juni 136, Berlin, Germany

    Gitta Kutyniok

  4. Faculty of Mathematics and Physics, Charles University, Sokolovska 83, 186 00, Prague 8, Czech Republic

    Jan Vybíral

Authors
  1. Holger Boche
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert Calderbank
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gitta Kutyniok
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jan Vybíral
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Holger Boche .

Editor information

Editors and Affiliations

  1. Lehrstuhl für Theoretische Informationstechnik, Technische Universität München, München, Bayern, Germany

    Holger Boche

  2. Department of Electrical and Computer Engineering, Duke University, Durham, North Carolina, USA

    Robert Calderbank

  3. Institut für Mathematik, Technische Universität Berlin, Berlin, Berlin, Germany

    Gitta Kutyniok

  4. Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic

    Jan Vybíral

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Boche, H., Calderbank, R., Kutyniok, G., Vybíral, J. (2015). A Survey of Compressed Sensing. In: Boche, H., Calderbank, R., Kutyniok, G., Vybíral, J. (eds) Compressed Sensing and its Applications. Applied and Numerical Harmonic Analysis. Birkhäuser, Cham. https://doi.org/10.1007/978-3-319-16042-9_1

Download citation

Publish with us

Policies and ethics

Search

Navigation