Advertisement

Constructive Approximation

, Volume 28, Issue 3, pp 253–263 | Cite as

A Simple Proof of the Restricted Isometry Property for Random Matrices

  • Richard Baraniuk
  • Mark Davenport
  • Ronald DeVore
  • Michael Wakin
Article

Abstract

We give a simple technique for verifying the Restricted Isometry Property (as introduced by Candès and Tao) for random matrices that underlies Compressed Sensing. Our approach has two main ingredients: (i) concentration inequalities for random inner products that have recently provided algorithmically simple proofs of the Johnson–Lindenstrauss lemma; and (ii) covering numbers for finite-dimensional balls in Euclidean space. This leads to an elementary proof of the Restricted Isometry Property and brings out connections between Compressed Sensing and the Johnson–Lindenstrauss lemma. As a result, we obtain simple and direct proofs of Kashin’s theorems on widths of finite balls in Euclidean space (and their improvements due to Gluskin) and proofs of the existence of optimal Compressed Sensing measurement matrices. In the process, we also prove that these measurements have a certain universality with respect to the sparsity-inducing basis.

Keywords

Compressed sensing Sampling Random matrices Concentration inequalities 

Mathematics Subject Classification (2000)

15N2 15A52 60F10 94A12 94A20 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Achlioptas, D.: Database-friendly random projections. In: Proc. ACM SIGACT-SIGMOD-SIGART Symp. on Principles of Database Systems, pp. 274–281, 2001 Google Scholar
  2. 2.
    Candès, E., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006) CrossRefGoogle Scholar
  3. 3.
    Candès, E., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2005) CrossRefGoogle Scholar
  4. 4.
    Candès, E., Tao, T.: Decoding by linear programing. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005) CrossRefGoogle Scholar
  5. 5.
    Cohen, A., Dahmen, W., DeVore, R.: Compressed sensing and best k-term approximation. Preprint (2006) Google Scholar
  6. 6.
    Cohen, A., Dahmen, W., DeVore, R.: Near optimal approximation of arbitrary signals from highly incomplete measurements. Preprint (2007) Google Scholar
  7. 7.
    Dasgupta, S., Gupta, A.: An elementary proof of the Johnson–Lindenstrauss lemma. Tech. Report Technical report 99-006, U.C. Berkeley (March, 1999) Google Scholar
  8. 8.
    Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006) CrossRefMathSciNetGoogle Scholar
  9. 9.
    Frankl, P., Maehara, H.: The Johnson–Lindenstrauss lemma and the sphericity of some graphs. J. Comb. Theory Ser. B 44(3), 355–362 (1988) MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Gilbert, A., Guha, S., Indyk, P., Muthukrishnan, S., Strauss, M.: Near-optimal sparse Fourier representations via sampling, 2005. In: ACM Symp. on Theoretical Computer Science, 2002 Google Scholar
  11. 11.
    Garnaev, A., Gluskin, E.D.: The widths of Euclidean balls. Dokl. An. SSSR 277, 1048–1052 (1984) MathSciNetGoogle Scholar
  12. 12.
    Indyk, P., Motwani, R.: Approximate nearest neighbors: towards removing the curse of dimensionality. In: Symp. on Theory of Computing, pp. 604–613, 1998 Google Scholar
  13. 13.
    Johnson, W.B., Lindenstrauss, J.: Extensions of Lipschitz mappings into a Hilbert space. In: Conf. in Modern Analysis and Probability, pp. 189–206, 1984 Google Scholar
  14. 14.
    Kashin, B.: The widths of certain finite dimensional sets and classes of smooth functions. Izvestia (41), 334–351 (1977) Google Scholar
  15. 15.
    Ledoux, M.: The Concentration of Measure Phenomenon. Am. Math. Soc., Providence (2001) MATHGoogle Scholar
  16. 16.
    Lorentz, G.G., von Golitschek, M., Makovoz, Yu.: Constructive Approximation: Advanced Problems, vol. 304. Springer, Berlin (1996) MATHGoogle Scholar
  17. 17.
    Milman, V.D., Pajor, A.: Regularization of star bodies by random hyperplane cut off. Studia Math. 159(2), 247–261 (2003) MATHMathSciNetCrossRefGoogle Scholar
  18. 18.
    Milman, V.D., Schechtman, G.: Asymptotic Theory of Finite-Dimensional Normed Spaces. Lecture Notes in Mathematics, vol. 1200. Springer, Berlin (1986) MATHGoogle Scholar
  19. 19.
    Mendelson, S., Pajor, A., Tomczack-Jaegermann, N.: Reconstruction and subgaussian operators in asymptotic geometric analysis. Preprint (2006) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Richard Baraniuk
    • 1
  • Mark Davenport
    • 1
  • Ronald DeVore
    • 2
  • Michael Wakin
    • 3
  1. 1.Department of Electrical and Computer EngineeringRice UniversityHoustonUSA
  2. 2.Industrial Mathematics Institute, Department of Mathematics and StatisticsUniversity of South CarolinaColumbiaUSA
  3. 3.Department of Electrical Engineering and Computer ScienceUniversity of MichiganAnn ArborUSA

Personalised recommendations