Abstract
In this paper we investigate the efficiency of the Orthogonal Matching Pursuit algorithm (OMP) for random dictionaries. We concentrate on dictionaries satisfying the Restricted Isometry Property. We also introduce a stronger Homogenous Restricted Isometry Property which we show is satisfied with overwhelming probability for random dictionaries used in compressed sensing. For these dictionaries we obtain upper estimates for the error of approximation by OMP in terms of the error of the best n-term approximation (Lebesgue-type inequalities). We also present and discuss some open problems about OMP. This is a development of recent results obtained by D.L. Donoho, M. Elad and V.N. Temlyakov.
Article PDF
Similar content being viewed by others
References
Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28(3), 253–263 (2008)
Candés, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005)
Christensen, O.: An Introduction to Frames and Riesz Bases. Applied and Numerical Harmonic Analysis. Birkhäuser, Boston (2003)
Cohen, A., Dahmen, W., DeVore, R.: Compressed sensing and best k-term approximation. J. Am. Math. Soc. 22(1), 211–231 (2009)
Davenport, M.A., Wakin, M.B.: Analysis of orthogonal matching pursuit using the restricted isometry property. Preprint (2009)
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
Donoho, D.L., Elad, M., Temlyakov, V.N.: On Lebesgue-type inequalities for greedy approximation. J. Approx. Theory 147(2), 185–195 (2007)
Gilbert, A.C., Muthukrishnan, S., Strauss, M.J.: Approximation of functions over redundant dictionaries using coherence. In: Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, Baltimore, MD, 2003, pp. 243–252. ACM, New York (2003)
Huber, P.J.: Projection pursuit. Ann. Stat. 13(2), 435–525 (1985). With discussion
Jones, L.K.: On a conjecture of Huber concerning the convergence of projection pursuit regression. Ann. Stat. 15(2), 880–882 (1987)
Kwapien, S., Pelczyński, A.: The main triangle projection in matrix spaces and its applications. Studia Math. 34, 43–68 (1970)
Mendelson, S., Pajor, A., Tomczak-Jaegermann, N.: Uniform uncertainty principle for Bernoulli and subgaussian ensembles. Constr. Approx. 28(3), 277–289 (2008)
Temlyakov, V.N.: Greedy approximation. Ada Numer. 17, 235–409 (2008)
Tropp, J.A.: Greed is good: algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50(10), 2231–2242 (2004)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Vladimir N. Temlyakov.
This research was partially supported by the Polish Ministry of Science and Higher Education grant no. N N201 269335.
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Bechler, P., Wojtaszczyk, P. Error Estimates for Orthogonal Matching Pursuit and Random Dictionaries. Constr Approx 33, 273–288 (2011). https://doi.org/10.1007/s00365-010-9122-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00365-010-9122-7
Keywords
- Orthogonal matching pursuit
- Coherence
- Restricted isometry property
- Random dictionaries
- Lebesgue inequalities
- Nonlinear approximation