Sparsity and incoherence in orthogonal matching pursuit



Recovery of sparse signals via approximation methods has been extensively studied in recently years. We consider the nonuniform recovery of orthogonal matching pursuit (OMP) from fewer noisy random measurements. Rows of sensing matrices are assumed to be drawn independently from a probability distribution obeying the isotropy property and the incoherence property. Our models not only include the standard sensing matrices in compressed sensing context, but also cover other new sensing matrices such as random convolutions, subsampled tight or continuous frames. Given m admissible random measurements of a fixed s-sparse signal \(\varvec{x}\in \mathbb {R}^n\), we show that OMP can recover the support of \(\varvec{x}\) exactly after s iterations with overwhelming probability provided that
$$\begin{aligned} m=O( s(s+\log (n-s))). \end{aligned}$$
It follows that the approximation order of OMP is
$$\begin{aligned} \Vert \varvec{x}- \varvec{x}_j\Vert =O(\eta ^j) \end{aligned}$$
where \(0<\eta <1\) and \(\varvec{x}_j\) denotes the recovered signal at j-th iteration. As a byproduct of the proof, the necessary number of measurements to ensure sparse recovery by \(l_1\)-minimization with random partial circulant or Toeplitz matrices is proved to be optimal.


Sparsity Orthogonal matching pursuit Isotropy property Incoherence property Support recovery 



This work is partially supported the NSF of China under Grant 11671358, the NSAF of China under Grant U1630116, the key project of NSF of China under Number 11531013. Ruifang Hu is also partially supported by the university visiting scholars program under Grant FX2017049. The authors are grateful to the editor and anonymous reviewers for their constructive suggestions and comments to improve the paper.


  1. Bajwa, W. U., Haupt, J., Raz, G., Wright, S. J., Nowak, R., & Toeplitz-structured compressed sensing matrices. (2007). IEEE/SP 14th workshop on statistical signal processing. Madison, WI, USA, pp. 294–298.Google Scholar
  2. Cai, T., & Wang, L. (2011). Orthogonal matching pursuit for sparse signal recovery with noise. IEEE Transactions on Information Theory, 57(11), 4680–4688.MathSciNetCrossRefMATHGoogle Scholar
  3. Cai, T., & Zhang, A. (2014). Sparse representation of a polytope and recovery of sparse signals and low-rank matrices. IEEE Transactions on Information Theory, 60, 122–132.MathSciNetCrossRefMATHGoogle Scholar
  4. Candès, E. J., & Plan, Y. (2011). A probabilistic and RIPless theory of compressed sensing. IEEE Transactions on Information Theory, 57(11), 7235–7254.MathSciNetCrossRefMATHGoogle Scholar
  5. Candès, E. J., & Romberg, J. (2007). Sparsity and incoherence in compressive sampling. Inverse Problems, 23(3), 969–985.MathSciNetCrossRefMATHGoogle Scholar
  6. Candès, E. J., & Tao, T. (2005). Decoding by linear programming. IEEE Transactions on Information Theory, 51(12), 4203–4215.MathSciNetCrossRefMATHGoogle Scholar
  7. Candès, E. J., & Tao, T. (2006). Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Transactions on Information Theory, 52(12), 5406–5425.MathSciNetCrossRefMATHGoogle Scholar
  8. Coifman, R., Geshwind, F., & Meyer, Y. (2001). Noiselets. Applied and Computational Harmonic Analysis, 10, 27–44.MathSciNetCrossRefMATHGoogle Scholar
  9. Cohen, A., Dahmen, W., & DeVore, R. (2017). Orthogonal matching pursuit under the restricted isometry property. Constructive Approximation, 45(1), 113–127.MathSciNetCrossRefMATHGoogle Scholar
  10. Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52, 1289–1306.MathSciNetCrossRefMATHGoogle Scholar
  11. Donoho, D. L., & Kutyniok, G. (2013). Microlocal analysis of the geometric separation problem. Communications on Pure and Applied Mathematics, 66, 1–47.MathSciNetCrossRefMATHGoogle Scholar
  12. Haupt, J., Bajwa, W. U., Raz, G., & Nowak, R. (2010). Toeplitz compressed sensing matrices with applications to sparse channel estimation. IEEE Transactions on Information Theory, 56(11), 5862–5875.MathSciNetCrossRefMATHGoogle Scholar
  13. King, E. J., Kutyniok, G., & Zhuang, X. (2011). Analysis of data separation and recovery problems using clustered sparsity. In SPIE proceedings: Wavelets and sparsity XIV, Vol. 8138Google Scholar
  14. King, E. J., Kutyniok, G., & Zhuang, X. (2012). Analysis of inpainting via clustered sparsity and microlocal analysis. Journal of Mathematical Imaging and Vision, 48(2), 205–234.MathSciNetCrossRefMATHGoogle Scholar
  15. Krahmer, F., Mendelson, S., & Rauhut, H. (2014). Suprema of chaos processes and the restricted isometry property. Communications on Pure and Applied Mathematics, 67(11), 1877–1904.MathSciNetCrossRefMATHGoogle Scholar
  16. Kunis, S., & Rauhut, H. (2008). Random sampling of sparse trigonometric polynomials II-orthogonal matching pursuit versus basis pursuit. Foundations of Computational Mathematics, 8, 737–763.MathSciNetCrossRefMATHGoogle Scholar
  17. Lin, J. H., & Li, S. (2013). Nonuniform support recovery from noisy random measurements by orthogonal matching pursuit. Journal of Approximation Theory, 165, 20–40.MathSciNetCrossRefMATHGoogle Scholar
  18. Mendelson, S., Pajor, A., & Tomczak-Jaegermann, N. (2011). Uniform uncertainty principle for Bernoulli and subgaussian ensembles. Constructive Approximation, 28, 277–289.MathSciNetCrossRefMATHGoogle Scholar
  19. Mo, Q. (2015). A sharp restricted isometry constant bound of orthogonal matching pursuit. arXiv:1501.01708.
  20. Mo, Q., & Shen, Y. (2012). A remark on the restricted isometry property in orthogonal matching pursuit. IEEE Transactions on Information Theory, 58(6), 3654–3656.MathSciNetCrossRefMATHGoogle Scholar
  21. Pati, Y. C., Rezaiifar, R., & Krishnaprasad, P. S. (1993). Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In Proceedings of 27th annual asilomar conference on signals, systems, and computers (Vol. 1, pp. 40–44). IEEE, Pacific Grove, CA.Google Scholar
  22. Pereira, P. M., Lovisolo, L., da Silva, E. A. B., & De Campos, M. L. R. (2014). On the design of maximally incoherent sensing matrices for compressed sensing using orthogonal bases and its extension for biorthogonal bases case. Digital Signal Processing, 27, 12–22.CrossRefGoogle Scholar
  23. Rauhut, H. (2009). Circulant and Toeplitz matrices in compressed sensing. Mathematics. arXiv:0902.4394.
  24. Romberg, J. (2008). Compressive sensing by random convolution. Siam Journal on Imaging Sciences, 2(4), 1098–1128.MathSciNetCrossRefMATHGoogle Scholar
  25. Shen, Y., & Li, S. (2015). Sparse signals recovery from noisy measurements by orthogonal matching pursuit. Inverse Problems and Imaging, 9(1), 231–238.MathSciNetCrossRefMATHGoogle Scholar
  26. Tropp, J. A. (2004). Greed is good: Algorithmic results for sparse approximation. IEEE Transactions on Information Theory, 50(10), 2231–2242.MathSciNetCrossRefMATHGoogle Scholar
  27. Tropp, J. A. (2012). User-friendly tail bounds for sums of random matrices. Foundations of Computational Mathematics, 12(4), 389–434.MathSciNetCrossRefMATHGoogle Scholar
  28. Tropp, J. A., & Gilbert, A. C. (2007). Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 53(12), 4655–4666.MathSciNetCrossRefMATHGoogle Scholar
  29. Wen, J., Zhou, Z., Wang, J., Tang, X., & Mo, Q. (2017). A sharp condition for exact support recovery with orthogonal matching pursuit. IEEE Transactions on Signal Processing, 65(6), 1370–1382.Google Scholar
  30. Xu, Z. (2012). A remark about orthogonal matching pursuit algorithm. Advances in Adaptive Data Analysis, 4(4), 1250026.Google Scholar
  31. Zhang, T. (2011). Sparse recovery with orthogonal matching pursuit under RIP. IEEE Transactions on Information Theory, 57(9), 6215–6221.MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of MathematicsZhejiang Sci–Tech UniversityHangzhouChina
  2. 2.Nanhu CollegeJiaxing UniversityJiaxingChina

Personalised recommendations