Journal of Fourier Analysis and Applications

, Volume 14, Issue 5–6, pp 655–687 | Cite as

Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms

  • Rémi Gribonval
  • Holger Rauhut
  • Karin SchnassEmail author
  • Pierre Vandergheynst


This paper provides new results on computing simultaneous sparse approximations of multichannel signals over redundant dictionaries using two greedy algorithms. The first one, p-thresholding, selects the S atoms that have the largest p-correlation while the second one, p-simultaneous matching pursuit (p-SOMP), is a generalisation of an algorithm studied by Tropp in (Signal Process. 86:572–588, 2006). We first provide exact recovery conditions as well as worst case analyses of all algorithms. The results, expressed using the standard cumulative coherence, are very reminiscent of the single channel case and, in particular, impose stringent restrictions on the dictionary.

We unlock the situation by performing an average case analysis of both algorithms. First, we set up a general probabilistic signal model in which the coefficients of the atoms are drawn at random from the standard Gaussian distribution. Second, we show that under this model, and with mild conditions on the coherence, the probability that p-thresholding and p-SOMP fail to recover the correct components is overwhelmingly small and gets smaller as the number of channels increases.

Furthermore, we analyse the influence of selecting the set of correct atoms at random. We show that, if the dictionary satisfies a uniform uncertainty principle (Candes and Tao, IEEE Trans. Inf. Theory, 52(12):5406–5425, 2006), the probability that simultaneous OMP fails to recover any sufficiently sparse set of atoms gets increasingly smaller as the number of channels increases.


Greedy algorithms OMP Thresholding Multi-channel Average analysis 

Mathematics Subject Classification (2000)

41A28 41A46 60D05 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. (to appear) Google Scholar
  2. 2.
    Baron, D., Duarte, M., Sarvotham, S., Wakin, M., Baraniuk, R.: An information-theoretic approach to distributed compressed sensing. In: Proc. 45rd Conference on Communication, Control, and Computing (2005) Google Scholar
  3. 3.
    Baron, D., Wakin, M., Duarte, M., Sarvotham, S., Baraniuk, R.: Distributed compressed sensing. Preprint (2005) Google Scholar
  4. 4.
    Candès, E., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006) zbMATHCrossRefGoogle Scholar
  5. 5.
    Candes, E., Tao, T.: Near optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006) CrossRefMathSciNetGoogle Scholar
  6. 6.
    Chen, J., Huo, X.: Sparse representations for multiple measurement vectors (MMV) in an over-complete dictionary. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP-2005) (2005) Google Scholar
  7. 7.
    Chen, J., Huo, X.: Theoretical results on sparse representations of multiple measurement vectors. IEEE Trans. Signal Process. 54(12), 4634–4643 (2006) CrossRefGoogle Scholar
  8. 8.
    DeVore, R., Lorentz, G.: Constructive Approximation. Springer, Berlin (1993) zbMATHGoogle Scholar
  9. 9.
    Donoho, D., Elad, M.: Maximal sparsity representation via l 1 minimization. Proc. Nat. Acad. Sci. 100(4), 369–388 (2003) MathSciNetGoogle Scholar
  10. 10.
    Donoho, D., Elad, M., Temlyakov, V.: Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inf. Theory 52(1), 6–18 (2006) CrossRefMathSciNetGoogle Scholar
  11. 11.
    Donoho, D., Vetterli, M., DeVore, R.A., Daubechies, I.: Data compression and harmonic analysis. IEEE Trans. Inf. Theory 44, 391–432 (1998) CrossRefMathSciNetGoogle Scholar
  12. 12.
    Gribonval, R., Nielsen, M.: Beyond sparsity: Recovering structured representations by l1 minimization and greedy algorithms. Publication interne 1684, IRISA, Rennes (2005) Google Scholar
  13. 13.
    Gribonval, R., Nielsen, M., Vandergheynst, P.: Towards an adaptive computational strategy for sparse signal approximation. Preprint of the Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA) (2006) Google Scholar
  14. 14.
    Grimmett, G.R., Stirzaker, D.R.: Probability and Random Processes. Oxford University Press, London (2001) Google Scholar
  15. 15.
    Ledoux, M.: The Concentration of Measure Phenomenon. Am. Math. Soc., Providence (2001) zbMATHGoogle Scholar
  16. 16.
    Ledoux, M., Talagrand, M.: Probability in Banach Spaces. Isoperimetry and Processes. Springer, Berlin (1991) zbMATHGoogle Scholar
  17. 17.
    Luo, Z., Gaspar, M., Liu, J., Swami, A.: Distributed signal processing in sensor networks. IEEE Signal Process. Mag. 23(4), 14–15 (2006) CrossRefGoogle Scholar
  18. 18.
    Rauhut, H.: Stability results for random sampling of sparse trigonometric polynomials. IEEE Trans. Inf. Theory (to appear) Google Scholar
  19. 19.
    Rauhut, H., Schnass, K., Vandergheynst, P.: Compressed sensing and redundant dictionaries. IEEE Trans. Inf. Theory 54(5), 2210–2219 (2008) CrossRefMathSciNetGoogle Scholar
  20. 20.
    Rudelson, M., Vershynin, R.: Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements. In: Proc. CISS 2006 (40th Annual Conference on Information Sciences and Systems) (2006) Google Scholar
  21. 21.
    Schnass, K., Vandergheynst, P.: Average Performance Analysis for Thresholding. IEEE Signal Process. Lett. 14(11), 828–831 (2007) CrossRefGoogle Scholar
  22. 22.
    Schnass, K., Vandergheynst, P.: Dictionary preconditioning for greedy algorithms. IEEE Trans. Signal Process. 56(5), 1994–2002 (2008) CrossRefMathSciNetGoogle Scholar
  23. 23.
    Taubman, D., Marcellin, W.: JPEG2000: Image Compression Fundamentals, Standards, and Practice. Springer, Berlin (2002) Google Scholar
  24. 24.
    Tropp, J.: Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50(10), 2231–2242 (2004) CrossRefMathSciNetGoogle Scholar
  25. 25.
    Tropp, J.: Topics in sparse approximation. Ph.D. Thesis, University of Texas at Austin (2004) Google Scholar
  26. 26.
    Tropp, J.: Just relax: Convex programming methods for subset selection and sparse approximation. IEEE Trans. Inf. Theory 51(3), 1030–1051 (2006) CrossRefMathSciNetGoogle Scholar
  27. 27.
    Tropp, J.: On the conditioning of random subdictionaries. Appl. Comput. Harmon. Anal. 25, 1–24 (2008) zbMATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    Tropp, J., Gilbert, A., Strauss, M.: Algorithms for simultaneous sparse approximations. Part I: Greedy pursuit. Signal Process. 86, 572–588 (2006). Special issue “Sparse approximations in signal and image processing” CrossRefGoogle Scholar

Copyright information

© Birkhäuser Boston 2008

Authors and Affiliations

  • Rémi Gribonval
    • 1
  • Holger Rauhut
    • 2
  • Karin Schnass
    • 3
    Email author
  • Pierre Vandergheynst
    • 3
  1. 1.Universitè Rennes I, IRISARennes CedexFrance
  2. 2.Hausdorff Center for MathematicsUniversity of BonnBonnGermany
  3. 3.Signal Processing Laboratories, LTS2Ecole Polytechnique Fédérale de Lausanne (EPFL)LausanneSwitzerland

Personalised recommendations