Abstract
This paper provides new results on computing simultaneous sparse approximations of multichannel signals over redundant dictionaries using two greedy algorithms. The first one, p-thresholding, selects the S atoms that have the largest p-correlation while the second one, p-simultaneous matching pursuit (p-SOMP), is a generalisation of an algorithm studied by Tropp in (Signal Process. 86:572–588, 2006). We first provide exact recovery conditions as well as worst case analyses of all algorithms. The results, expressed using the standard cumulative coherence, are very reminiscent of the single channel case and, in particular, impose stringent restrictions on the dictionary.
We unlock the situation by performing an average case analysis of both algorithms. First, we set up a general probabilistic signal model in which the coefficients of the atoms are drawn at random from the standard Gaussian distribution. Second, we show that under this model, and with mild conditions on the coherence, the probability that p-thresholding and p-SOMP fail to recover the correct components is overwhelmingly small and gets smaller as the number of channels increases.
Furthermore, we analyse the influence of selecting the set of correct atoms at random. We show that, if the dictionary satisfies a uniform uncertainty principle (Candes and Tao, IEEE Trans. Inf. Theory, 52(12):5406–5425, 2006), the probability that simultaneous OMP fails to recover any sufficiently sparse set of atoms gets increasingly smaller as the number of channels increases.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. (to appear)
Baron, D., Duarte, M., Sarvotham, S., Wakin, M., Baraniuk, R.: An information-theoretic approach to distributed compressed sensing. In: Proc. 45rd Conference on Communication, Control, and Computing (2005)
Baron, D., Wakin, M., Duarte, M., Sarvotham, S., Baraniuk, R.: Distributed compressed sensing. Preprint (2005)
Candès, E., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)
Candes, E., Tao, T.: Near optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006)
Chen, J., Huo, X.: Sparse representations for multiple measurement vectors (MMV) in an over-complete dictionary. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP-2005) (2005)
Chen, J., Huo, X.: Theoretical results on sparse representations of multiple measurement vectors. IEEE Trans. Signal Process. 54(12), 4634–4643 (2006)
DeVore, R., Lorentz, G.: Constructive Approximation. Springer, Berlin (1993)
Donoho, D., Elad, M.: Maximal sparsity representation via l 1 minimization. Proc. Nat. Acad. Sci. 100(4), 369–388 (2003)
Donoho, D., Elad, M., Temlyakov, V.: Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inf. Theory 52(1), 6–18 (2006)
Donoho, D., Vetterli, M., DeVore, R.A., Daubechies, I.: Data compression and harmonic analysis. IEEE Trans. Inf. Theory 44, 391–432 (1998)
Gribonval, R., Nielsen, M.: Beyond sparsity: Recovering structured representations by l1 minimization and greedy algorithms. Publication interne 1684, IRISA, Rennes (2005)
Gribonval, R., Nielsen, M., Vandergheynst, P.: Towards an adaptive computational strategy for sparse signal approximation. Preprint of the Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA) (2006)
Grimmett, G.R., Stirzaker, D.R.: Probability and Random Processes. Oxford University Press, London (2001)
Ledoux, M.: The Concentration of Measure Phenomenon. Am. Math. Soc., Providence (2001)
Ledoux, M., Talagrand, M.: Probability in Banach Spaces. Isoperimetry and Processes. Springer, Berlin (1991)
Luo, Z., Gaspar, M., Liu, J., Swami, A.: Distributed signal processing in sensor networks. IEEE Signal Process. Mag. 23(4), 14–15 (2006)
Rauhut, H.: Stability results for random sampling of sparse trigonometric polynomials. IEEE Trans. Inf. Theory (to appear)
Rauhut, H., Schnass, K., Vandergheynst, P.: Compressed sensing and redundant dictionaries. IEEE Trans. Inf. Theory 54(5), 2210–2219 (2008)
Rudelson, M., Vershynin, R.: Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements. In: Proc. CISS 2006 (40th Annual Conference on Information Sciences and Systems) (2006)
Schnass, K., Vandergheynst, P.: Average Performance Analysis for Thresholding. IEEE Signal Process. Lett. 14(11), 828–831 (2007)
Schnass, K., Vandergheynst, P.: Dictionary preconditioning for greedy algorithms. IEEE Trans. Signal Process. 56(5), 1994–2002 (2008)
Taubman, D., Marcellin, W.: JPEG2000: Image Compression Fundamentals, Standards, and Practice. Springer, Berlin (2002)
Tropp, J.: Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50(10), 2231–2242 (2004)
Tropp, J.: Topics in sparse approximation. Ph.D. Thesis, University of Texas at Austin (2004)
Tropp, J.: Just relax: Convex programming methods for subset selection and sparse approximation. IEEE Trans. Inf. Theory 51(3), 1030–1051 (2006)
Tropp, J.: On the conditioning of random subdictionaries. Appl. Comput. Harmon. Anal. 25, 1–24 (2008)
Tropp, J., Gilbert, A., Strauss, M.: Algorithms for simultaneous sparse approximations. Part I: Greedy pursuit. Signal Process. 86, 572–588 (2006). Special issue “Sparse approximations in signal and image processing”
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Anna Gilbert.
Rights and permissions
About this article
Cite this article
Gribonval, R., Rauhut, H., Schnass, K. et al. Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms. J Fourier Anal Appl 14, 655–687 (2008). https://doi.org/10.1007/s00041-008-9044-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00041-008-9044-y