The problem of optimally approximating a function with a linear expansion over a redundant dictionary of waveforms is NP-hard. The greedy matching pursuit algorithm and its orthogonalized variant produce suboptimal function expansions by iteratively choosing dictionary waveforms that best match the function’s structures. A matching pursuit provides a means of quickly computing compact, adaptive function approximations.
Numerical experiments show that the approximation errors from matching pursuits initially decrease rapidly, but the asymptotic decay rate of the errors is slow. We explain this behavior by showing that matching pursuits are chaotic, ergodic maps. The statistical properties of the approximation errors of a pursuit can be obtained from the invariant measure of the pursuit. We characterize these measures using group symmetries of dictionaries and by constructing a stochastic differential equation model.
We derive a notion of the coherence of a signal with respect to a dictionary from our characterization of the approximation errors of a pursuit. The dictionary elements slected during the initial iterations of a pursuit correspond to a function’s coherent structures. The tail of the expansion, on the other hand, corresponds to a noise which is characterized by the invariant measure of the pursuit map.
When using a suitable dictionary, the expansion of a function into its coherent structures yields a compact approximation. We demonstrate a denoising algorithm based on coherent function expansions.
Key words and phrasesMatching pursuit Adaptive approximations Greedy algorithms Denoising Overcomplete signal representation Time frequency analysis
Unable to display preview. Download preview PDF.
- 4.T. H. Cormen, C. E. Leiserson, R. L. Rivest (1991): Introduction to Algorithms. New York: McGraw-Hill.Google Scholar
- 5.I. Daubechies (1991): Ten Lectures on Wavelets. CBMS-NSF Series in Appl. Math. Philadelphia: SIAM.Google Scholar
- 6.G. Davis (1994): Adaptive Nonlinear Approximations. PhD dissertation, Department of Mathematics, New York University.Google Scholar
- 11.D. L. Donoho (1993):Wavelet shrinkage and W.V.D.: A 10-minute tour. In: Progress in Wavelet Analysis and Applications (Y. Meyer, S. Roques, eds.). Gif-sur-Yvette, France: Editions Frontieres, pp. 109–128.Google Scholar
- 12.C. W. Gardiner (1985): Handbook of Stochastic Methods. New York: Springer-Verlag.Google Scholar
- 18.Y. C. Pati, R. Rezaiifar, P. S. Krishnaprasad (1993):Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. Proceedings of the 27th Annual Asilomar Conference on Signals, Systems, and Computers, pp. 40–44. IEEE Computer Society Press.Google Scholar
- 20.M. Reed, B. Simon (1972): Methods of Modern Mathematical Statistics, Vol. 1. New York: Academic Press.Google Scholar
- 21.N. Saito (1994):Simultaneous noise suppression and signal compression using a library of orthonormal bases and the minimum description length criterion. Wavelets in Geophysics (Efi Foufoula-Georgiou, Praveen Kumar, eds.). Academic Press, San Diego.Google Scholar
- 24.Z. Zhang (1993): Matching Pursuit. PhD dissertation, New York University.Google Scholar