MAP versus MMSE Estimation
So far we kept the description of the pursuit algorithms on a deterministic level, as an intuitive optimization procedure. We mentioned in Chapter 9 that these algorithms correspond to an approximation of the Maximum-A’posteriori-Probability (MAP) estimator, but this connection was not explicitly derived. In this chapter we make this claim exact by defining the quest for sparse representations as an estimation task. As we shall see, this calls for a clear and formal definition of the stochastic model assumed to generate the sparse representation vector. A benefit of such treatment is an ability to derive the Minimum-Mean-Squared-Error (MMSE) estimator as well, and this in turn leads to the need to approximate it. These and more are the topics we cover in this chapter.
KeywordsSparse Representation Multivariate Gaussian Distribution Estimation Task Sparse Vector MMSE Estimation
Unable to display preview. Download preview PDF.
- 2.A. Antoniadis, J. Bigot, and T. Sapatinas, Wavelet estimators in nonparametric regression: a comparative simulation study, J. Stat. Software, 6(6):1–83, 2001.Google Scholar
- 3.M. Clyde and E.I. George, Empirical Bayes estimation in wavelet nonparametric regression. In Bayesian Inference in Wavelet Based Models, P. Muller and B. Vidakovic (Eds.), Lect. Notes Statist., 141:309–322, New York, Springer-Verlag, 1998.Google Scholar
- 7.J. Turek, I. Yavneh, M. Protter, and M. Elad, On MMSE and MAP denoising under sparse representation modeling over a unitary dictionary, submitted to Applied Computational Harmonic Analysis.Google Scholar
- 11.M. Protter, I. Yavneh and M. Elad, Closed-form MMSE for denoising signals under sparse-representation modelling, The IEEE 25-th Convention of Electrical and Electronics Engineers in Israel, Eilat Israel, December 3–5, 2008.Google Scholar
- 12.M. Protter, I. Yavneh and M. Elad, Closed-Form MMSE estimation for signal denoising under sparse representation modelling over a unitary dictionary, submitted to IEEE Trans. on Signal Processing.Google Scholar
- 13.E.P. Simoncelli and E.H. Adelson, Noise removal via Bayesian wavelet coring, in Proc. ICIP, Laussanne, Switzerland, pp. 379–382, September 1996.Google Scholar
- 14.P. Schnitter, L. C. Potter, and J. Ziniel, Fast Bayesian matching pursuit, Proc. Workshop on Information Theory and Applications (ITA), (La Jolla, CA), Jan. 2008.Google Scholar
- 15.P. Schintter, L.C. Potter, and J. Ziniel, Fast Bayesian matching pursuit: Model uncertainty and parameter estimation for sparse linear models, submitted to IEEE Transactions on Signal Processing.Google Scholar