MAP versus MMSE Estimation

  • Michael Elad


So far we kept the description of the pursuit algorithms on a deterministic level, as an intuitive optimization procedure. We mentioned in Chapter 9 that these algorithms correspond to an approximation of the Maximum-A’posteriori-Probability (MAP) estimator, but this connection was not explicitly derived. In this chapter we make this claim exact by defining the quest for sparse representations as an estimation task. As we shall see, this calls for a clear and formal definition of the stochastic model assumed to generate the sparse representation vector. A benefit of such treatment is an ability to derive the Minimum-Mean-Squared-Error (MMSE) estimator as well, and this in turn leads to the need to approximate it. These and more are the topics we cover in this chapter.


Sparse Representation Multivariate Gaussian Distribution Estimation Task Sparse Vector MMSE Estimation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Further Reading

  1. 1.
    F. Abramovich, T. Sapatinas and B.W. Silverman, Wavelet thresholding via a Bayesian approach, J. R. Statist. Soc. B, 60:725–749, 1998.MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    A. Antoniadis, J. Bigot, and T. Sapatinas, Wavelet estimators in nonparametric regression: a comparative simulation study, J. Stat. Software, 6(6):1–83, 2001.Google Scholar
  3. 3.
    M. Clyde and E.I. George, Empirical Bayes estimation in wavelet nonparametric regression. In Bayesian Inference in Wavelet Based Models, P. Muller and B. Vidakovic (Eds.), Lect. Notes Statist., 141:309–322, New York, Springer-Verlag, 1998.Google Scholar
  4. 4.
    M. Clyde and E.I. George, Flexible empirical Bayes estimation for wavelets, J. R. Statist. Soc. B, 62:681–698, 2000.MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    M. Clyde, G. Parmigiani and B. Vidakovic, Multiple shrinkage and subset selection in wavelets, Biometrika, 85:391–401, 1998.MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    M. Elad and I. Yavneh, A weighted average of sparse representations is better than the sparsest one alone, IEEE Transactions on Information Theory, 55(10):4701–4714, October 2009.CrossRefMathSciNetGoogle Scholar
  7. 7.
    J. Turek, I. Yavneh, M. Protter, and M. Elad, On MMSE and MAP denoising under sparse representation modeling over a unitary dictionary, submitted to Applied Computational Harmonic Analysis.Google Scholar
  8. 8.
    E. Larsson and Y. Selen, Linear regression with a sparse parameter vector, IEEE Transactions on Signal Processing, 55:451–460, 2007.CrossRefMathSciNetGoogle Scholar
  9. 9.
    S. Mallat and Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Trans. on Signal Processing, 41(12):3397–3415, 1993.MATHCrossRefGoogle Scholar
  10. 10.
    P. Moulin and J. Liu, Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors, IEEE Trans. Inf. Theory, 45(3):909–919, April 1999.MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    M. Protter, I. Yavneh and M. Elad, Closed-form MMSE for denoising signals under sparse-representation modelling, The IEEE 25-th Convention of Electrical and Electronics Engineers in Israel, Eilat Israel, December 3–5, 2008.Google Scholar
  12. 12.
    M. Protter, I. Yavneh and M. Elad, Closed-Form MMSE estimation for signal denoising under sparse representation modelling over a unitary dictionary, submitted to IEEE Trans. on Signal Processing.Google Scholar
  13. 13.
    E.P. Simoncelli and E.H. Adelson, Noise removal via Bayesian wavelet coring, in Proc. ICIP, Laussanne, Switzerland, pp. 379–382, September 1996.Google Scholar
  14. 14.
    P. Schnitter, L. C. Potter, and J. Ziniel, Fast Bayesian matching pursuit, Proc. Workshop on Information Theory and Applications (ITA), (La Jolla, CA), Jan. 2008.Google Scholar
  15. 15.
    P. Schintter, L.C. Potter, and J. Ziniel, Fast Bayesian matching pursuit: Model uncertainty and parameter estimation for sparse linear models, submitted to IEEE Transactions on Signal Processing.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Computer Science DepartmentThe Technion – Israel Institute of TechnologyHaifaIsrael

Personalised recommendations