Advertisement

Maximum a posteriori estimators as a limit of Bayes estimators

  • Robert Bassett
  • Julio Deride
Full Length Paper Series B

Abstract

Maximum a posteriori and Bayes estimators are two common methods of point estimation in Bayesian statistics. It is commonly accepted that maximum a posteriori estimators are a limiting case of Bayes estimators with 0–1 loss. In this paper, we provide a counterexample which shows that in general this claim is false. We then correct the claim that by providing a level-set condition for posterior densities such that the result holds. Since both estimators are defined in terms of optimization problems, the tools of variational analysis find a natural application to Bayesian point estimation.

Mathematics Subject Classification

62C10 62F10 62F15 65K10 

Notes

Acknowledgements

Both authors express their gratitude to Roger J.-B. Wets for his guidance and supervision. This paper is dedicated to him, in honor of his 80th birthday.

References

  1. 1.
    Attouch, H., Wets, R.J.-B.: Epigraphical Processes: Laws of Large Numbers for Random LSC Functions. Dép. des Sciences Mathématiques, Paris (1994)zbMATHGoogle Scholar
  2. 2.
    Beer, G.: Topologies on Closed and Closed Convex Sets. Mathematics and Its Applications. Springer, New York (1993)CrossRefzbMATHGoogle Scholar
  3. 3.
    Chan, S.-O., Diakonikolas, I., Servedio, R.A., Sun, X.: Efficient density estimation via piecewise polynomial approximation. In: Proceedings of the 46th Annual ACM Symposium on Theory of Computing, pp. 604–613. ACM (2014)Google Scholar
  4. 4.
    Dümbgen, L., Rufibach, K., et al.: Maximum likelihood estimation of a log-concave density and its distribution function: basic properties and uniform consistency. Bernoulli 15(1), 40–68 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Dupacová, J., Wets, R.: Asymptotic behavior of statistical estimators and of optimal solutions of stochastic optimization problems. Ann. Stat. 16, 1517–1549 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Figueiredo, M.A.T.: Lecture Notes on Bayesian Estimation and Classification. Instituto de Telecomunicacoes-Instituto Superior Tecnico, Lisboa (2004)Google Scholar
  7. 7.
    Friedman, J., Hastie, T., Tibshirani, R.: The Elements of Statistical Learning, vol. 1. Springer, New York (2001)zbMATHGoogle Scholar
  8. 8.
    Geweke, J.: Contemporary Bayesian Econometrics and Statistics, vol. 537. Wiley, New York (2005)CrossRefzbMATHGoogle Scholar
  9. 9.
    Geyer, C.J.: On the asymptotics of constrained m-estimation. Ann. Stat. 22, 1993–2010 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Hoegele, W., Loeschel, R., Dobler, B., Koelbl, O., Zygmanski, P.: Bayesian estimation applied to stochastic localization with constraints due to interfaces and boundaries. Math. Probl. Eng. 2013, 5–6 (2013)Google Scholar
  11. 11.
    King, A.J., Rockafellar, R.T.: Asymptotic theory for solutions in statistical estimation and stochastic programming. Math. Oper. Res. 18(1), 148–162 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    King, A.J., Wets, R.J.B.: Epi-consistency of convex stochastic programs. Stoch. Stoch. Rep. 34(1–2), 83–92 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Knight, K: Epi-convergence in distribution and stochastic equi-semicontinuity. Unpublished manuscript, 37 (1999)Google Scholar
  14. 14.
    Knight, K.: Limiting distributions of linear programming estimators. Extremes 4(2), 87–103 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Knight, K., Fu, W.: Asymptotics for lasso-type estimators. Ann. Stat. 28, 1356–1378 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Lee, P.M.: Bayesian Statistics: An Introduction. Wiley, New York (2012)zbMATHGoogle Scholar
  17. 17.
    Pflug, G.C.: Asymptotic Dominance and Confidence for Solutions of Stochastic Programs. International Institute for Applied Systems Analysis, Laxenburg (1991)Google Scholar
  18. 18.
    Pflug, G.C.: Asymptotic stochastic programs. Math. Oper. Res. 20(4), 769–789 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Robert, C.: The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation. Springer, New York (2007)zbMATHGoogle Scholar
  20. 20.
    Rockafellar, R.T., Wets, R.J.B.: Variational Analysis. Grundlehren der mathematischen Wissenschaften. Springer, Berlin (2009)Google Scholar
  21. 21.
    Royset, J.O., Wets, R.J.B.: Nonparametric density estimation via exponential epi-eplines: fusion of soft and hard information. Technical report (2013)Google Scholar
  22. 22.
    Rufibach, K.: Computing maximum likelihood estimators of a log-concave density function. J. Stat. Comput. Simul. 77(7), 561–574 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Salinetti, G., Wets, R.J.-B.: On the convergence in distribution of measurable multifunctions (random sets) normal integrands, stochastic processes and stochastic infima. Math. Oper. Res. 11(3), 385–419 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Shapiro, A.: Asymptotic analysis of stochastic programs. Ann. Oper. Res. 30(1), 169–186 (1991)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2018

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of California DavisDavisUSA

Personalised recommendations