Skip to main content

Empirical Bayes methods in classical and Bayesian inference

METRON volume 72, pages 201–215 (2014)Cite this article

Abstract

Empirical Bayes methods are often thought of as a bridge between classical and Bayesian inference. In fact, in the literature the term empirical Bayes is used in quite diverse contexts and with different motivations. In this article, we provide a brief overview of empirical Bayes methods highlighting their scopes and meanings in different problems. We focus on recent results about merging of Bayes and empirical Bayes posterior distributions that regard popular, but otherwise debatable, empirical Bayes procedures as computationally convenient approximations of Bayesian solutions.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2

References

  1. Antoniak, C.E.: Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Stat. 2, 1152–1174 (1974)

    Article  MATH  MathSciNet  Google Scholar 

  2. Belitser, E., Enikeeva, F.: Empirical Bayesian test of the smoothness. Math. Methods Stat. 17, 1–18 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  3. Belitser, E., Levit, B.: On the empirical Bayes approach to adaptive filtering. Math. Methods Stat. 12, 131–154 (2003)

    MathSciNet  Google Scholar 

  4. Berry, D.A., Christensen, R.: Empirical Bayes estimation of a binomial parameter via mixtures of Dirichlet processes. Ann. Stat. 7, 558–568 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  5. Clyde, M.A., George, E.I.: Flexible empirical Bayes estimation for wavelets. J. R. Stat. Soc. Ser. B 62, 681–698 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  6. Copas, J.B.: Compound decisions and empirical Bayes (with discussion). J. R. Stat. Soc. Ser. B 31, 397–425 (1969)

    MATH  MathSciNet  Google Scholar 

  7. Cui, W., George, E.I.: Empirical Bayes vs. fully Bayes variable selection. J. Stat. Plann. Inference 138, 888–900 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  8. Deely, J.J., Lindley, D.V.: Bayes empirical Bayes. J. Am. Stat. Assoc. 76, 833–841 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  9. Diaconis, P., Freedman, D.: On the consistency of Bayes estimates. Ann. Stat. 14, 1–26 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  10. Efron, B.: Large-scale inference. Empirical Bayes methods for estimation, testing, and prediction. Cambridge University Press, Cambridge (2010)

    Book  MATH  Google Scholar 

  11. Efron, B., Morris, C.: Limiting the risk of Bayes and empirical Bayes estimators. II. The empirical Bayes case. J. Am. Stat. Assoc. 67, 130–139 (1972a)

    MATH  MathSciNet  Google Scholar 

  12. Efron, B., Morris, C.: Empirical Bayes on vector observations: an extension of Stein’s method. Biometrika 59, 335–347 (1972b)

    Article  MATH  MathSciNet  Google Scholar 

  13. Efron, B., Morris, C.: Stein’s estimation rule and its competitors-an empirical Bayes approach. J. Am. Stat. Assoc. 68, 117–130 (1973a)

    MATH  MathSciNet  Google Scholar 

  14. Efron, B., Morris, C.: Combining possibly related estimation problems. (With discussion by Lindley, D.V., Copas, J.B., Dickey, J.M., Dawid, A.P., Smith, A.F.M., Birnbaum, A., Bartlett, M.S., Wilkinson, G.N., Nelder, J.A., Stein, C., Leonard, T., Barnard, G.A., Plackett, R.L.). J. R. Stat. Soc. Ser. B 35, 379–421 (1973b)

  15. Efron, B., Morris, C.N.: Data analysis using Stein’s estimator and its generalizations. J. Am. Stat. Assoc. 70, 311–319 (1973c)

    Article  MathSciNet  Google Scholar 

  16. Favaro, S., Lijoi, A., Mena, R.H., Prünster, I.: Bayesian nonparametric inference for species variety with a two parameter Poisson–Dirichlet process prior. J. R. Stat. Soc. Ser. B 71, 993–1008 (2009)

    Article  Google Scholar 

  17. Fisher, R.A., Corbet, A.S., Williams, C.B.: The relation between the number of species and the number of individuals in a random sample of an animal population. J. Anim. Ecol. 12, 42–58 (1943)

    Article  Google Scholar 

  18. Forcina, A.: Gini’s contributions to the theory of inference. Int. Stat. Rev. 50, 65–70 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  19. George, E.I., Foster, D.P.: Calibration and empirical Bayes variable selection. Biometrika 87, 731–747 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  20. Ghosh, J.K., Ramamoorthi, R.V.: Bayesian nonparametrics. Springer, New York (2003)

    MATH  Google Scholar 

  21. Gini, C.: Considerazioni sulla probabilità a posteriori e applicazioni al rapporto dei sessi nelle nascite umane. Studi Economico-Giuridici. Università di Cagliari. III. Reprinted in Metron, vol. 15, pp. 133–172 (1911)

  22. Good, I.J.: Breakthroughs in statistics: foundations and basic theory. In: Johnson, N.L., Kotz, S. (eds.) Introduction to Robbins (1992) An empirical Bayes approach to statistics, pp. 379–387. Springer, Berlin (1995)

    Google Scholar 

  23. James, W., Stein, C.: Estimation with quadratic loss. In: Proceedings of Fourth Berkeley Symposium on Mathematics Statistics and Probability, vol. 1, pp. 361–379. University of California Press, California (1961)

  24. Lehmann, E.L., Casella, G.: Theory of point estimation, 2nd edn. Springer, New York (1998)

    MATH  Google Scholar 

  25. Liang, F., Paulo, R., Molina, G., Clyde, M.A., Berger, J.O.: Mixtures of \(g\)-priors for Bayesian variable selection. J. Am. Stat. Assoc. 103, 410–423 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  26. Liu, J.S.: Nonparametric hierarchical Bayes via sequential imputation. Ann. Stat. 24, 911–930 (1996)

    Article  MATH  Google Scholar 

  27. Maritz, J.S., Lwin, T.: Empirical Bayes methods, 2nd edn. Chapman and Hall, London (1989)

    MATH  Google Scholar 

  28. McAuliffe, J.D., Blei, D.M., Jordan, M.I.: Nonparametric empirical Bayes for the Dirichlet process mixture model. Stat. Comput. 16, 5–14 (2006)

    Article  MathSciNet  Google Scholar 

  29. Morris, C.N.: Parametric empirical Bayes inference: theory and applications. J. Am. Stat. Assoc. 78, 47–55 (1983)

    Article  MATH  Google Scholar 

  30. Petrone, S., Rousseau, J., Scricciolo, C.: Bayes and empirical Bayes: do they merge? Biometrika 101(2), 285–302 (2014)

  31. Robbins, H.: An empirical Bayes approach to statistics. In: Proceedings of Third Berkeley Symposium on Mathematics, Statistics and Probability, vol. 1, pp. 157–163. University of California Press, California (1956)

  32. Scott, J.G., Berger, J.O.: Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. Ann. Stat. 38, 2587–2619 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  33. Stein, C.: Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In: Proceedings of Third Berkeley Symposium on Mathematics, Statistics and Probability, vol. 1, pp. 197–206. University of California Press, California (1956)

  34. Szabó, B.T., van der Vaart, A.W., van Zanten, J.H.: Empirical Bayes scaling of Gaussian priors in the white noise model. Electron. J. Stat. 7, 991–1018 (2013)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Bocconi University, Milan, Italy

    Sonia Petrone, Stefano Rizzelli & Catia Scricciolo

  2. CREST-ENSAE and CEREMADE, Université Paris Dauphine, Paris, France

    Judith Rousseau

Authors
  1. Sonia Petrone
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefano Rizzelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Judith Rousseau
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Catia Scricciolo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sonia Petrone.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Petrone, S., Rizzelli, S., Rousseau, J. et al. Empirical Bayes methods in classical and Bayesian inference. METRON 72, 201–215 (2014). https://doi.org/10.1007/s40300-014-0044-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40300-014-0044-1

Keywords

  • Bayesian weak merging
  • Compound experiments
  • Frequentist strong merging
  • Hyper-parameter oracle value
  • Latent distribution
  • Maximum marginal likelihood estimation
  • Shrinkage estimation