Abstract
Empirical Bayes methods are often thought of as a bridge between classical and Bayesian inference. In fact, in the literature the term empirical Bayes is used in quite diverse contexts and with different motivations. In this article, we provide a brief overview of empirical Bayes methods highlighting their scopes and meanings in different problems. We focus on recent results about merging of Bayes and empirical Bayes posterior distributions that regard popular, but otherwise debatable, empirical Bayes procedures as computationally convenient approximations of Bayesian solutions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Antoniak, C.E.: Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Stat. 2, 1152–1174 (1974)
Belitser, E., Enikeeva, F.: Empirical Bayesian test of the smoothness. Math. Methods Stat. 17, 1–18 (2008)
Belitser, E., Levit, B.: On the empirical Bayes approach to adaptive filtering. Math. Methods Stat. 12, 131–154 (2003)
Berry, D.A., Christensen, R.: Empirical Bayes estimation of a binomial parameter via mixtures of Dirichlet processes. Ann. Stat. 7, 558–568 (1979)
Clyde, M.A., George, E.I.: Flexible empirical Bayes estimation for wavelets. J. R. Stat. Soc. Ser. B 62, 681–698 (2000)
Copas, J.B.: Compound decisions and empirical Bayes (with discussion). J. R. Stat. Soc. Ser. B 31, 397–425 (1969)
Cui, W., George, E.I.: Empirical Bayes vs. fully Bayes variable selection. J. Stat. Plann. Inference 138, 888–900 (2008)
Deely, J.J., Lindley, D.V.: Bayes empirical Bayes. J. Am. Stat. Assoc. 76, 833–841 (1981)
Diaconis, P., Freedman, D.: On the consistency of Bayes estimates. Ann. Stat. 14, 1–26 (1986)
Efron, B.: Large-scale inference. Empirical Bayes methods for estimation, testing, and prediction. Cambridge University Press, Cambridge (2010)
Efron, B., Morris, C.: Limiting the risk of Bayes and empirical Bayes estimators. II. The empirical Bayes case. J. Am. Stat. Assoc. 67, 130–139 (1972a)
Efron, B., Morris, C.: Empirical Bayes on vector observations: an extension of Stein’s method. Biometrika 59, 335–347 (1972b)
Efron, B., Morris, C.: Stein’s estimation rule and its competitors-an empirical Bayes approach. J. Am. Stat. Assoc. 68, 117–130 (1973a)
Efron, B., Morris, C.: Combining possibly related estimation problems. (With discussion by Lindley, D.V., Copas, J.B., Dickey, J.M., Dawid, A.P., Smith, A.F.M., Birnbaum, A., Bartlett, M.S., Wilkinson, G.N., Nelder, J.A., Stein, C., Leonard, T., Barnard, G.A., Plackett, R.L.). J. R. Stat. Soc. Ser. B 35, 379–421 (1973b)
Efron, B., Morris, C.N.: Data analysis using Stein’s estimator and its generalizations. J. Am. Stat. Assoc. 70, 311–319 (1973c)
Favaro, S., Lijoi, A., Mena, R.H., Prünster, I.: Bayesian nonparametric inference for species variety with a two parameter Poisson–Dirichlet process prior. J. R. Stat. Soc. Ser. B 71, 993–1008 (2009)
Fisher, R.A., Corbet, A.S., Williams, C.B.: The relation between the number of species and the number of individuals in a random sample of an animal population. J. Anim. Ecol. 12, 42–58 (1943)
Forcina, A.: Gini’s contributions to the theory of inference. Int. Stat. Rev. 50, 65–70 (1982)
George, E.I., Foster, D.P.: Calibration and empirical Bayes variable selection. Biometrika 87, 731–747 (2000)
Ghosh, J.K., Ramamoorthi, R.V.: Bayesian nonparametrics. Springer, New York (2003)
Gini, C.: Considerazioni sulla probabilità a posteriori e applicazioni al rapporto dei sessi nelle nascite umane. Studi Economico-Giuridici. Università di Cagliari. III. Reprinted in Metron, vol. 15, pp. 133–172 (1911)
Good, I.J.: Breakthroughs in statistics: foundations and basic theory. In: Johnson, N.L., Kotz, S. (eds.) Introduction to Robbins (1992) An empirical Bayes approach to statistics, pp. 379–387. Springer, Berlin (1995)
James, W., Stein, C.: Estimation with quadratic loss. In: Proceedings of Fourth Berkeley Symposium on Mathematics Statistics and Probability, vol. 1, pp. 361–379. University of California Press, California (1961)
Lehmann, E.L., Casella, G.: Theory of point estimation, 2nd edn. Springer, New York (1998)
Liang, F., Paulo, R., Molina, G., Clyde, M.A., Berger, J.O.: Mixtures of \(g\)-priors for Bayesian variable selection. J. Am. Stat. Assoc. 103, 410–423 (2008)
Liu, J.S.: Nonparametric hierarchical Bayes via sequential imputation. Ann. Stat. 24, 911–930 (1996)
Maritz, J.S., Lwin, T.: Empirical Bayes methods, 2nd edn. Chapman and Hall, London (1989)
McAuliffe, J.D., Blei, D.M., Jordan, M.I.: Nonparametric empirical Bayes for the Dirichlet process mixture model. Stat. Comput. 16, 5–14 (2006)
Morris, C.N.: Parametric empirical Bayes inference: theory and applications. J. Am. Stat. Assoc. 78, 47–55 (1983)
Petrone, S., Rousseau, J., Scricciolo, C.: Bayes and empirical Bayes: do they merge? Biometrika 101(2), 285–302 (2014)
Robbins, H.: An empirical Bayes approach to statistics. In: Proceedings of Third Berkeley Symposium on Mathematics, Statistics and Probability, vol. 1, pp. 157–163. University of California Press, California (1956)
Scott, J.G., Berger, J.O.: Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. Ann. Stat. 38, 2587–2619 (2010)
Stein, C.: Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In: Proceedings of Third Berkeley Symposium on Mathematics, Statistics and Probability, vol. 1, pp. 197–206. University of California Press, California (1956)
Szabó, B.T., van der Vaart, A.W., van Zanten, J.H.: Empirical Bayes scaling of Gaussian priors in the white noise model. Electron. J. Stat. 7, 991–1018 (2013)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Petrone, S., Rizzelli, S., Rousseau, J. et al. Empirical Bayes methods in classical and Bayesian inference. METRON 72, 201–215 (2014). https://doi.org/10.1007/s40300-014-0044-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40300-014-0044-1