Consistency of Bayes Estimates for Nonparametric Regression: A Review

  • P. Diaconis
  • D. A. Freedman


This paper reviews some recent studies of frequentist properties of Bayes estimates. In nonparametric regression, natural priors can lead to inconsistent estimators; although in some problems, such priors do give consistent estimates.


Nonparametric Regression Predictive Probability Statistical Decision Theory Normal Regression Inconsistent Estimator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bernstein, S. (1934), Theory of Probability, GTTI, Moscow. (Russian).Google Scholar
  2. Breiman, L., Le Cam, L. & Schwartz, L. (1964), ‘Consistent estimates and zero-one sets’, Annals of Mathematical Statistics 35, 157–161.MathSciNetMATHCrossRefGoogle Scholar
  3. Cox, D. (1993), ‘An analysis of Bayesian inference for nonparametric regression’, Annals of Statistics 21, 903–923.MathSciNetMATHCrossRefGoogle Scholar
  4. de Finetti, B. (1959), La probabilità, la statistica, nei rapporti con l’induzione, secondo diversi punti di vista,Centro Internazionale Matematica Estivo Cremonese, Rome. English translation in de Finetti (1972).Google Scholar
  5. de Finetti, B. (1972), Probability, Induction, and Statistics, Wiley, New York.Google Scholar
  6. Diaconis, P. (1988), Bayesian numerical analysis, in S. S. Gupta & J. O. Berger, eds, ‘Statistical Decision Theory and Related Topics IV’, Vol. 1, pp. 163–177.CrossRefGoogle Scholar
  7. Diaconis, P. & Freedman, D. (1986), ‘On the consistency of Bayes estimates (with discussion)’, Annals of Statistics 14, 1–67.MathSciNetMATHCrossRefGoogle Scholar
  8. Diaconis, P. & Freedman, D. (1990), ‘On the uniform consistency of Bayes estimates for multinomial probabilities’, Annals of Statistics 18, 1317–1327.MathSciNetMATHCrossRefGoogle Scholar
  9. Diaconis, P. & Freedman, D. (1993a), ‘Nonparametric binary regression: a Bayesian approach’, Annals of Statistics 21, 2108–2137.MathSciNetMATHCrossRefGoogle Scholar
  10. Diaconis, P. & Freedman, D. (1993b), Nonparametric binary regression with random covariates, Technical Report 291, Department of Statistics, University of California, Berkeley. (To appear in Probability and Mathematical Statistics.). Google Scholar
  11. Diaconis, P. & Freedman, D. (1994), Consistency of Bayes estimates for nonparametric regression: normal theory, Technical Report 414, Department of Statistics, University of California, Berkeley.Google Scholar
  12. Doss, H. (1984), ‘Bayesian estimation in the symmetric location problem’, Zeitschrift fir Wahrscheinlichkeitstheorie and Verwandte Gebiete 68, 127–147.MathSciNetMATHCrossRefGoogle Scholar
  13. Doss, H. (1985a), ‘Bayesian nonparametric estimation of the median; part I: Computation of the estimates’, Annals of Statistics 13, 1432–1444.MathSciNetMATHCrossRefGoogle Scholar
  14. Doss, H. (1985b), ‘Bayesian nonparametric estimation of the median; part II: Asymptotic properties of the estimates’, Annals of Statistics 13,1445–1464.MathSciNetMATHCrossRefGoogle Scholar
  15. Ferguson, T. (1974), ‘Prior distributions on spaces of probability measures’, Annals of Statistics 2, 615–629.MathSciNetMATHCrossRefGoogle Scholar
  16. Freedman, D. (1963), ‘On the asymptotic behavior of Bayes estimates in the discrete case’, Annals of Mathematical Statistics 34, 1386–1403.MathSciNetCrossRefGoogle Scholar
  17. Ghosh, J. K., Sinha, B. K. & Joshi, S. N. (1982), Expansions for posterior probability and integrated Bayes risk, in S. S. Gupta & J. O. Berger, eds, ‘Statistical Decision Theory and Related Topics III’, Vol. 1, Academic Press, New York, pp. 403–456.Google Scholar
  18. Johnson, R. (1967), ‘An asymptotic expansion for posterior distributions’, Annals of Mathematical Statistics 38, 1899–1906.MathSciNetMATHCrossRefGoogle Scholar
  19. Johnson, R. (1970), ‘Asymptotic expansions associated with posterior distributions’, Annals of Mathematical Statistics 41, 851–864.MathSciNetMATHCrossRefGoogle Scholar
  20. Kimeldorf, G. & Wahba, G. (1970), ‘A correspondence between Bayesian estimation on stochastic processes and smoothing by splines’, Annals of Mathematical Statistics 41, 495–502.MathSciNetMATHCrossRefGoogle Scholar
  21. Kohn, R. & Ansley, C. (1987), ‘A new algorithm for spline smoothing and interpolation based on smoothing a stochastic process’, SIAM Journal on Scientific and Statistical Computing 8, 33–48.MathSciNetMATHCrossRefGoogle Scholar
  22. Laplace, P. S. (1774), ‘Memoire sur la probabilité des causes par les évènements’, Memoires de mathématique et de physique presentés a l’académie royale des sciences, par divers savants, et lûs dans ses assemblées. Reprinted in Laplace’s Oeuvres Complètes 8 27–65. English translation by S. Stigler (1986) Statistical Science 1 359–378.MathSciNetGoogle Scholar
  23. Le Cam, L. (1953), ‘On some asymptotic properties of maximum likelihood estimates and related Bayes estimates’, University of California Publications in Statistics 1, 277–330.Google Scholar
  24. Le Cam, L. (1958), ‘Les propriétés asymptotiques des solutions de Bayes’, Publications de l’Institut de Statistique de l’Université de Paris 7, 17–35.Google Scholar
  25. Le Cam, L. (1982), On the risk of Bayes estimates, in S. S. Gupta & J. O. Berger, eds, ‘Statistical Decision Theory and Related Topics III’, Vol. 2, Academic Press, New York, pp. 121–138.Google Scholar
  26. Le Cam, L. (1986), Asymptotic Methods in Statistical Decision Theory, Springer-Verlag, New York.MATHCrossRefGoogle Scholar
  27. Le Cam, L. (1990), ‘Maximum likelihood: an introduction’, International Statistical Review 58,153–172. MATHCrossRefGoogle Scholar
  28. Le Cam, L. & Yang, G. L. (1990), Asymptotics in Statistics: Some Basic Concepts, Springer-Verlag.MATHCrossRefGoogle Scholar
  29. Lindley, D. & Smith, A. (1972), ‘Bayes estimates for the linear model’, Journal of the Royal Statistical Society 67, 1–19. MathSciNetGoogle Scholar
  30. von Mises, R. (1964), Mathematical Theory of Probability and Statistics, Academic Press, New York. H. Geiringer, ed.Wahba, G. (1990), Spline Models for Observational Data, Society for Industrial and Applied Mathematics, Philadelphia.Google Scholar

Copyright information

© Springer Science+Business Media New York 1997

Authors and Affiliations

  • P. Diaconis
    • 1
  • D. A. Freedman
    • 2
  1. 1.Harvard UniversityUSA
  2. 2.University of California at BerkeleyBerkeleyUSA

Personalised recommendations