Computing Nonparametric Hierarchical Models

  • Michael D. Escobar
  • Mike West
Part of the Lecture Notes in Statistics book series (LNS, volume 133)


Bayesian models involving Dirichlet process mixtures are at the heart of the modern nonparametric Bayesian movement. Much of the rapid development of these models in the last decade has been a direct result of advances in simulation-based computational methods. Some of the very early work in this area, circa 1988-1991, focused on the use of such nonparametric ideas and models in applications of otherwise standard hierarchical models. This chapter provides some historical review and perspective on these developments, with a prime focus on the use and integration of such nonparametric ideas in hierarchical models. We illustrate the ease with which the strict parametric assumptions common to most standard Bayesian hierarchical models can be relaxed to incorporate uncertainties about functional forms using Dirichlet process components, partly enabled by the approach to computation using MCMC methods. The resulting methodology is illustrated with two examples taken from an unpublished 1992 report on the topic.


Posterior Distribution Markov Chain Monte Carlo Conditional Distribution Marginal Distribution Hierarchical Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andrews, D.F. and Mallows, C.L. (1974), “Scale mixtures of normal distributions,”Journal of the Royal Statistical Society Series B36, 99–102.MathSciNetzbMATHGoogle Scholar
  2. Antoniak, C.E. (1974), “Mixtures of Dirichlet processes with applications to nonparametric problems,”The Annals of Statistics2, 1152–1174.MathSciNetzbMATHCrossRefGoogle Scholar
  3. Blackwell, D., and MacQueen, J.B. (1973), “Ferguson Distribution via Polya Urn Schemes,”The Annals of Statistics1, 353–355.MathSciNetzbMATHCrossRefGoogle Scholar
  4. Brunner, L.J. (1995), “ Bayesian linear regression with error terms that have symmetric unimodal densities,”Journal of Nonparametric Statistics4, 335–348.MathSciNetzbMATHCrossRefGoogle Scholar
  5. Bush, C.A. and MacEachern, S.N. (1996), “A semi-parametric Bayesian model for randomized block designs,”Biometrika83, 275–285.zbMATHCrossRefGoogle Scholar
  6. Carlin, B.P., and Poison, N.G. (1991), “An expected utility approach to influence diagnostics,”Journal OftheAmericanStatisticalAssociation, 86, 1013–1021.CrossRefGoogle Scholar
  7. Doss, H. (1994), `Bayesian nonparametric estimation for incomplete data via successive substitution sampling,“The Annals of Statistics22, 1763–1786.MathSciNetzbMATHCrossRefGoogle Scholar
  8. Erkanli, A., Stangl, D., and Müller, P. (1993), “Analysis of ordinal data by the mixture of probit links,”Discussion Paper #93-A01ISDS, Duke University.Google Scholar
  9. Escobar, M.D. (1988), Estimating the means of several normal populations by nonparametric estimation of the distribution of the means, Unpublished dissertation, Yale University.Google Scholar
  10. Escobar, M.D. (1992), Invited comment of “Bayesian analysis of mixtures: sonne results on exact estimability and identification,” by Florens, Mouchart, and Rolin.Bayesian Statistics 4(eds: J.M. Bernardo, J.O. Berger, A.P. Dawid and A.F.M. Smith). Oxford: University press, 142–144.Google Scholar
  11. Escobar, M.D. (1994), “Estimating normal means with a Dirichlet process prior,”Journal of the American Statistical Association89, 268–277.MathSciNetzbMATHCrossRefGoogle Scholar
  12. Escobar, M.D., (1995), “Nonparametrics Bayesian Methods in Hierarchical Models,”The Journal of Statistical Inference and Planning43, 97106.MathSciNetGoogle Scholar
  13. Escobar, M.D. (1998), “The effect of the prior on nonparametric Bayesian methods,” (in preparation).Google Scholar
  14. Escobar, M.D. and West, M. (1995), “Bayesian density estimation and inference using mixtures,”Journal of the American Statistical Association90, 577–588.MathSciNetzbMATHCrossRefGoogle Scholar
  15. Evans, M. and Swartz, T. (1995) “Methods for approximating integrals in statistics with special emphasis on Bayesian integration problems - with discussion,”Statistical Science10, 254–272.MathSciNetzbMATHCrossRefGoogle Scholar
  16. Ferguson, T.S. (1973), “A Bayesian analysis of some nonparametric problems,”The Annals of Statistics 1209–230.MathSciNetzbMATHCrossRefGoogle Scholar
  17. Ferguson, T.S. (1974), “Prior distributions on spaces of probability measures,”The Annals of Statistics2, 615–629.MathSciNetzbMATHCrossRefGoogle Scholar
  18. Freedman, D. (1963), “On the asymptotic behavior of Bayes estimates in the discrete case,”Annals of Mathematical Statistics34, 1386–1403.MathSciNetCrossRefGoogle Scholar
  19. Gelfand, A.E., Hills, S.E., Racine-Poon, A., and Smith, A.F.M. (1990), “Illustration of Bayesian inference in normal data models using Gibbs sampling.”Journal Of the American Statistical Association85, 972–985.CrossRefGoogle Scholar
  20. George, E.I., Makov, U.E., and Smith, A.F.M. (1994), “Fully Bayesian hierarchical analysis for exponential families via Monte Carlo computation,”Aspects of Uncertainty: A Tribute to D.V. Lindley (eds: AFM Smith and PR Freeman), London: John Wiley and Sons, 181–199.Google Scholar
  21. Gilks, W.R., Richardson, S., and Spiegelhalter, D.J., eds. (1996)Markov Chain Monte Carlo in PracticeLondon: Chapman and Hall.zbMATHGoogle Scholar
  22. Kuo, L. and Mallick, B. (1998), “Bayesian semiparametric inference for the accelerated failure,”Canadian Journal of Statisticsto appear.Google Scholar
  23. Liu, J.S. (1996), “Nonparametric hierarchical Bayes via sequential imputations,”The Annals of Statistics24, 911–930.MathSciNetzbMATHCrossRefGoogle Scholar
  24. MacEachern, S.N. (1988), Sequential Bayesian bioassay design. Unpublished dissertation. University of Minnesota.Google Scholar
  25. MacEachern, S.N. (1994), “Estimating normal means with a conjugate style Dirichlet process prior,”Communications in Statistics: Simulationand Computation, 23, 727–741.MathSciNetzbMATHCrossRefGoogle Scholar
  26. MacEachern, S.N. and Müller, P. (1998), “Estimating mixture of Dirichlet process models,”Journal of Computational and GraphicalStatistics, to appear.Google Scholar
  27. Mukhopadhyay, S. and Gelfand, A.E. (1997), “Dirichlet process mixed generalized linear models,”Journal Of the American Statistical Association92, 633–639.MathSciNetzbMATHCrossRefGoogle Scholar
  28. Müller, P., Erkanli, A., and West, M. (1996), “Bayesian curve fitting using multivariate normal mixtures,”Biometrika83, 67–79.MathSciNetzbMATHCrossRefGoogle Scholar
  29. Müller, P., West, M., and MacEachern, S.N. (1997), “Bayesian models for non-linear auto-regressions,”Journal of Time Series Analysis(in press).Google Scholar
  30. Naylor, J.C. and Smith, A.F.M. (1982), “Applications of a method for the efficient computation of posterior distributions,”AppliedStatistics, 31, 214–235.MathSciNetzbMATHCrossRefGoogle Scholar
  31. Pauler, D.K., Escobar, M.D., Sweeney, J.A., and Greenhouse, J. (1996), “Mixture Models for Eye-Tracking Data: A Case Study”Statistics in Medicine15, 1365–1376.CrossRefGoogle Scholar
  32. Roeder, K., Escobar, M., Kadane, J., and Balazs, I., (1998), “Measuring Heterogeneity in Forensic Databases”Biometrikato appear.Google Scholar
  33. Sweeney, J.A., Clementz, B.A., Escobar, M.D., Li, S., Pauler, D.K., and Haas, G.L. (1993), “Mixture analysis of pursuit eye tracking dysfunction in schizophrenia,”Biological Psychiatry34, 331–340.CrossRefGoogle Scholar
  34. Tanner, M.A. and Wong, W.H. (1987), “The calculation of posterior distributions by data augmentation (with discussion),”Journal of the American Statistical Association82, 528–550.MathSciNetzbMATHCrossRefGoogle Scholar
  35. Tomlinson, G. (1998), Analysis of densities with Dirichlet process priors. Unpublished dissertation. University of Toronto.Google Scholar
  36. Turner, D.A., and West, M. (1993), “Statistical analysis of mixtures applied to postsynpotential fluctuations,”Journal of Neuroscience Methods47, 1–23.CrossRefGoogle Scholar
  37. West, M. (1987), “On scale mixtures of normal distributions,”Biometrika74, 646–648.MathSciNetzbMATHCrossRefGoogle Scholar
  38. West, M. (1990), “Bayesian kernel density estimation,”ISDS Discussion Paper #90-A 02Duke University.Google Scholar
  39. West, M. (1992), “Hyperparameter estimation in Dirichlet process mixture models,”ISDS Discussion Paper #92-A03Duke University.Google Scholar
  40. West, M. (1997), “Hierarchical mixture models in neurological transmission analysis,”Journal Of the American Statistical Association92, 587–608.MathSciNetzbMATHCrossRefGoogle Scholar
  41. West, M., and Cao, G. (1993), “Assessing mechanisms of neural synaptic activity,” InBayesian Statistics in Science and Technology: Case Studies(eds: C.A. Gatsonis, J.S. Hodges, R.E. Kass, and N.D. Singpur-walla), New York: Springer-Verlag.Google Scholar
  42. West, M., Müller, P., and Escobar, M.D. (1994), “Hierarchial Priors and Mixture Models, with Applications in Regression and Density Estimation,”Aspects of Uncertainty: A Tribute to D. V. Lindley(eds: AFM Smith and PR Freeman), London: John Wiley and Sons, 363–386.Google Scholar
  43. West, M., and Turner, D.A. (1994), “Deconvolution of mixtures in analysis of neural synaptic transmission,”The Statistician43, 31–43.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1998

Authors and Affiliations

  • Michael D. Escobar
  • Mike West

There are no affiliations available

Personalised recommendations