Computational Statistics

, Volume 15, Issue 3, pp 315–336 | Cite as

Posterior inference in the random intercept model based on samples obtained with Markov chain Monte Carlo methods

  • Herbert Hoijtink


Many papers (including most of the papers in this issue of Computational Statistics) deal with Markov Chain Monte Carlo (MCMC) methods. This paper will give an introduction to the augmented Gibbs sampler (a special case of MCMC), illustrated using the random intercept model. A’ nonstandard’ application of the augmented Gibbs sampler will be discussed to give an illustration of the power of MCMC methods. Furthermore, it will be illustrated that the posterior sample resulting from an application of MCMC can be used for more than determination of convergence and the computation of simple estimators like the a posteriori expectation and standard deviation. Posterior samples give access to many other inferential possibilities. Using a simulation study, the frequency properties of some of these possibilities will be evaluated.


Data Augmentation EM Algorithm Gibbs Sampler MCMC Multilevel Model Posterior Inference 


  1. Box, G.E.P. and Tiao, C. (1973). Bayesian Inference in Statistical Analysis. London: Addison-Wesley.zbMATHGoogle Scholar
  2. Browne, W.J. (1998). Applying MCMC Methods to Multi-level Models. Unpublished doctoral dissertation, University of Bath, United Kingdom.Google Scholar
  3. Bryk, A.S. and Raudenbush S.W. (1992). Hierarchical Linear Models: Applications and Data Analysis Methods. London: SAGE. temGoogle Scholar
  4. Carlin, B.P. and Louis, T.A. (1996). Bayes and Empirical Bayes Methods for Data Analysis. London: Chapman and Hall.zbMATHGoogle Scholar
  5. Casella, G. and George, E. (1992). Explaining the Gibbs sampler. The American Statistician, 46, 167–174.MathSciNetGoogle Scholar
  6. Chib, S. and Greenberg, E. (1995). Understanding the Metropolis-Hastings Algorithm. The American Statistician, 49, 327–335.Google Scholar
  7. Cowles, M.K. and Carlin B.P. (1996). Markov chain Monte Carlo convergence diagnostics: A comparative review. Journal of the American Statistical Association, 91, 883–904.MathSciNetCrossRefGoogle Scholar
  8. Dempster, A.P., Laird, N.M. and Rubin, D.B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39, 1–38.MathSciNetzbMATHGoogle Scholar
  9. DiCiccio, T.J., Kass, R.E., Raftery, A. and Wasserman, L. (1997). Computing Bayes factors by combining simulation and asymptotic approximations. Journal of the American Statistical Association, 92, 903–915.MathSciNetCrossRefGoogle Scholar
  10. Gelfand, A.E., Hills, S.E., Racine-Poon, A. and Smith, A.F.M. (1990). Illustration of Bayesian inference in normal data models using Gibbs sampling. Journal of the American Statistical Association, 85, 972–985.CrossRefGoogle Scholar
  11. Gelfand, A.E. and Smith, A.F.M. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.MathSciNetCrossRefGoogle Scholar
  12. Gelman, A., Carlin, J.B., Stern, H.S. and Rubin, D.B. (1995). Bayesian Data Analysis. Chapman and Hall, London.CrossRefGoogle Scholar
  13. Gelman, A., Meng, X. and Stern, H.S. (1996). Posterior predictive assessment of model fitness via realized discrepancies (with discussion). Statistica Sinica, 6, 733–760.MathSciNetzbMATHGoogle Scholar
  14. Goldstein, H. (1987). Multilevel Models in Educational and Social Research. London: Charles Griffin.Google Scholar
  15. Goldstein, H. and Spiegelhalter, D.J. (1996). League tables and their limitations: statistical issues in comparisons of institutional performance (with discussion). Journal of the Royal Statistical Society, Series A, 159, 385–444.CrossRefGoogle Scholar
  16. Hobert, J.P. and Casella, G. (1996). The effect of improper priors on Gibbs sampling in hierarchical linear mixed models. Journal of the American Statistical Association, 91, 1461–1473.MathSciNetCrossRefGoogle Scholar
  17. Hoijtink, H. (1998). Constrained latent class analysis using the Gibbs sampler and posterior predictive p-values: applications to educational testing. Statistica Sinica, 8, 691–711.MathSciNetzbMATHGoogle Scholar
  18. Kass, R.E. and Raftery, A.E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795.MathSciNetCrossRefGoogle Scholar
  19. Meng, X.L. (1994). Posterior Predictive p-Values. The Annals of Statistics, 22, 1142–1160.MathSciNetCrossRefGoogle Scholar
  20. Mortimore, P., Sammons, P., Stoll, L., Lewis, D. and Ecob, R. (1988). School Matters, the Junior Years. Wells, Open Books.Google Scholar
  21. Multilevel Models Project (1991). ML3 Data Library. London: Multilevel Models Project, Institute of Education, University of London.Google Scholar
  22. Rubin, D.B. (1984). Bayesian justifiable and relevant frequency calculations for the applied statistician. The Annals of Statistics, 12, 1151–1172.MathSciNetCrossRefGoogle Scholar
  23. Self, S.G. and Liang, K.Y. (1987). Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. Journal of the American Statistical Association, 82, 605–610.MathSciNetCrossRefGoogle Scholar
  24. Smith, A.F.M. and Roberts, G.O. (1993). Bayesian computation via the Gibbs sampler and related Markov Chain Monte Carlo methods. Journal of the Royal Statistical Society, Series B, 55, 3–24.MathSciNetzbMATHGoogle Scholar
  25. Spiegelhalter, D., Thomas, A., Best, N. and Gilks, W. (1996a). BUGS 0.5 Bayesian Inference Using Gibbs Sampling. Manual (version ii).Google Scholar
  26. Spiegelhalter, D., Thomas, A., Best, N. and Gilks, W. (1996b). BUGS 0.5 Examples. Volume 1 (version i).Google Scholar
  27. Tanner, M.A. (1993). Tools for Statistical Inference. Springer, New York.CrossRefGoogle Scholar
  28. Tanner, M.A. and Wong, W.H. (1987). The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82, 528–540.MathSciNetCrossRefGoogle Scholar
  29. Tierney, L. (1994). Markov chains for exploring posterior distributions (with discussion). Annals of Statistics, 22, 1701–1762.MathSciNetCrossRefGoogle Scholar
  30. Zeger, S.L. and Karim, M.R. (1991). Generalized linear models with random effect; a Gibbs sampling approach. Journal of the American Statistical Association, 86, 79–86.MathSciNetCrossRefGoogle Scholar

Copyright information

© Physica-Verlag 2000

Authors and Affiliations

  • Herbert Hoijtink
    • 1
  1. 1.Department of Methodology and StatisticsUtrecht UniversityUtrechtThe Netherlands

Personalised recommendations