Statistical Methods & Applications

, Volume 20, Issue 3, pp 259–290 | Cite as

Sampling schemes for generalized linear Dirichlet process random effects models

  • Minjung Kyung
  • Jeff Gill
  • George Casella


We evaluate MCMC sampling schemes for a variety of link functions in generalized linear models with Dirichlet process random effects. First, we find that there is a large amount of variability in the performance of MCMC algorithms, with the slice sampler typically being less desirable than either a Kolmogorov–Smirnov mixture representation or a Metropolis–Hastings algorithm. Second, in fitting the Dirichlet process, dealing with the precision parameter has troubled model specifications in the past. Here we find that incorporating this parameter into the MCMC sampling scheme is not only computationally feasible, but also results in a more robust set of estimates, in that they are marginalized-over rather than conditioned-upon. Applications are provided with social science problems in areas where the data can be difficult to model, and we find that the nonparametric nature of the Dirichlet process priors for the random effects leads to improved analyses with more reasonable inferences.


Linear mixed models Generalized linear mixed models Hierarchical models Gibbs sampling Metropolis–Hastings algorithm Slice sampling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abramowitz M, Stegun IA (1972) Stirling numbers of the second kind. Section 24.1.4. In: Handbook of mathematical functions with formulas, graphs, and mathematical tables, 9th printing. Dover, New York, pp 824–825.Google Scholar
  2. Albert JH, Chib S (1993) Bayesian analysis of binary and polychotomous response data. J Am Stat Assoc 88: 669–679MathSciNetzbMATHCrossRefGoogle Scholar
  3. Andrews DF, Mallows CL (1974) Scale mixtures of normal distributions. J R Stat Soc Ser B 36: 99–102MathSciNetzbMATHGoogle Scholar
  4. Antoniak CE (1974) Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann Stat 2: 1152–1174MathSciNetzbMATHCrossRefGoogle Scholar
  5. Balakrishnan N (1992) Handbook of the logistic distribution. CRC Press, Boca RatonzbMATHGoogle Scholar
  6. Blackwell D, MacQueen JB (1973) Discreteness of Ferguson selections. Ann Stat 1: 358–365MathSciNetGoogle Scholar
  7. Breslow NE, Clayton DG (1993) Approximate inference in generalized linear mixed models. J Am Stat Assoc 88: 9–25zbMATHCrossRefGoogle Scholar
  8. Buonaccorsi JP (1996) Measurement error in the response in the general linear model. J Am Stat Assoc 91: 633–642MathSciNetzbMATHCrossRefGoogle Scholar
  9. Chib S, Greenberg E, Chen Y (1998) MCMC methods for fitting and comparing multinomial response models. Technical Report, Economics Working Paper Archive, Washington University at St. Louis,
  10. Chib S, Winkelmann R (2001) Markov chain Monte Carlo analysis of correlated count data. J Bus Econ Stat 19: 428–435MathSciNetCrossRefGoogle Scholar
  11. Damien P, Wakefield J, Walker S (1999) Gibbs sampling for Bayesian non-conjugate and hierarchical models by using auxiliary variables. J R Stat Soc Ser B 61: 331–344MathSciNetzbMATHCrossRefGoogle Scholar
  12. Devroye L (1986) Non-uniform random variate generation. Springer, New YorkzbMATHGoogle Scholar
  13. Dey DK, Ghosh SK, Mallick BK (2000) Generalized linear models: a Bayesian perspective. Marcel Dekker, New YorkzbMATHGoogle Scholar
  14. Dorazio RM, Mukherjee B, Zhang L, Ghosh M, Jelks HL, Jordan F (2007) Modelling unobserved sources of heterogeneity in animal abundance using a Dirichlet process prior. Biometrics (online publication August 3, 2007)Google Scholar
  15. Escobar MD, West M (1995) Bayesian density estimation and inference using mixtures. J Am Stat Assoc 90: 577–588MathSciNetzbMATHCrossRefGoogle Scholar
  16. Fahrmeir L, Tutz G (2001) Multivariate statistical modelling based on generalized linear models. 2. Springer, New YorkzbMATHGoogle Scholar
  17. Ferguson TS (1973) A Bayesian analysis of some nonparametric problems. Ann Stat 1: 209–230MathSciNetzbMATHCrossRefGoogle Scholar
  18. Gill J, Casella G (2009) Nonparametric priors for ordinal Bayesian social science models: specification and estimation. J Am Stat Assoc 104: 453–464MathSciNetCrossRefGoogle Scholar
  19. Gurr TR, Marshall MG, Jaggers K (2003) PolityIV,
  20. Ishwaran H, James LF (2001) Gibbs sampling methods for stick-breaking priors. J Am Stat Assoc 96: 161–173MathSciNetzbMATHCrossRefGoogle Scholar
  21. Jiang J (2007) Linear and generalized linear mixed models and their applications. Springer, New YorkzbMATHGoogle Scholar
  22. Koch MT, Cranmer S (2007) Terrorism than governments of the right? Testing the ‘Dick Cheney’ hypothesis: do governments of the left attract more than governments of the right?. Conflict Manage Peace Sci 24: 311–326CrossRefGoogle Scholar
  23. Korwar RM, Hollander M (1973) Contributions to the theory of Dirichlet processes. Ann Probab 1: 705–711MathSciNetzbMATHCrossRefGoogle Scholar
  24. Kyung M, Gill J, Casella G (2009) Characterizing the variance improvement in linear Dirichlet random effects models. Stat Probab Lett 79: 2343–2350MathSciNetzbMATHCrossRefGoogle Scholar
  25. Kyung M, Gill J, Casella G (2010) Estimation in Dirichlet random effects models. Ann Stat 38: 979–1009MathSciNetzbMATHCrossRefGoogle Scholar
  26. Kyung M, Gill J, Casella G (2011) New findings from terrorism data: Dirichlet process random effects models for latent groups. J R Stat Soc Ser C (Forthcoming)Google Scholar
  27. Liu JS (1996) Nonparametric hierarchical Bayes via sequential imputations. Ann Stat 24: 911–930zbMATHCrossRefGoogle Scholar
  28. Lo AY (1984) On a class of Bayesian nonparametric estimates: I. Density estimates. Ann Stat 12: 351–357zbMATHCrossRefGoogle Scholar
  29. MacEachern SN, Müller P (1998) Estimating mixture of Dirichlet process model. J Comput Graph Stat 7: 223–238CrossRefGoogle Scholar
  30. McAuliffe JD, Blei DM, Jordan MI (2006) Nonparametric empirical Bayes for the Dirichlet process mixture model. Stat Comput 16: 5–14MathSciNetCrossRefGoogle Scholar
  31. McCullagh P, Nelder JA (1989) Generalized linear models. 2. Chapman & Hall, New YorkzbMATHGoogle Scholar
  32. McCullagh P, Yang J (2006) Stochastic classification models. Int Congr Math III: 669–686MathSciNetGoogle Scholar
  33. McCulloch CE, Searle SR (2001) Generalized, linear, and mixed models. Wiley, New YorkzbMATHGoogle Scholar
  34. Mira A, Tierney L (2002) Efficiency and convergence properties of slice samplers. Scand J Stat 29: 1–12MathSciNetzbMATHCrossRefGoogle Scholar
  35. Neal RM (2000) Markov chain sampling methods for Dirichlet process mixture models. J Comput Graph Stat 9: 249–265MathSciNetCrossRefGoogle Scholar
  36. Neal RM (2003) Slice sampling. Ann Stat 31: 705–741MathSciNetzbMATHCrossRefGoogle Scholar
  37. Robert C, Casella G (2004) Monte Carlo statistical methods. Springer, New YorkzbMATHGoogle Scholar
  38. Sethuraman J (1994) A constructive definition of Dirichlet priors. Statistica Sinica 4: 639–650MathSciNetzbMATHGoogle Scholar
  39. Teh YW, Jordan MI, Beal MJ, Blei DM (2006) Hierarchical Dirichlet processes. J Am Stat Assoc 101: 1566–1581MathSciNetzbMATHCrossRefGoogle Scholar
  40. Tierney L, Kadane JB (1986) Accurate approximations for posterior moments and marginal densities. J Am Stat Assoc 81: 82–86MathSciNetzbMATHCrossRefGoogle Scholar
  41. Wang N, Lin X, Gutierrez RG, Carroll RJ (1998) Bias analysis and SIMEX approach in generalized linear mixed measurement error models. J Am Stat Assoc 93: 249–261MathSciNetzbMATHCrossRefGoogle Scholar
  42. West M (1987) On scale mixtures of normal distributions. Biometrika 74: 646–648MathSciNetzbMATHCrossRefGoogle Scholar
  43. Wolfinger R, O’Connell M (1993) Generalized linear mixed models: a pseudolikelihood approach. J Stat Comput Simul 48: 233–243zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Department of StatisticsDuksung Women’s UniversitySeoulKorea
  2. 2.Department of Political ScienceWashington UniversitySt. LouisUSA
  3. 3.Department of BiostatisticsWashington UniversitySt. LouisUSA
  4. 4.Department of StatisticsUniversity of FloridaGainesvilleUSA

Personalised recommendations