The local Dirichlet process

  • Yeonseung ChungEmail author
  • David B. Dunson


As a generalization of the Dirichlet process (DP) to allow predictor dependence, we propose a local Dirichlet process (lDP). The lDP provides a prior distribution for a collection of random probability measures indexed by predictors. This is accomplished by assigning stick-breaking weights and atoms to random locations in a predictor space. The probability measure at a given predictor value is then formulated using the weights and atoms located in a neighborhood about that predictor value. This construction results in a marginal DP prior for the random measure at any specific predictor value. Dependence is induced through local sharing of random components. Theoretical properties are considered and a blocked Gibbs sampler is proposed for posterior computation in lDP mixture models. The methods are illustrated using simulated examples and an epidemiologic application.


Dependent Dirichlet process Blocked Gibbs sampler Mixture model Non-parametric Bayes Stick-breaking representation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Beal, M., Ghahramani, Z., Rasmussen, C. (2002). The infinite hidden Markov model. In Neural information processing systems (Vol. 14). Cambridge: MIT Press.Google Scholar
  2. Blei, D., Griffiths, T., Jordan, M., Tenenbaum, J. (2004). Hierarchical topic models and the nested Chinese restaurant process. In Neural information processing systems (Vol. 16). Cambridge: MIT Press.Google Scholar
  3. Caron, F., Davy, M., Doucet, A., Duflos, E., Vanheeghe, P. (2006). Bayesian inference for dynamic models with Dirichet process mixtures. In International conference on information fusion, Italia, July 10–13.Google Scholar
  4. De Iorio M., Müller P., Rosner G.L., MacEachern S.N. (2004) An Anova model for dependent random measures. Journal of the American Statistical Association 99: 205–215zbMATHCrossRefMathSciNetGoogle Scholar
  5. Dowse K.G., Zimmet P.Z., Alberti G.M.M., Bringham L., Carlin J.B., Tuomlehto J., Knight L.T., Gareeboo H. (1993) Serum insulin distributions and reproducibility of the relationship between 2-hour insulin and plasma glucose levels in Asian Indian, Creole, and Chinese Mauritians. Metabolism 42: 1232–1241CrossRefGoogle Scholar
  6. Duan, J. A., Guidani, M., Gelfand, A. E. (2005). Generalized spatial Dirichlet process models. ISDS Discussion Paper, 05-23, Durham: Duke University.Google Scholar
  7. Dunson D.B. (2006) Bayesian dynamic modeling of latent trait distributions. Biostatistics 7: 551–568zbMATHCrossRefGoogle Scholar
  8. Dunson D.B., Park J.-H. (2008) Kernel stick-breaking process. Biometrika 95: 307–323zbMATHCrossRefMathSciNetGoogle Scholar
  9. Dunson D.B., Peddada S.D. (2008) Bayesian nonparametric inference on stochastic ordering. Biometrika 95: 859–874zbMATHCrossRefMathSciNetGoogle Scholar
  10. Dunson D.B., Pillai N., Park J.-H. (2007) Bayesian density regression. Journal of the Royal Statistical Society, Series B 69: 163–183zbMATHCrossRefMathSciNetGoogle Scholar
  11. Escobar M.D. (1994) Estimating normal means with a Dirichlet process prior. Journal of the American Statistical Association 89: 268–277zbMATHCrossRefMathSciNetGoogle Scholar
  12. Escobar M.D., West M. (1995) Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association 90: 577–588zbMATHCrossRefMathSciNetGoogle Scholar
  13. Ferguson T.S. (1973) A Bayesian analysis of some nonparametric problems. The Annals of Statistics 1: 209–230zbMATHCrossRefMathSciNetGoogle Scholar
  14. Ferguson T.S. (1974) Prior distributions on spaces of probability measures. The Annals of Statistics 2: 615–629zbMATHCrossRefMathSciNetGoogle Scholar
  15. Fraley C., Raftery A.E. (2002) Model-based clustering, discriminant analysis, and density estimation. Journal of the American Statistical Association 97: 611–631zbMATHCrossRefMathSciNetGoogle Scholar
  16. Gelfand A.E., Kottas A., MacEachern S.N. (2004) Bayesian nonparametric spatial modeling with Dirichlet process mixing. Journal of the American Statistical Association 100: 1021–1035CrossRefMathSciNetGoogle Scholar
  17. Ghosal S., Vander Vaart A.W. (2007) Posterior convergence rates of Dirichlet mixtures at smooth densities. The Annals of Statistics 35(2): 697–723zbMATHCrossRefMathSciNetGoogle Scholar
  18. Ghosal S., Ghosh J.K., Ramamoorthi R.V. (1999) Posterior consistency of Dirichlet mixtures in density estimation. The Annals of Statistics 27: 143–158zbMATHCrossRefMathSciNetGoogle Scholar
  19. Griffin J.E., Steel M.F.J. (2006) Order-based dependent Dirichlet processes. Journal of the American Statistical Association 101: 179–194zbMATHCrossRefMathSciNetGoogle Scholar
  20. Griffin, J. E., Steel, M. F. J. (2008). Bayesian nonparametric modeling with the Dirichlet process regression smoother. Technical Report, University of Warwick.Google Scholar
  21. Ishwaran H., James L.F. (2001) Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association 96: 161–173zbMATHCrossRefMathSciNetGoogle Scholar
  22. Kim S., Tadesse M.G., Vannucci M. (2006) Variable selection in clustering via Dirichlet process mixture models. Biometrika 94: 877–893CrossRefMathSciNetGoogle Scholar
  23. Lijoi A., Prünster I., Walker S.G. (2005) On consistency of non-parametric normal mixtures for Bayesian density estimation. Journal of the American Statistical Association 100: 1292–1296zbMATHCrossRefMathSciNetGoogle Scholar
  24. Lo A.Y. (1984) On a class of Bayesian nonparametric estimates: I. Density estimates. The Annals of Statistics 12: 351–357zbMATHGoogle Scholar
  25. MacEachern, S. N. (1999). Dependent nonparametric processes. In ASA proceedings of the section on bayesian statistical science. Alexandria: American Statistical Association.Google Scholar
  26. MacEachern, S. N. (2000). Dependent Dirichlet processes. Unpublished manuscript, Department of Statistics, The Ohio State University.Google Scholar
  27. MacEachern S.N. (2001) Decision theoretic aspects of dependent nonparametric processes. In: George E. (eds) Bayesian methods with applications to science, policy and official statistics. ISBA, Creta, pp 551–560Google Scholar
  28. Müller P., Quintana F., Rosner G. (2004) A method for combining inference across related nonparametricBayesian models. Journal of the Royal Statistical Society B 66: 735–749zbMATHCrossRefGoogle Scholar
  29. Pennell M.L., Dunson D.B. (2006) Bayesian semiparametric dynamic frailty models for multiple event time data. Biometrics 62: 1044–1052zbMATHCrossRefMathSciNetGoogle Scholar
  30. Pitman J. (1995) Exchangeable and partially exchangeable random partitions. Probability Theory and Related Fields 102: 145–158zbMATHCrossRefMathSciNetGoogle Scholar
  31. Pitman, J. (1996). Some developments of the Blackwell-MacQueen urn scheme. In T. S. Ferguson, L. S. Shapley, J. B. MacQueen (Eds.), Statistics, probability and game theory. IMS Lecture Notes-Monograph Series (Vol. 30, pp. 245–267), Hayward: Institute of Mathematical Statistics.Google Scholar
  32. Quintana F.A. (2006) A predictive view of Bayesian clustering. Journal of Statistical Planning and Inference 136: 2407–2429zbMATHCrossRefMathSciNetGoogle Scholar
  33. Quintana F.A., Iglesias P.L. (2003) Bayesian Clustering and product partition models. Journal of the Royal Statistical Society B 65: 557–574zbMATHCrossRefMathSciNetGoogle Scholar
  34. Sethuraman J. (1994) A constructive definition of the Dirichlet process prior. Statistica Sinica 2: 639–650MathSciNetGoogle Scholar
  35. Smith, J. W., Everhart, J. E., Dickson, W. C., Knowler, W. C., Johannes, R. S. (1988). Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the symposium on computer applications in medical care, pp. 261–265.Google Scholar
  36. Xing, E. P., Sharan, R., Jordan, M. (2004). Bayesian haplotype inference via the Dirichlet process. In Proceedings of the international conference on machine learning (ICML).Google Scholar

Copyright information

© The Institute of Statistical Mathematics, Tokyo 2009

Authors and Affiliations

  1. 1.Department of BiostatisticsHarvard School of Public HealthBostonUSA
  2. 2.Department of Statistical ScienceDuke UniversityDurhamUSA

Personalised recommendations