Skip to main content
SpringerLink
Log in

Density Estimation: DP Models

  • Chapter
Bayesian Nonparametric Data Analysis

Part of the book series: Springer Series in Statistics ((SSS))

  • 7238 Accesses

Abstract

We discuss the use of nonparametric Bayesian models in density estimation, arguably one of the most basic statistical inference problems. In this chapter we introduce the Dirichlet process prior and variations of it that are the by far most commonly used nonparametric Bayesian models used in this context. Variations include the Dirichlet process mixture and the finite Dirichlet process. One critical reason for the extensive use of these models is the availability of computation efficient methods for posterior simulation. We discuss several such methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Antoniak CE (1974) Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann Stat 2:1152–1174

    Article  MATH  MathSciNet  Google Scholar 

  • Blackwell D, MacQueen JB (1973) Ferguson distributions via Pólya urn schemes. Ann Stat 1:353–355

    Article  MATH  MathSciNet  Google Scholar 

  • Bush CA, MacEachern SN (1996) A semiparametric Bayesian model for randomised block designs. Biometrika 83:275–285

    Article  MATH  Google Scholar 

  • Dahl DB (2005) Sequentially-allocated merge-split sampler for conjugate and nonconjugate Dirichlet process mixture models. Techical Report, Texas AM University

    Google Scholar 

  • Dey D, Müller P, Sinha D (1998) Practical nonparametric and semiparametric Bayesian statistics. Springer, New York

    Book  MATH  Google Scholar 

  • Dubins LE, Freedman DA (1967) Random distribution functions. In: Proceedings of the fifth Berkeley symposium on mathematics, statistics and probability, vol 2, pp 183–214

    MathSciNet  Google Scholar 

  • Escobar MD (1988) Estimating the means of several normal populations by nonparametric estimation of the distributions of the means. Unpublished doctoral thesis, Deparment of Statistics, Yale University

    Google Scholar 

  • Escobar MD (1994) Estimating normal means with a Dirichlet process prior. J Am Stat Assoc 89:268–277

    Article  MATH  MathSciNet  Google Scholar 

  • Escobar MD, West M (1995) Bayesian density estimation and inference using mixtures. J Am Stat Assoc 90:577–588

    Article  MATH  MathSciNet  Google Scholar 

  • Fabius J (1964) Asymptotic behavior of Bayes’ estimates. Ann Math Stat 35:846–856

    Article  MATH  MathSciNet  Google Scholar 

  • Ferguson TS (1973) A Bayesian analysis of some nonparametric problems. Ann Stat 1:209–230

    Article  MATH  MathSciNet  Google Scholar 

  • Ferguson TS (1974) Prior distribution on the spaces of probability measures. Ann Stat 2:615–629

    Article  MATH  MathSciNet  Google Scholar 

  • Ferguson TS (1983) Bayesian density estimation by mixtures of normal distributions. In: Siegmund D, Rustage J, Rizvi GG (eds) Recent advances in statistics. Papers in honor of Herman Chernoff on his sixtieth birthday, Bibliohound, pp 287–302

    Google Scholar 

  • Freedman D (1963) On the asymptotic distribution of Bayes’ estimates in the discrete case. Ann Math Stat 34:1386–1403

    Article  Google Scholar 

  • Ghosh JK, Ramamoorthi RV (2003) Bayesian nonparametrics. Springer, New York

    MATH  Google Scholar 

  • Green PJ, Richardson S (2001) Modelling heterogeneity with and without the Dirichlet process. Scand J Stat 28(2):355–375

    Article  MATH  MathSciNet  Google Scholar 

  • Griffin JE, Walker SG (2011) Posterior simulation of normalized random measure mixtures. J Comput Graph Stat 20(1):241–259. doi:10.1198/jcgs.2010.08176

    Article  MathSciNet  Google Scholar 

  • Guindani M, Sepúlveda N, Paulino CD, Müller P (2014) A Bayesian semi-parametric approach for the differential analysis of sequence counts data. Appl Stat 63:385–404

    Google Scholar 

  • Hjort NL (1996) Bayesian approaches to non- and semiparametric density estimation. In: Bernadro JM, Berger JO, Dawid AP, M SAF (eds) Bayesian statistics, vol 5. Oxford University Press, Oxford, pp 223–253

    Google Scholar 

  • Ishwaran H, James LF (2001) Gibbs sampling methods for stick-breaking priors. J Am Stat Assoc 96:161–173

    Article  MATH  MathSciNet  Google Scholar 

  • Ishwaran H, James LF (2002) Approximate Dirichlet process computing in finite normal mixtures: smoothing and prior information. J Comput Graph Stat 11:508–532

    Article  MathSciNet  Google Scholar 

  • Ishwaran H, James LF (2003) Generalized weighted Chinese restaurant processes for species sampling mixture models. Stat Sin 13:1211–1235

    MATH  MathSciNet  Google Scholar 

  • Ishwaran H, Zarepour M (2000) Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models. Biometrika 87:371–390

    Article  MATH  MathSciNet  Google Scholar 

  • Jain S, Neal RM (2004) A split-merge Markov chain Monte Carlo procedure for the Dirichlet Process mixture model. J Comput Graph Stat 13:158–182

    Article  MathSciNet  Google Scholar 

  • Kalli M, Griffin J, Walker S (2011) Slice sampling mixture models. Stat Comput 21:93–105. doi:10.1007/s11222-009-9150-y. http://dx.doi.org/10.1007/s11222-009-9150-y

  • Kolmogorov AN (1933) Foundations of the theory of probability, 2nd edn. Nathan Morrison, New York

    Google Scholar 

  • Korwar RM, Hollander M (1973) Contributions to the theory of Dirichlet processes. Ann Probab 1:705–711

    Article  MATH  MathSciNet  Google Scholar 

  • Kraft CM (1964) A class of distribution function processes which have derivatives. J Appl Probab 1:385–388

    Article  MATH  MathSciNet  Google Scholar 

  • Liu JS (1996) Nonparametric hierarchical Bayes via sequential imputations. Ann Stat 24:911–930

    Article  MATH  Google Scholar 

  • Lo AY (1984) On a class of Bayesian nonparametric estimates I: density estimates. Ann Stat 12:351–357

    Article  MATH  Google Scholar 

  • MacEachern SN, Müller P (1998) Estimating mixture of Dirichlet process models. J Comput Graph Stat 7(2):223–338

    Google Scholar 

  • MacEachern SN, Clyde M, Liu JS (1999) Sequential importance sampling for nonparametric Bayes models: the next generation. Can J Stat 27:251–267

    Article  MATH  MathSciNet  Google Scholar 

  • Metivier M (1971) Sur la construction de mesures aleatoires presque surement absolument continues par rapport a une mesure donnee. Zeitschrift fur Wahrscheinlichkeitstheorie und Verwandte Gebiete 20:332–334

    Article  MATH  MathSciNet  Google Scholar 

  • Muliere P, Secchi P (1995) A note on a proper Bayesian Bootstrap. Techical Report, Università degli Studi di Pavia, Dipartamento di Economia Politica e Metodi Quantitativ

    Google Scholar 

  • Muliere P, Tardella L (1998) Approximating distributions of random functionals of Ferguson-Dirichlet priors. Can J Stat 26:283–297

    Article  MATH  MathSciNet  Google Scholar 

  • Neal RM (2000) Markov chain sampling methods for Dirichlet process mixture models. J Comput Graph Stat 9:249–265

    MathSciNet  Google Scholar 

  • Newton MA, Zhang Y (1999) A recursive algorithm for nonparametric analysis with missing data. Biometrika 86:15–26

    Article  MATH  MathSciNet  Google Scholar 

  • Newton MA, Quintana FA, Zhang Y (1998) Nonparametric Bayes methods using predictive updating. In: Dey D, Müller P, Sinha D (eds) Practical nonparametric and semiparametric Bayesian statistics. Springer, New York, pp 45–62

    Chapter  Google Scholar 

  • Petrone S, Veronese P (2002) Nonparametric mixture priors based on an exponential random scheme. Stat Methods Appl 11:1–20

    Article  Google Scholar 

  • Phillips DB, Smith AFM (1996) Bayesian model comparisons via jump diffusions. In: Gilks WR, Ricahrdson S, Spiegelhalter DJ (eds) Markov chain Monte Carlo in practice. Chapman and Hall, New York

    Google Scholar 

  • Pitman J (1996) Some developments of the Blackwell-MacQueen urn scheme. In: Ferguson TS, Shapeley LS, MacQueen JB (eds) Statistics, probability and game theory. Papers in honor of David Blackwell. IMS lecture notes - monograph series. IMS, Hayward, pp 245–268

    Chapter  Google Scholar 

  • Pitman J, Yor M (1997) The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Ann Probab 25:855–900

    Article  MATH  MathSciNet  Google Scholar 

  • Richardson S, Green PJ (1997) On Bayesian analysis of mixtures with an unknown number of components. J R Stat Soc B 59:731–792

    Article  MATH  MathSciNet  Google Scholar 

  • Sethuraman J (1994) A constructive definition of Dirichlet prior. Stat Sin 2:639–650

    MathSciNet  Google Scholar 

  • Walker SG (2007) Sampling the Dirichlet mixture model with slices. Commnun Stat Simul Comput 36:45–54

    Article  MATH  Google Scholar 

  • West M, Müller P, Escobar MD (1994) Hierarchical priors and mixture models, with application in regression and density estimation. In: Freeman PR, Smith AFM (eds) Aspects of uncertainty. Wiley, New York, pp 363–368

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Texas at Austin, Austin, TX, USA

    Peter Müller

  2. Departamento de Estadística, Pontificia Universidad Católica, Santiago, Chile

    Fernando Andrés Quintana & Alejandro Jara

  3. Department of Statistics, University of South Carolina, Columbia, SC, USA

    Tim Hanson

Authors
  1. Peter Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernando Andrés Quintana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alejandro Jara
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tim Hanson
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Müller, P., Quintana, F.A., Jara, A., Hanson, T. (2015). Density Estimation: DP Models. In: Bayesian Nonparametric Data Analysis. Springer Series in Statistics. Springer, Cham. https://doi.org/10.1007/978-3-319-18968-0_2

Download citation

Publish with us

Policies and ethics

Search

Navigation