Skip to main content
SpringerLink
Log in

Dirichlet Processes

  • Chapter
  • First Online:
Partitions, Hypergeometric Systems, and Dirichlet Processes in Statistics

Part of the book series: SpringerBriefs in Statistics ((JSSRES))

  • 627 Accesses

Abstract

After preparing some basic facts in probability theory, this chapter introduces de Finetti’s representation theorem. We, then, introduce the Dirichlet process and the Poisson–Dirichlet distribution, which are closely related to exchangeability. We discuss two constructions of the Dirichlet process: one based on the normalized gamma process, the other using the stick breaking. The relationship between these two constructions is revealed in terms of biased permutations. The sequential sampling scheme is known as Blackwell–MacQueen’s urn scheme. A sample from the Dirichlet process follows the Ewens sampling formula, which was encountered as a measure on partitions in Chap. 2. The Ewens sampling formula is an example of exchangeable partition probability function. The Dirichlet process possesses several nice properties for statistical applications. Therefore, the Dirichlet process has been used as a fundamental prior process in Bayesian nonparametrics. We will discuss some prior processes as generalizations of the Dirichlet process. Several prior processes naturally appear in connection with infinite exchangeable Gibbs partitions introduced in Chap. 2.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Billingsley, P.: Probability and Measure, 3rd edn. Wiley, New York (1995)

    MATH  Google Scholar 

  2. Durrett, R.: Probability: Theory and Examples, 4th edn. Cambridge University Press, New York (2010)

    Book  Google Scholar 

  3. Rogers, L.C.G., Williams, D.: Diffusions, Markov Processes and Martingales, vol. 1, 2nd edn. Cambridge University Press, New York (2000)

    Google Scholar 

  4. Funaki, T.: Kakuritsu-ron. Asakura, Tokyo (2004). in Japanese

    Google Scholar 

  5. Itô, K.: Kakuritsu-ron. Iwanami, Tokyo (1991). in Japanese

    Google Scholar 

  6. Doob, J.L.: Stochastic Processes. Wiley, New York (1953)

    MATH  Google Scholar 

  7. Aldous, D.J.: Exchangeability and related topics. In: Ecole d’Été de Probabilités de Saint Flour. Lecture Notes in Mathematics, vol. 1117. Springer, Berlin (1985)

    Google Scholar 

  8. Lehmann, E.L., Romano, J.P.: Testing Statistical Hypotheses, 3rd edn. Springer, New York (2005)

    MATH  Google Scholar 

  9. Hewitt, E., Savage, L.J.: Symmetric measures on Cartesian products. Trans. Am. Math. Soc. 80, 470–501 (1955)

    Article  MathSciNet  Google Scholar 

  10. Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. Ann. Stat. 1, 209–230 (1973)

    Article  MathSciNet  Google Scholar 

  11. Ghosh, J.K., Ramamoorthi, R.V.: Bayesian Nonparametrics. Springer, New York (2003)

    MATH  Google Scholar 

  12. Ghosal, S.: The Dirichlet process, related priors and posterior asymptotics. In: Hjort, N.L., Holmes, C., Müller, P., Walker, S.G. (eds.) Bayesian Nonparametrics. Cambridge University Press, Cambridge (2010)

    Google Scholar 

  13. Ghosal, S., van der Vaart, A.: Fundamentals of Nonparametric Bayesian Inference. Cambridge University Press, Cambridge (2017)

    Book  Google Scholar 

  14. Sato, K.-I.: Lévy Processes and Infinitely Divisible Distributions. Cambridge University Press, New York (1999)

    MATH  Google Scholar 

  15. Sethuraman, J.: A constructive definition of Dirichlet priors. Stat. Sinica 4, 639–650 (1994)

    MathSciNet  MATH  Google Scholar 

  16. Ishwaran, H., James, L.F.: Gibbs sampling methods for stick-breaking priors. J. Am. Stat. Assoc. 96, 161–173 (2001)

    Article  MathSciNet  Google Scholar 

  17. Pitman, J.: Combinatorial Stochastic Processes. In: Ecole d’Été de Probabilités de Saint Flour. Lecture Notes in Mathematics, vol. 1875. Springer, Berlin (2006)

    Google Scholar 

  18. Blackwell, D., MacQueen, J.B.: Ferguson distributions via Pólya urn schemes. Ann. Stat. 1, 353–355 (1973)

    Article  Google Scholar 

  19. Hoppe, F.: Pólya-like urns and the Ewens’ sampling formula. J. Math. Biol. 20, 91–94 (1984)

    Article  MathSciNet  Google Scholar 

  20. Antoniak, C.: Mixture of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Stat. 2, 1152–1174 (1974)

    Article  MathSciNet  Google Scholar 

  21. Kingman, J.F.C.: The representation of partition structures. J. Lond. Math. Soc. 18, 374–380 (1978)

    Article  MathSciNet  Google Scholar 

  22. Kingman, J.F.C.: Random partitions in population genetics. Proc. R. Soc. Lond. Ser. A. 361, 1–20 (1978)

    Article  MathSciNet  Google Scholar 

  23. Lo, A.Y.: Weak convergence for Dirichlet processes. Sankhyā A 45, 105–111 (1983)

    MathSciNet  MATH  Google Scholar 

  24. Efron, B., Thisted, R.: Estimating the number of unseen species: How many words did Shakespeare Know? Biometrika 63, 435–447 (1976)

    MATH  Google Scholar 

  25. Lijoi, A., Mena, R.H., Prünster, I.: Bayesian nonparametric estimation of the probability of discovering new species. Biometrika 94, 769–786 (2007)

    Article  MathSciNet  Google Scholar 

  26. Lijoi, A., Prünster, I., Walker, S.G.: Bayesian nonparametric estimators derived from conditional Gibbs structures. Ann. Appl. Probab. 18, 1519–1547 (2008)

    Article  MathSciNet  Google Scholar 

  27. Sibuya, M.: Prediction in Ewens-Pitman sampling formula and random samples from number partitions. Ann. Inst. Stat. Math. 66, 833–864 (2014)

    Article  MathSciNet  Google Scholar 

  28. Keener, R., Rothman, E., Starr, N.: Distribution of partitions. Ann. Stat. 15, 1466–1481 (1978)

    Article  MathSciNet  Google Scholar 

  29. Lijoi, A., Prünster, I.: Models beyond the Dirichlet process. In: Hjort, N.L., et al. (eds.) Bayesian Nonparametrics. Cambridge University Press, Cambridge (2010)

    Google Scholar 

  30. Hjort, N.L., et al. (eds.): Bayesian Nonparametrics. Cambridge University Press, Cambridge (2010)

    MATH  Google Scholar 

  31. Phadia, E.G.: Prior Processes and Their Applications: Nonparametric Bayesian Estimation, 2nd edn. Springer, Switzerland (2013)

    Book  Google Scholar 

  32. Escobar, M.D., West, M.: Bayesian density estimation and inference using mixtures. J. Am. Stat. Assoc. 90, 577–588 (1995)

    Article  MathSciNet  Google Scholar 

  33. Ishwaran, H., James, F.L.: Generalized weighted Chinese restaurant processes for species sampling mixture models. Stat. Sinica 13, 1211–1235 (2003)

    MathSciNet  MATH  Google Scholar 

  34. Lee, J., Quintana, F.A., Müller, P., Trippa, L.: Defining predictive probability functions for species sampling models. Stat. Sci. 28, 209–222 (2013)

    Article  MathSciNet  Google Scholar 

  35. Pitman, J., Yor, M.: The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann. Probab. 25, 855–900 (1997)

    Article  MathSciNet  Google Scholar 

  36. Carlton, M.A.: Applications of the two-parameter Poisson–Dirichlet distribution. Dissertation, University of California, Los Angeles (1999)

    Google Scholar 

  37. James, L.F.: Large sample asymptotics for the two-parameter Poisson–Dirichlet process. Inst. Math. Stat. Collect. 3, 187–199 (2008)

    Article  MathSciNet  Google Scholar 

  38. Jang, G.H., Lee, J., Lee, S.: Posterior consistency of species sampling priors. Stat. Sinica 20, 581–593 (2010)

    MathSciNet  MATH  Google Scholar 

  39. Lo, A.Y.: On a class of Bayesian nonparametric estimates: I. density estimates. Ann. Stat. 12, 351–357 (1984)

    Article  MathSciNet  Google Scholar 

  40. Ghosal, S., Ghosh, J.K., Ramamoorthi, R.V.: Posterior consistency of Dirichlet mixtures in density estimation. Ann. Stat. 27, 143–158 (1999)

    Article  MathSciNet  Google Scholar 

  41. Lijoi, A., Prüenster, I., Walker, S.G.: On consistency of nonparametric normal mixtures for Bayesian density estimation. J. Am. Stat. Assoc. 100, 1292–2196 (2005)

    Article  MathSciNet  Google Scholar 

  42. Gnedin, A., Pitman, J.: Exchangeable Gibbs partitions and Stirling triangles. Zap. Nauchn. Sem. S. POMI 325, 83–102 (2005). English translation: J. Math. Sci. 138, 5674–5685 2006)

    Google Scholar 

  43. Kerov, S.V: Coherent random allocations, and the Ewens-Pitman formula. Zap. Nauchn. Semi. POMI 325, 127–145 (1995). English translation: J. Math. Sci. 138, 5699–5710 (2006)

    Google Scholar 

  44. De Iorio, M., Müller, P., Rosner, G.L., MacEachern, S.N.: An ANOVA model for dependent random measures. J. Am. Stat. Assoc. 99, 205–215 (2004)

    Google Scholar 

  45. Gelfand, A., Kottas, A., MacEachern, S.N.: Bayesian nonparametric spatial modeling with Dirichlet process mixing. J. Am. Stat. Assoc. 100, 1021–1035 (2005)

    Article  MathSciNet  Google Scholar 

  46. Griffin, J.E., Steel, M.F.J.: Order-based dependent Dirichlet process. J. Am. Stat. Assoc. 101, 179–194 (2006)

    Article  Google Scholar 

  47. Airoldi, E.M., Costa, T., Bassetti, F., Leisen, F., Guindani, M.: Generalized species sampling priors with latent beta reinforcements. J. Am. Stat. Assoc. 109, 1466–1480 (2014)

    Article  MathSciNet  Google Scholar 

  48. Jo, S., Lee, J., Müller, P., Quintana, F.A., Trippa, L.: Dependent species sampling models for spatial density estimation. Bayesian Anal. 12, 379–406 (2017)

    Article  MathSciNet  Google Scholar 

  49. Griffiths, R.C., Spaǹo, D.: Record indices and age-ordered frequencies in exchangeable Gibbs partitions. Electron. J. Probab. 40, 1101–1130 (2007)

    Article  MathSciNet  Google Scholar 

  50. Lijoi, A., Mena, R.H., Prünster, I.: Controlling the reinforcement in Bayesian nonparametric mixture models. J. R. Stat. Soc. Ser. B 69, 715–740 (2007)

    Article  MathSciNet  Google Scholar 

  51. Feng, S.: The Poisson–Dirichlet Distributions and Related Topics. Springer, Berlin (2010)

    Book  Google Scholar 

  52. Perman, M., Pitman, J., Yor, M.: Size-biased sampling of Poisson point processes and excursions. Probab. Theory Relat. Fields 92, 21–39 (1992)

    Article  MathSciNet  Google Scholar 

  53. Pitman, J.: Poisson-Kingman partitions. Institute of Mathematical Statistics Lecture Notes-Monograph Series, vol. 40, pp. 245–267 (2003)

    Google Scholar 

  54. Pitman, J.: Random discrete distributions invariant under size-biased permutation. Adv. Appl. Probab. 28, 525–539 (1996)

    Article  MathSciNet  Google Scholar 

  55. Lijoi, A., Prünster, I., Walker, S.G.: Investigating nonparametric priors with Gibbs structure. Stat. Sinica 18, 1653–1668 (2008)

    MathSciNet  MATH  Google Scholar 

  56. Lijoi, A., Mena, R.H., Walker, S.G.: Hierarchical mixture modeling with normalized inverse Gaussian priors. J. Am. Stat. Assoc. 100, 1278–1291 (2005)

    Article  MathSciNet  Google Scholar 

  57. Ho, M.-W., James, L.F., Lau, J.W. Gibbs partitions (EPPF’s) derived from a stable subordinator are Fox \(H\)- and Meijer \(G\)-transforms. arXiv: 0708.0619

Download references

Author information

Authors and Affiliations

  1. The Institute of Statistical Mathematics, Tachikawa, Tokyo, Japan

    Shuhei Mano

Authors
  1. Shuhei Mano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuhei Mano .

Rights and permissions

Reprints and permissions

Copyright information

© 2018 The Author(s)

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Mano, S. (2018). Dirichlet Processes. In: Partitions, Hypergeometric Systems, and Dirichlet Processes in Statistics. SpringerBriefs in Statistics(). Springer, Tokyo. https://doi.org/10.1007/978-4-431-55888-0_4

Download citation

Publish with us

Policies and ethics

Search

Navigation