Skip to main content
SpringerLink
Log in

A representation of exchangeable hierarchies by sampling from random real trees

  • Published:
Probability Theory and Related Fields Aims and scope Submit manuscript

Abstract

A hierarchy on a set S, also called a total partition of S, is a collection \(\mathcal {H}\) of subsets of S such that \(S \in \mathcal {H}\), each singleton subset of S belongs to \(\mathcal {H}\), and if \(A, B \in \mathcal {H}\) then \(A \cap B\) equals either A or B or \(\varnothing \). Every exchangeable random hierarchy of positive integers has the same distribution as a random hierarchy \(\mathcal {H}\) associated as follows with a random real tree \(\mathcal {T}\) equipped with root element 0 and a random probability distribution p on the Borel subsets of \(\mathcal {T}\): given \((\mathcal {T},p)\), let \(t_1,t_2, \ldots \) be independent and identically distributed according to p, and let \(\mathcal {H}\) comprise all singleton subsets of \({\mathbb {N}}\), and every subset of the form \(\{j:t_j \in F(x)\}\) as x ranges over \(\mathcal {T}\), where F(x) is the fringe subtree of \(\mathcal {T}\) rooted at x. There is also the alternative characterization: every exchangeable random hierarchy of positive integers has the same distribution as a random hierarchy \(\mathcal {H}\) derived as follows from a random hierarchy \({\mathscr {H}}\) on [0, 1] and a family \((U_j)\) of i.i.d. Uniform [0,1] random variables independent of \({\mathscr {H}}\): let \(\mathcal {H}\) comprise all sets of the form \(\{j:U_j \in B\}\) as B ranges over the members of \({\mathscr {H}}\).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Aldous, D.: The continuum random tree. I. Ann. Probab. 19(1), 1–28 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  2. Aldous, D.: The continuum random tree. II. An overview. In Stochastic analysis (Durham, 1990), volume 167 of London Math. Soc. Lecture Note Ser., pp. 23–70. Cambridge University Press, Cambridge (1991)

  3. Aldous, D.: The continuum random tree. III. Ann. Probab. 21(1), 248–289 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  4. Aldous, D.: Brownian excursions, critical random graphs and the multiplicative coalescent. Ann. Probab. 25(2), 812–854 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  5. Aldous, D., Krikun, M., Popovic, L.: Stochastic models for phylogenetic trees on higher-order taxa. J. Math. Biol. 56(4), 525–557 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Aldous, D., Pitman, J.: A family of random trees with random edge lengths. Random Struct. Algorithms 15(2), 176–195 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  7. Aldous, D.J.: Representations for partially exchangeable arrays of random variables. J. Multivar. Anal. 11(4), 581–598 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  8. Aldous, D.J.: Exchangeability and related topics. In: École d’été de probabilités de Saint-Flour, XIII—1983, vol. 1117 of Lecture Notes in Math., pp. 1–198. Springer, Berlin (1985)

  9. Aldous, D.J.: Stochastic models and descriptive statistics for phylogenetic trees, from Yule to today. Stat. Sci. 16(1), 23–34 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  10. Austin, T.: On exchangeable random variables and the statistics of large graphs and hypergraphs. Probab. Surv. 5, 80–145 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  11. Austin, T., Panchenko, D.: A hierarchical version of the de Finetti and Aldous–Hoover representations. Probab. Theory Relat. Fields 159(3–4), 809–823 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  12. Berestycki, J.: Ranked fragmentations. ESAIM Probab. Stat. 6, 157–175 (2002). (electronic)

  13. Berestycki, J., Berestycki, N., Schweinsberg, J.: Small-time behavior of beta coalescents. Ann. Inst. Henri Poincaré Probab. Stat. 44(2), 214–238 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  14. Bertoin, J.: A fragmentation process connected to Brownian motion. Probab. Theory Relat. Fields 117(2), 289–301 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  15. Bertoin, J.: Eternal additive coalescents and certain bridges with exchangeable increments. Ann. Probab. 29(1), 344–360 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  16. Bertoin, J.: Self-similar fragmentations. Ann. Inst. H. Poincaré Probab. Stat. 38(3), 319–340 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  17. Bertoin, J.: Random covering of an interval and a variation of Kingman’s coalescent. Random Struct. Algorithms 25(3), 277–292 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  18. Bertoin, J.: Random Fragmentation and Coagulation Processes, Cambridge Studies in Advanced Mathematics, vol. 102. Cambridge University Press, Cambridge (2006)

    Book  MATH  Google Scholar 

  19. Bertoin, J.: Homogeneous Multitype Fragmentations. In: In and out of equilibrium. 2, vol. 60 of Progr. Probab., pp. 161–183. Birkhäuser, Basel (2008)

  20. Bertoin, J., Le Gall, J.-F.: Stochastic flows associated to coalescent processes. Probab. Theory Relat. Fields 126(2), 261–288 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  21. Bertoin, J., Rouault, A.: Discretization methods for homogeneous fragmentations. J. Lond. Math. Soc. (2) 72(1), 91–109 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  22. Blei, D.M., Griffiths, T.L., Jordan, M.I.: The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies. J. ACM 57(2), 7 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  23. Blundell, C., Teh, Y.W., Heller, K.A.: Bayesian rose trees. In: Proceedings of the International Conference on Uncertainty in Artificial Intelligence (2010)

  24. Chen, B., Winkel, M.: Restricted exchangeable partitions and embedding of associated hierarchies in continuum random trees. Ann. Inst. Henri Poincaré Probab. Stat. 49(3), 839–872 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  25. Diaconis, P., Freedman, D.: de Finetti’s theorem for Markov chains. Ann. Probab. 8(1), 115–130 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  26. Diaconis, P., Freedman, D.: Finite exchangeable sequences. Ann. Probab. 8(4), 745–764 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  27. Diaconis, P., Janson, S.: Graph limits and exchangeable random graphs. Rend. Mat. Appl. (7) 28(1), 33–61 (2008)

    MathSciNet  MATH  Google Scholar 

  28. Donnelly, P., Evans, S.N., Fleischmann, K., Kurtz, T.G., Zhou, X.: Continuum-sites stepping-stone models, coalescing exchangeable partitions and random trees. Ann. Probab. 28(3), 1063–1110 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  29. Donnelly, P., Joyce, P.: Consistent ordered sampling distributions: characterization and convergence. Adv. Appl. Probab. 23(2), 229–258 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  30. Donnelly, P., Kurtz, T.G.: Particle representations for measure-valued population models. Ann. Probab. 27(1), 166–205 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  31. Duquesne, T.: A limit theorem for the contour process of conditioned Galton–Watson trees. Ann. Probab. 31(2), 996–1027 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  32. Duquesne, T., Le Gall, J-F.: Random trees, Lévy processes and spatial branching processes. Astérisque, (281):vi+147 (2002)

  33. Durrett, R.: Probability: Theory and Examples, third edn. Thomson, Belmont (2005)

    MATH  Google Scholar 

  34. Evans, S.N.: Kingman’s coalescent as a random metric space. In: Stochastic models (Ottawa, ON, 1998), vol. 26 of CMS Conference Proceedings, pp. 105–114. American Mathematical Society, Providence, RI (2000)

  35. Evans, S.N.: Probability and real trees, vol. 1920 of Lecture Notes in Mathematics. Springer, Berlin, 2008. Lectures from the 35th Summer School on Probability Theory held in Saint-Flour, July 6–23 (2005)

  36. Evans, S.N., Grübel, R., Wakolbinger, A.: Doob–Martin boundary of Rémy’s tree growth chain. Ann. Probab. 45(1), 225–277 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  37. Freedman, D., Diaconis, P.: de Finetti’s theorem for symmetric location families. Ann. Stat. 10(1), 184–189 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  38. Gelbaum, B.R., Olmsted, J.M.H.: Counterexamples in Analysis. Dover Publications, Inc., Mineola, NY, (2003). Corrected reprint of the second (1965) edition

  39. Ghahramani, Z., Jordan, M.I., Adams, R.P.: Tree-structured stick breaking for hierarchical data. In: Advances in neural information processing systems, pp. 19–27 (2010)

  40. Gnedin, A., Iksanov, A., Möhle, M.: On asymptotics of exchangeable coalescents with multiple collisions. J. Appl. Probab. 45(4), 1186–1195 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  41. Gnedin, A.., Olshanski, G.: A \(q\)-analogue of de Finetti’s theorem. Electron. J. Combin. 16(1):Research Paper 78, 16 (2009)

  42. Gnedin, A.V.: On a class of exchangeable sequences. Stat. Probab. Lett. 25(4), 351–355 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  43. Gnedin, A.V.: The representation of composition structures. Ann. Probab. 25(3), 1437–1450 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  44. Goldschmidt, C., Martin, J.B.: Random recursive trees and the Bolthausen–Sznitman coalescent. Electron. J. Probab 10(21), 718–745 (2005). (electronic)

  45. Gufler, S.: A representation for exchangeable coalescent trees and generalized tree-valued fleming-viot processes. arXiv:1608.08074v2 [math.PR], October (2016)

  46. Haas, B., Miermont, G.: The genealogy of self-similar fragmentations with negative index as a continuum random tree. Electron. J. Probab. 9(4), 57–97 (2004). (electronic)

  47. Haas, B., Miermont, G., Pitman, J., Winkel, M.: Continuum tree asymptotics of discrete fragmentations and applications to phylogenetic models. Ann. Probab. 36(5), 1790–1837 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  48. Haas, B., Pitman, J., Winkel, M.: Spinal partitions and invariance under re-rooting of continuum random trees. Ann. Probab. 37(4), 1381–1411 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  49. Heller, K.A., Ghahramani, Z.: Bayesian hierarchical clustering. In: Proceedings of the 22nd International Conference on Machine learning, pp. 297–304. ACM (2005)

  50. Hewitt, E., Savage, L.J.: Symmetric measures on Cartesian products. Trans. Am. Math. Soc. 80, 470–501 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  51. Hirth, U.: Exchangeable random ordered trees by positive definite functions. J. Theor. Probab. 16(2), 339–344 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  52. Hirth, U., Ressel, P.: Random partitions by semigroup methods. Semigroup Forum 59(1), 126–140 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  53. Hirth, U., Ressel, P.: Exchangeable random orders and almost uniform distributions. J. Theor. Probab. 13(3), 609–634 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  54. Jacka, S., Warren, J.: Random orderings of the integers and card shuffling. Stoch. Process. Appl. 117(6), 708–719 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  55. Janson, S.: Poset limits and exchangeable random posets. Institut Mittag-Leffler preprint 02 (2009)

  56. Kallenberg, O.: Exchangeable random measures in the plane. J. Theor. Probab. 3(1), 81–136 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  57. Kallenberg, O.: Probabilistic Symmetries and Invariance Principles. Probability and its Applications New York. Springer, New York (2005)

    MATH  Google Scholar 

  58. Kerov, S.V.: Asymptotic representation theory of the symmetric group and its applications in analysis, vol. 219 of Translations of Mathematical Monographs. American Mathematical Society, Providence, RI: Translated from the Russian manuscript by N, vol. Tsilevich. With a foreword by A. Vershik and comments by G, Olshanski (2003)

  59. Kingman, J.F.C.: The representation of partition structures. J. Lond. Math. Soc. (2) 18(2), 374–380 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  60. Kingman, J.F.C.: The coalescent. Stoch. Process. Appl. 13(3), 235–248 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  61. Kingman, J.F.C.: Exchangeability and the evolution of large populations. In: Exchangeability in probability and statistics (Rome, 1981), pp. 97–112. North-Holland, Amsterdam (1982)

  62. Le Gall, J.-F.: Random trees and applications. Probab. Surv. 2, 245–311 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  63. Meeds, E.W., Ross, D., Zemel, R.S., Roweis, S.T., et al.: Learning stick-figure models using nonparametric bayesian priors over trees. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008)

  64. Möhle, M., Sagitov, S.: A classification of coalescent processes for haploid exchangeable population models. Ann. Probab. 29(4), 1547–1562 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  65. Pemantle, R.: Automorphism invariant measures on trees. Ann. Probab. 20(3), 1549–1566 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  66. Pitman, J.: Combinatorial stochastic processes, vol. 1875 of Lecture Notes in Mathematics. Springer-Verlag, Berlin, (2006). Lectures from the 32nd Summer School on Probability Theory held in Saint-Flour, July 7–24, 2002, With a foreword by Jean Picard

  67. Pitman, J.: Exchangeable and partially exchangeable random partitions. Probab. Theory Relat. Fields 102(2), 145–158 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  68. Pitman, J., Winkel, M.: Regenerative tree growth: binary self-similar continuum random trees and Poisson–Dirichlet compositions. Ann. Probab. 37(5), 1999–2041 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  69. Ressel, P.: Subdiagonal and almost uniform distributions. Electron. Commun. Probab. 7, 97–101 (2002). (electronic)

    Article  MathSciNet  MATH  Google Scholar 

  70. Ressel, P.: Exchangeability and semigroups. Rend. Mat. Appl. (7) 28(1), 63–81 (2008)

    MathSciNet  MATH  Google Scholar 

  71. Sagitov, S.: The general coalescent with asynchronous mergers of ancestral lines. J. Appl. Probab. 36(4), 1116–1125 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  72. Stephenson, R.: General fragmentation trees. Electron. J. Probab. 18(101), 45 (2013)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Washington, Box 354350, Seattle, WA, 98195, USA

    Noah Forman

  2. Google, Mountain View, CA, USA

    Chris Haulk

  3. Statistics Department, Evans Hall, University of California, Berkeley, Berkeley, CA, 94720, USA

    Jim Pitman

Authors
  1. Noah Forman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chris Haulk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jim Pitman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Noah Forman.

Additional information

Research supported in part by NSF Grants DMS-0806118 and DMS-1444084 and EPSRC Grant EP/K029797/1.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Forman, N., Haulk, C. & Pitman, J. A representation of exchangeable hierarchies by sampling from random real trees. Probab. Theory Relat. Fields 172, 1–29 (2018). https://doi.org/10.1007/s00440-017-0799-4

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00440-017-0799-4

Keywords

Mathematics Subject Classification

Search

Navigation