Journal of Mathematical Biology

, Volume 73, Issue 3, pp 627–664 | Cite as

Mutational pattern of a sample from a critical branching population

  • Cécile Delaporte
  • Guillaume Achaz
  • Amaury Lambert


We study a universal object for the genealogy of a sample in populations with mutations: the critical birth-death process with Poissonian mutations, conditioned on its population size at a fixed time horizon. We show how this process arises as the law of the genealogy of a sample in a large class of nearly critical branching populations with rare mutations at birth, namely populations converging, in a large population asymptotic, towards the continuum random tree. We extend this model to populations with random foundation times, with (potentially improper) prior distributions \(g_i:x\mapsto x^{-i}\), \(i\in \mathbb Z_+\), including the so-called uniform (\(i=0\)) and log-uniform (\(i=1\)) priors. We first investigate the mutational patterns arising from these models, by studying the site frequency spectrum of a sample with fixed size, i.e. the number of mutations carried by k individuals in the sample. Explicit formulae for the expected frequency spectrum of a sample are provided, in the cases of a fixed foundation time, and of a uniform and log-uniform prior on the foundation time. Second, we establish the convergence in distribution, for large sample sizes, of the (suitably renormalized) tree spanned by the sample with prior \(g_i\) on the time of origin. We finally prove that the limiting genealogies with different priors can all be embedded in the same realization of a given Poisson point measure.


Critical birth-death process Sampling Coalescent point process Site frequency spectrum Infinite-site model  Poisson point measure Invariance principle 

Mathematics Subject Classification

Primary 92D10 60J80 Secondary 92D25 60F17 60G55 60G57 60J85 


  1. Aldous D (1993) The continuum random tree III. Ann Probab 248–289Google Scholar
  2. Aldous D, Popovic L (2005) A critical branching process model for biodiversity. Adv Appl Probab 37(4):1094–1115MathSciNetCrossRefMATHGoogle Scholar
  3. Champagnat N, Lambert A (2012a) Splitting trees with neutral Poissonian mutations I: Small families. Stoch Process Appl 122(3):1003–1033MathSciNetCrossRefMATHGoogle Scholar
  4. Champagnat N, Lambert A (2012b) Splitting trees with neutral Poissonian mutations II: Largest and Oldest families. Stoch Process Appl 123(4):1368–1414MathSciNetCrossRefMATHGoogle Scholar
  5. Champagnat N, Lambert A, Richard M (2012) Birth and death processes with neutral mutations. Int J Stoch Anal 2012:569081. doi:10.1155/2012/569081
  6. David H, Nagaraja H (2003) Order statistics. Wiley Online Library, NYGoogle Scholar
  7. Delaporte C (2013) Lévy processes with marks II : Invariance principle for branching processes with mutations. Eprint arXiv:1305.6491
  8. Durrett R (2008) Probability models for DNA sequence evolution, 2nd edn. Springer, NYGoogle Scholar
  9. Ewens WJ (1972) The sampling theory of selectively neutral alleles. Theor Popul Biol 3(1):87–112MathSciNetCrossRefMATHGoogle Scholar
  10. Fu YX (1995) Statistical properties of segregating sites. Theor Popul Biol 48:172–197CrossRefMATHGoogle Scholar
  11. Geiger J (1996) Size-biased and conditioned random splitting trees. Stoch Process Appl 65(2):187–207MathSciNetCrossRefMATHGoogle Scholar
  12. Geiger J, Kersting G (1997) Depth–first search of random trees, and Poisson point processes in Classical and modern branching processes (Minneapolis, 1994). IMA Math Appl 84Google Scholar
  13. Gernhard T (2008) New analytic results for speciation times in neutral models. Bull Math Biol 70(4):1082–1097MathSciNetCrossRefMATHGoogle Scholar
  14. Hoehna S, Stadler T, Ronquist F, Britton T (2008) Inferring speciation and extinction rates under different sampling schemes. Mol Biol Evol 28(9):2577–2589Google Scholar
  15. Kallenberg O (2002) Foundations of modern probability, 2nd edn. Springer, NYGoogle Scholar
  16. Kimura M (1969) The number of heterozygous nucleotide sites maintained in a finite population due to steady flux of mutations. Genetics 61(4):893Google Scholar
  17. Kingman J (1982a) On the genealogy of large populations. J Appl Probab 19:27–43MathSciNetCrossRefMATHGoogle Scholar
  18. Kingman J (1982b) The coalescent. Stoch Process Appl 13(3):235–248MathSciNetCrossRefMATHGoogle Scholar
  19. Lambert A (2009) The allelic partition for coalescent point processes. Markov Proc Relat Fields 15:359–386MathSciNetMATHGoogle Scholar
  20. Lambert A (2010) The contour of splitting trees is a Lévy process. Ann Probab 38(1):348–395Google Scholar
  21. Lambert A, Stadler T (2013) Birth-death models and coalescent point processes: the shape and probability of reconstructed phylogenies. Theor Popul Biol 90:118–128CrossRefMATHGoogle Scholar
  22. Le Gall JF (2005) Random trees and applications. Probab Surv 2:245–311MathSciNetCrossRefMATHGoogle Scholar
  23. Popovic L (2004) Asymptotic genealogy of a critical branching process. Ann Appl Probab 14(4):2120–2148MathSciNetCrossRefMATHGoogle Scholar
  24. Richard M (2014) Splitting trees with neutral mutations at birth. Stoch Process Appl 124(10):3206–3230MathSciNetCrossRefMATHGoogle Scholar
  25. Stadler T (2008) Lineages-through-time plots of neutral models for speciation. Math Biosci 216(2):163–171MathSciNetCrossRefMATHGoogle Scholar
  26. Stadler T (2009) On incomplete sampling under birth-death models and connections to the sampling-based coalescent. J Theor Biol 261(1):58–66MathSciNetCrossRefGoogle Scholar
  27. Wakeley J (2008) Coalescent theory: an introduction. Roberts and Company, South WalesGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Cécile Delaporte
    • 1
    • 2
  • Guillaume Achaz
    • 2
    • 3
    • 4
  • Amaury Lambert
    • 1
    • 2
  1. 1.Laboratoire de Probabilités et Modèles AléatoiresUMR 7599 CNRS and UPMC Univ Paris 06ParisFrance
  2. 2.Centre Interdisciplinaire de Recherche en BiologieUMR 7241 CNRS and Collège de FranceParisFrance
  3. 3.Évolution Paris-SeineUMR 7138 CNRS and UPMC Univ Paris 06ParisFrance
  4. 4.Atelier de Bioinformatique, MNHNParisFrance

Personalised recommendations