Lines of Descent Under Selection


We review recent progress on ancestral processes related to mutation-selection models, both in the deterministic and the stochastic setting. We mainly rely on two concepts, namely, the killed ancestral selection graph and the pruned lookdown ancestral selection graph. The killed ancestral selection graph gives a representation of the type of a random individual from a stationary population, based upon the individual’s potential ancestry back until the mutations that define the individual’s type. The pruned lookdown ancestral selection graph allows one to trace the ancestry of individuals from a stationary distribution back into the distant past, thus leading to the stationary distribution of ancestral types. We illustrate the results by applying them to a prototype model for the error threshold phenomenon.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12


  1. 1.

    In the deterministic setting, it is more common to assume a neutral reproduction rate of 1 rather than 1 / 2. We work with the rate 1 / 2 here in order to obtain the pair coalescence rate of 1 that is standard in coalescence theory (see Sect. 3). Note that the deterministic dynamics (1) is unaffected by the neutral reproduction rate anyway.


  1. 1.

    Athreya, K.B., Ney, P.E.: Branching Processes. Springer, New York (1970)

    Google Scholar 

  2. 2.

    Athreya, S.R., Swart, J.M.: Branching-coalescing particle systems. Prob. Theory Relat. Fields 131, 376–414 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Baake, E., Baake, M., Wagner, H.: The Ising quantum chain is equivalent to a model of biological evolution, Phys. Rev. Lett. 78, 559–562 (1997), and Erratum Phys. Rev. Lett. 79, 1782 (1997)

  4. 4.

    Baake, E., Baake, M., Bovier, A., Klein, M.: An asymptotic maximum principle for essentially linear evolution models. J. Math. Biol. 50, 83–114 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  5. 5.

    Baake, E., Cordero, F., Hummel, S.: A probabilistic view on the deterministic mutation-selection equation: dynamics, equilibria, and ancestry via individual lines of descent. arXiv:1710.04573 (submitted)

  6. 6.

    Baake, E., Gabriel, W.: Biological evolution through mutation, selection, and drift: an introductory review. In: Stauffer, D. (ed.) Annual Reviews of Computational Physics, vol. 7, pp. 203–264. World Scientific, Singapore (2000)

    Google Scholar 

  7. 7.

    Baake, E., Georgii, H.-O.: Mutation, selection, and ancestry in branching models: a variational approach. J. Math. Biol. 54, 257–303 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  8. 8.

    Baake, E., Lenz, U., Wakolbinger, A.: The common ancestor type distribution of a \(\Lambda \)-Wright-Fisher process with selection and mutation. Electron. Commun. Probab. 21, 1–16 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Baake, E., Wakolbinger, A.: Feller’s contributions to mathematical biology. In: Schilling, R.L., Vondracek, Z., Woyczyński, W.A. (eds.) Selected Works of William Feller, vol. 2, pp. 25–43. Springer, Berlin (2015)

    Google Scholar 

  10. 10.

    Bürger, R.: The Mathematical Theory of Selection, Recombination, and Mutation. Wiley, Chichester (2000)

    Google Scholar 

  11. 11.

    Cordero, F.: The deterministic limit of the Moran model: a uniform central limit theorem. Markov Processes Relat. Fields 23, 313–324 (2017)

    MathSciNet  MATH  Google Scholar 

  12. 12.

    Cordero, F.: Common ancestor type distribution: a Moran model and its deterministic limit. Stoch. Proc. Appl. 127, 590–621 (2017)

    MathSciNet  Article  MATH  Google Scholar 

  13. 13.

    Crow, J.F., Kimura, M.: Some genetic problems in natural populations. In: J. Neyman (ed.) Proceedings of the Third Berkeley Symposium on Probability and Mathematical Statistics, vol. 4. University of California Press, Berkeley, pp. 1–22 (1956)

  14. 14.

    Crow, J.F., Kimura, M.: An Introduction to Population Genetics Theory. Harper & Row, New York (1970)

    Google Scholar 

  15. 15.

    Donnelly, P., Kurtz, T.G.: Genealogical processes for Fleming-Viot models with selection and recombination. Ann. Appl. Probab. 9, 1091–1148 (1999)

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Durrett, R.: Probability Models for DNA Sequence Evolution, 2nd edn. Springer, New York (2008)

    Google Scholar 

  17. 17.

    Eigen, M.: Selforganization of matter and the evolution of biological macromolecules. Naturwiss. 58, 465–523 (1971)

    ADS  Article  Google Scholar 

  18. 18.

    Eigen, M., McCaskill, J., Schuster, P.: The molecular quasi-species. Adv. Chem. Phys. 75, 149–263 (1989)

    Google Scholar 

  19. 19.

    Ethier, S.N., Kurtz, T.G.: Markov Processes: Characterization and Convergence. Wiley, New York (1986). reprint 2005

    Google Scholar 

  20. 20.

    Ewens, W.J.: Mathematical Population Genetics I. Theoretical Introduction, 2nd edn. Springer, New York (2004)

    Google Scholar 

  21. 21.

    Fearnhead, P.: The common ancestor at a nonneutral locus. J. Appl. Probab. 39, 38–54 (2002)

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Feller, W.: Diffusion processes in genetics. In: J. Neyman (ed.) Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability 1950, pp. 227–246. University of California Press, Berkeley (1951)

  23. 23.

    Fisher, R.A.: The Genetical Theory of Natural Selection. Clarendon Press, Oxford (1930)

    Google Scholar 

  24. 24.

    Garske, T.: Error thresholds in a mutation-selection model with Hopfield-type fitness. Bull. Math. Biol. 68, 1715–1746 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  25. 25.

    Garske, T., Grimm, U.: Maximum principle and mutation thresholds for four-letter sequence evolution. Bull. Math. Biol. 66, 397–421 (2004)

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Georgii, H.O., Baake, E.: Supercritical multitype branching processes: the ancestral types of typical individuals. Adv. Appl. Probab. 35, 1090–1110 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  27. 27.

    Hermisson, J., Redner, O., Wagner, H., Baake, E.: Mutation-selection balance: ancestry, load, and maximum principle. Theor. Popul. Biol. 62, 9–46 (2002)

    Article  MATH  Google Scholar 

  28. 28.

    Hoppe, F.: Polya-like urns and the Ewens’ sampling formula. J. Math. Biol. 20, 91–94 (1984)

    MathSciNet  Article  MATH  Google Scholar 

  29. 29.

    Jagers, P., Nerman, O.: The stable doubly infinite pedigree process of supercritical branching populations. Z. für Wahrscheinlichkeitstheorie und verwandte Gebiete 65, 445–460 (1984)

    Article  MATH  Google Scholar 

  30. 30.

    Jagers, P.: General branching processes as Markov fields. Stoch. Proc. Appl. 32, 183–242 (1989)

    MathSciNet  Article  MATH  Google Scholar 

  31. 31.

    Jagers, P.: Stabilities and instabilities in population dynamics. J. Appl. Probab. 29, 770–780 (1992)

    MathSciNet  Article  MATH  Google Scholar 

  32. 32.

    Jansen, S., Kurt, N.: On the notion(s) of duality for Markov processes. Probab. Surv. 11, 59–120 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  33. 33.

    Kimura, M.: On the probability of fixation of mutant genes in a population. Genetics 47, 713719 (1962)

    Google Scholar 

  34. 34.

    Kingman, J.F.C.: The coalescent. Stoch. Proc. Appl. 13, 235–248 (1982)

    MathSciNet  Article  MATH  Google Scholar 

  35. 35.

    Kingman, J.F.C.: On the genealogy of large populations. J. Appl. Probab. 19A, 27–43 (1982)

    MathSciNet  Article  MATH  Google Scholar 

  36. 36.

    Krone, S.M., Neuhauser, C.: Ancestral processes with selection. Theor. Popul. Biol. 51, 210–237 (1997)

    Article  MATH  Google Scholar 

  37. 37.

    Kurtz, T.G.: Limit theorems for sequences of jump Markov processes approximating ordinary differential processes. J. Appl. Probab. 8, 344–356 (1971)

    MathSciNet  Article  MATH  Google Scholar 

  38. 38.

    Leibler, S., Kussell, E.: Individual histories and selection in heterogeneous populations. Proc. Natl. Acad. Sci. USA 107, 13183–13188 (2010)

    ADS  Article  Google Scholar 

  39. 39.

    Lenz, U., Kluth, S., Baake, E., Wakolbinger, A.: Looking down in the ancestral selection graph: a probabilistic approach to the common ancestor type distribution. Theor. Popul. Biol. 103, 27–37 (2015)

    Article  MATH  Google Scholar 

  40. 40.

    Leuthäusser, I.: An exact correspondence between Eigen’s evolution model and a two-dimensional Ising system. J. Chem. Phys. 84, 1884–1885 (1986)

    ADS  MathSciNet  Article  Google Scholar 

  41. 41.

    Leuthäusser, I.: Statistical mechanics of Eigen’s evolution model. J. Stat. Phys. 48, 343–360 (1987)

    ADS  MathSciNet  Article  Google Scholar 

  42. 42.

    Liggett, T.M.: Continuous Time Markov Processes: An Introduction. AMS, Providence (2010)

    Google Scholar 

  43. 43.

    Malécot, G.: Les Mathématiques de l’Hérédité. Masson, Paris (1948)

    Google Scholar 

  44. 44.

    Moran, P.A.P.: Random processes in genetics. Proc. Camb. Philos. Soc. 54, 60–71 (1958)

    ADS  MathSciNet  Article  MATH  Google Scholar 

  45. 45.

    Nagylaki, T.: Gustave Malécot and the transition from classical to modern population genetics. Genetics 122, 253–268 (1989)

    Google Scholar 

  46. 46.

    Peliti, L.: Quasispecies evolution in general mean-field landscapes. Europhys. Lett. 57, 745–751 (2002)

    ADS  Article  Google Scholar 

  47. 47.

    Sughiyama, Y., Kobayashi, T.J.: Steady-state thermodynamics for population growth in fluctuating environments. Phys. Rev. E 95, 012131 (2017)

    ADS  Article  Google Scholar 

  48. 48.

    Tarazona, P.: Error threshold for molecular quasispecies as phase transition: from simple landscapes to spin glass models. Phys. Rev. A 45, 6038–6050 (1992)

    ADS  Article  Google Scholar 

  49. 49.

    Taylor, J.E.: The common ancestor process for a Wright-Fisher diffusion. Electron. J. Probab. 12, 808–847 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  50. 50.

    Wright, S.: Evolution in Mendelian populations. Genetics 16, 97–159 (1931)

    Google Scholar 

Download references


It is our pleasure to thank Fernando Cordero, Sebastian Hummel, and Ute Lenz for fruitful discussions. This project received financial support from Deutsche Forschungsgemeinschaft (Priority Programme SPP 1590 Probabilistic Structures in Evolution, Grant Nos. BA 2469/5-1 and WA 967/4-1).

Author information



Corresponding author

Correspondence to Ellen Baake.

Additional information

This article is dedicated to the memory of Hans-Otto Georgii, whose joint work with the first author on ancestral lines in multitype branching processes [7, 26] laid foundations and provided motivation for the line of research reviewed here.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Baake, E., Wakolbinger, A. Lines of Descent Under Selection. J Stat Phys 172, 156–174 (2018).

Download citation


  • Mutation-selection model
  • Killed ancestral selection graph
  • Pruned lookdown ancestral selection graph
  • Error threshold

Mathematics Subject Classification

  • 60J27
  • 60J75
  • 92D15
  • 05C80