Journal of Mathematical Biology

, Volume 54, Issue 2, pp 257–303 | Cite as

Mutation, selection, and ancestry in branching models: a variational approach

  • Ellen BaakeEmail author
  • Hans-Otto Georgii


We consider the evolution of populations under the joint action of mutation and differential reproduction, or selection. The population is modelled as a finite-type Markov branching process in continuous time, and the associated genealogical tree is viewed both in the forward and the backward direction of time. The stationary type distribution of the reversed process, the so-called ancestral distribution, turns out as a key for the study of mutation–selection balance. This balance can be expressed in the form of a variational principle that quantifies the respective roles of reproduction and mutation for any possible type distribution. It shows that the mean growth rate of the population results from a competition for a maximal long-term growth rate, as given by the difference between the current mean reproduction rate, and an asymptotic decay rate related to the mutation process; this tradeoff is won by the ancestral distribution. We then focus on the case when the type is determined by a sequence of letters (like nucleotides or matches/mismatches relative to a reference sequence), and we ask how much of the above competition can still be seen by observing only the letter composition (as given by the frequencies of the various letters within the sequence). If mutation and reproduction rates can be approximated in a smooth way, the fitness of letter compositions resulting from the interplay of reproduction and mutation is determined in the limit as the number of sequence sites tends to infinity. Our main application is the quasispecies model of sequence evolution with mutation coupled to reproduction but independent across sites, and a fitness function that is invariant under permutation of sites. In this model, the fitness of letter compositions is worked out explicitly. In certain cases, their competition leads to a phase transition.


Mutation–selection models Branching processes Quasispecies model Variational analysis Large deviations 

Mathematics Subject Classification (2000)

92D15 60J80 60F10 90C46 15A18 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Akin E. (1979): The Geometry of Population Genetics. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  2. 2.
    Athreya K.B., Ney P.E. (1972): Branching Processes. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  3. 3.
    Baake E., Baake M., Bovier A., Klein M. (2005): An asymptotic maximum principle for essentially nonlinear evolution models. J. Math. Biol. 50, 83–114; ArXiv:q-bio.PE/0311020CrossRefMathSciNetGoogle Scholar
  4. 4.
    Baake E., Wagner H. (2001): Mutation–selection models solved exactly with methods from statistical mechanics. Genet. Res. 78, 93–117CrossRefGoogle Scholar
  5. 5.
    Bürger R. (2000): The Mathematical Theory of Selection, Recombination, and Mutation. Wiley, ChichesterzbMATHGoogle Scholar
  6. 6.
    Crow J. F., Kimura M. (1970): An Introduction to Population Genetics Theory. Harper & Row, New YorkGoogle Scholar
  7. 7.
    Dembo A., Zeitouni O. (1998): Large Deviations: Techniques and Applications. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  8. 8.
    Edwards A. (2002): The fundamental theorem of natural selection. Theor. Pop. Biol. 61, 335–337CrossRefGoogle Scholar
  9. 9.
    Eigen M. (1971): Selforganization of matter and the evolution of biological macromolecules. Naturwiss. 58, 465–523CrossRefGoogle Scholar
  10. 10.
    Eigen M., McCaskill J., Schuster P. (1989): The molecular quasi-species. Adv. Chem. Phys. 75, 149–263Google Scholar
  11. 11.
    Ewens W. (2004): Mathematical Population Genetics, 2nd edn. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  12. 12.
    Ewens W., Grant G. (2005): Statistical Methods in Bioinformatics, 2nd edn. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  13. 13.
    Garske, T.: Error thresholds in a mutation-selection model with Hopfield-type fitness. Bull. Math. Biol. (in press) arXiv:q-bio.PE/0505056. DOI 10.1007/s11538-006-9072-1Google Scholar
  14. 14.
    Garske T., Grimm U. (2004): A maximum principle for the mutation-selection equilibrium of nucleotide sequences. Bull. Math. Biol. 66, 397–421; arXiv:physics/0303053v2CrossRefMathSciNetGoogle Scholar
  15. 15.
    Georgii H.-O., Baake E. (2003): Multitype branching processes: the ancestral types of typical individuals. Adv. Appl. Prob. 35, 1090–1110;arXiv:math.PR/0302049CrossRefMathSciNetGoogle Scholar
  16. 16.
    Gerland U., Hwa T. (2002): On the selection and evolution of regulatory DNA motifs. J. Mol. Evol. 55, 386–400CrossRefGoogle Scholar
  17. 17.
    Hein J., Schierup M., Wiuf C. (2005): Gene genealogies, variation and evolution : a primer in coalescent theory. Oxford University Press, OxfordzbMATHGoogle Scholar
  18. 18.
    Hermisson J., Redner O., Wagner H., Baake E. (2002): Mutation–selection balance: Ancestry, load, and maximum principle. Theor. Pop. Biol. 62, 9–46; arXiv:cond-mat/0202432CrossRefGoogle Scholar
  19. 19.
    Hofbauer J. (1985): The selection mutation equation. J. Math. Biol. 23, 41–53MathSciNetGoogle Scholar
  20. 20.
    den Hollander F. (2000). Large Deviations. AMS, Providence, RIzbMATHGoogle Scholar
  21. 21.
    Jagers P., Nerman O. (1984): The stable doubly infinite pedigree process of supercritical branching populations. Z für Wahrscheinlichkeitstheorie und verwandte Gebiete 65, 445–460CrossRefMathSciNetGoogle Scholar
  22. 22.
    Jagers P. (1989): General branching processes as Markov fields. Stoch. Proc. Appl. 32, 183–242CrossRefMathSciNetGoogle Scholar
  23. 23.
    Jagers P. (1992): Stabilities and instabilities in population dynamics. J. Appl. Prob. 29, 770–780CrossRefMathSciNetGoogle Scholar
  24. 24.
    Kamp C. (2003): A quasispecies approach to viral evolution in the context of an adaptive immune system. Microbes Infect. 5, 1397–1405CrossRefGoogle Scholar
  25. 25.
    Karlin K.S., Taylor H.M. (1975): A first course in stochastic processes, 2nd edn. Academic Press, San DiegozbMATHGoogle Scholar
  26. 26.
    Kemeny J.G., Snell J.L. (1981): Finite Markov Chains. Springer, Berlin Heidelberg New YorkGoogle Scholar
  27. 27.
    Kesten H., Stigum B.P. (1966): A limit theorem for multidimensional Galton-Watson processes. Ann. Math. Statist. 37, 1211–1233MathSciNetGoogle Scholar
  28. 28.
    Kurtz T., Lyons R., Pemantle R., Peres Y. (1997): A conceptual proof of the Kesten-Stigum theorem for multi-type branching processes. In: Athreya K.B., Jagers P (eds) Classical and Modern Branching Processes. pp. 181–185, Springer, New YorkGoogle Scholar
  29. 29.
    Lindvall T. (1992): Lectures on the Coupling Method. Wiley, New YorkzbMATHGoogle Scholar
  30. 30.
    Lyons R., Pemantle R., Peres Y. (1995): Conceptual proofs of LlogL criteria for mean behaviour of branching processes. Ann. Prob. 23, 1125–1138MathSciNetGoogle Scholar
  31. 31.
    Mitrinovic D. (1970): Analytic inequalities. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  32. 32.
    Rockafellar R.T. (1970): Convex Analysis. Princeton University Press, PrincetonzbMATHGoogle Scholar
  33. 33.
    Stannat W. (2004): On the convergence of genetic algorithms—a variational approach. Probab. Theory Relat. Fields 129, 113–132CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  1. 1.Faculty of TechnologyBielefeld UniversityBielefeldGermany
  2. 2.Department of MathematicsUniversity of MunichMünchenGermany

Personalised recommendations