Abstract
Background
Red Queen dynamics are defined as long term coevolutionary dynamics, often with oscillations of genotype abundances driven by fluctuating selection in hostparasite systems. Much of our current understanding of these dynamics is based on theoretical concepts explored in mathematical models that are mostly (i) deterministic, inferring an infinite population size and (ii) evolutionary, thus ecological interactions that change population sizes are excluded. Here, we recall the different mathematical approaches used in the current literature on Red Queen dynamics. We then compare models from game theory (evo) and classical theoretical ecology models (ecoevo), that are all derived from individual interactions and are thus intrinsically stochastic. We assess the influence of this stochasticity through the time to the first loss of a genotype within a host or parasite population.
Results
The time until the first genotype is lost (“extinction time”), is shorter when ecological dynamics, in the form of a changing population size, is considered. Furthermore, when individuals compete only locally with other individuals extinction is even faster. On the other hand, evolutionary models with a fixed population size and competition on the scale of the whole population prolong extinction and therefore stabilise the oscillations. The stabilising properties of intraspecific competitions become stronger when population size is increased and the deterministic part of the dynamics gain influence. In general, the loss of genotype diversity can be counteracted with mutations (or recombination), which then allow the populations to recurrently undergo negative frequencydependent selection dynamics and selective sweeps.
Conclusion
Although the models we investigated are equal in their biological motivation and interpretation, they have diverging mathematical properties both in the derived deterministic dynamics and the derived stochastic dynamics. We find that models that do not consider intraspecific competition and that include ecological dynamics by letting the population size vary, lose genotypes – and thus Red Queen oscillations – faster than models with competition and a fixed population size.
Background
Diversity, induced by continuous coevolution can theoretically be maintained by the intense antagonistic relationship of hosts and parasites. This is the central part of the Red Queen hypothesis, verbally first formulated by van Valen in 1973 [1]. The hypothesis has been mathematically formulated in many models. However, owing to the modern usage of the term ‘Red Queen’ for different but related phenomena [2–9], the models have diverging foci and many lack the implementation of stochastic forces and ecological dynamics. A common synonym for the term Red Queen dynamics is fluctuating selection dynamics (FSD). Such fluctuations can be induced by coevolving hosts and parasites and, as one possibility, be driven by negative frequencydependent selection (NFDS), where host and parasite genotype abundances oscillate in time and every genotype can temporally be best adapted. In detail, since parasites are selected to target the most common resource, being a rare host genotype is advantageous. This temporary high fitness makes the genotype grow in relative abundance, but before it can take over the whole population, it is severely diminished by the profiting parasites genotypes, which target this now common host type. By contrast, in arms race dynamics (ARD) novel favoured genotypes spread in the entire population by recurrent selective sweeps. The terms NFDS and ARD are both referred to as Red Queen dynamics [10–12] and describe an ongoing coevolutionary change without approaching an equilibrium. In this paper, we use the term Red Queen dynamics for NFDS, as is commonly done in the literature, but return to other definitions of the Red Queen in the discussion.
Although Red Queen dynamics is a wellknown and frequently cited concept, there is only little evidence for the ubiquitous prevalence of long term Red Queen dynamics in nature – empirical challenges preclude the observation of more than a few subsequent oscillations, as these require a major amounts of intensive and challenging lab work [13–17]. Thus, most work on the actual long term temporal dynamics is theoretical, often dealing with evolutionary dynamics or epidemiological dynamics in a deterministic fashion. We have summarised some of the literature in the context of these assumptions in Table 1 (methods in the Additional file 1). Similar literature summaries exist with a focus on sexual vs. asexual reproduction [8] or hostparasite coevolution models [18]. Many theoretical studies build on evolutionary game theory [19] and a zerosum assumption, where the harm done to the host equals the benefit for the parasite, which was already envisioned by van Valen at the time. Some of the models are implemented with equations that describe both species’ dynamics (explicit hostparasite HP dynamics), other studies, especially on the evolution and maintenance of sexual reproduction (Red Queen Hypothesis) revert to epidemiological models (susceptibleinfected SI models), sometimes in the pursuit of including population dynamics. The present work focuses on evolutionary hostparasite models in comparison with ecoevolutionary models that include population dynamics without using SI models.
While many studies assess the occurrence of oscillating selection dynamics and show under what assumptions oscillations dominate [18, 20–27], only few studies include both ecological population dynamics and stochastic noise, although the combination of the two has been shown to result in a fast loss of genotypes in either population [28]. It has been difficult to derive a stochastic model that easily switches between constant and changing population size using a single parameter. For example Gokhale et al. [28] artificially normalised population size every few generations. Here, we take a different approach and compare the modelling framework of evolutionary game theory, where population size is constant by design, to ecoevolutionary dynamics from the field of theoretical ecology, where population size is inherently free to change over time. Our goal is not to present the one model that is the best description of reality, but to illustrate how different modelling assumptions can drive the results from such models.
Specifically, we use individualbased models, since ecological and evolutionary dynamics of populations are driven by events on the individual level. The models are based on haploid and asexual populations that live in a wellmixed environment where encounters are density dependent. Individuals are born, interact with other individuals of their own or opposing species and die. Generally, we will consider at least two genotypes and track the associated abundances H_{1},H_{2},P_{1},P_{2} and the total population sizes N_{H},N_{P} of hosts and parasites over time. We simulate the dynamics using a uniformly distributed initial standing genetic variation and the simple matching allele interaction profile, where parasites are highly specialised [29, 30] on a particular host genotype and identical in all other aspects. Yet, the way this interaction profile enters in the dynamical equations and thus defines fitness for the individual genotypes is very different between the models. In population dynamics models these events happen at constant rates and depending on the density of the interacting individuals. A similarly simple, yet completely different approach is the stochastic birthdeath process which tracks only the evolutionary dynamics. In each time step one individual is born, proportional to its current ‘fitness’ and another individual dies proportional to its density.
These models all produce Red Queen dynamics (NFDS) and we assess the robustness of those by measuring the time to extinction, which we define as the earliest time that any genotype from the initial genetic variation is lost in either population. Further, we consider the impact of the derived deterministic dynamics and the influence of ecology in the form of a populationsizechange on this extinction time. The time to extinction of a genotype represents the durability of the stochastic oscillations. Without the immigration or reemergence of extinct genotypes, the diversity of both populations declines in the long run.
Results
Evolutionary dynamics depict the change of relative genotype abundances over time and can be examined without keeping track of population size changes. However, it is well known that ecological dynamics can feed back on evolutionary dynamics. We want to understand this feedback in the context of Red Queen dynamics. To this end we compare models from evolutionary game theory, that do not include population size changes and theoretical ecology models that do. The models have been widely used in the literature and represent the simplest case of Red Queen dynamics with a matching allele interaction profile (for details see the “Methods” section below and Additional file 1).
The matchingallele hostparasite Red Queen dynamics in evolutionary and ecoevolutionary models
In an evolutionary birthdeath process one individual is born and another dies in each population, here host or parasite, and in each time step. Thereby, population size remains constant and the focus lies on the genotypic composition of a population. The Evo ^{+} and Evo processes (see Table 2 and in the “Methods” section for a definition) are such birthdeath processes [64, and references therein; 65]. Individuals are chosen to die randomly, but the individual that reproduces is chosen proportional to the fitness advantages of that genotype relative to other genotypes in the population. The fitness effects are imposed by the current state of the antagonist population and an interaction matrix. In the Evo ^{+} process, the fitness effect is normalised by the average fitness effect over the whole host population, which leads to a kind of intraspecific competition (+) while in the Evo process the difference in fitness effects is compared between a genotypeone individual and a genotypetwo individual, thus competition is pairwise. Because of the population size constraint, both processes can be analytically treated (see Additional file 1) when implemented in discrete time (prefix dt).
In models adapted from theoretical ecology the events of birth, death and interaction happen independently with external rates and, importantly, between populations (EcoEvo, comparable with the LotkaVolterra dynamics in [28]). Host and parasite individuals encounter one another based on their densities and if they match, an interaction is carried out with a constant rate upon which a host dies or a parasite reproduces. When competition between hosts (+) is included, the host population grows logistically with a carrying capacity K. The host population size N_{H} reaches the carrying capacity in the absence of parasites. However, in the presence of parasites the population size is smaller than K because of the additional mortality from parasites (see Table 3 for details on the parameters in all models).
Both Evo and EcoEvo modelling approaches are combined in an intermediate model with selfcontrolled, but not fixed, population size (Hybrid). The model is implemented as an individual interactions model, where reactions take place also between populations, but the rates of these events are taken from the game theory models: Host death and parasite birth happen according to the fitness effects, host death and parasite birth rates are then adapted dynamically to keep population size nearly constant.
From the derivations of the models (details in the “Methods” section and Additional file 1) some basic properties of the dynamics are obtained and summarised in Table 2. The evolutionary game theory models have a constant population size by design, whereas population size can change in all other models. The average behaviour of the individualbased stochastic processes is captured in the deterministic selection term and the noise term, which together determine the stochastic dynamics. The noise term is discussed in the “Results” section (Fig. 1). The role of intraspecific competition in the deterministic part is discussed in the “Discussion” section (Fig. 2).
In models with ecological dynamics genotypes die out faster
It is clear that populations with low total population sizes are more prone to genetic drift and the loss of genotypes than large populations. We now show that it is not only the population size but the possibility of population size change that speeds up this process. We exemplify our argument here, but provide a more detailed analysis below. As an example with two host genotypes and two parasite genotypes we select the Evo ^{+} process and the EcoEvo ^{+} process (Fig. 1). Starting with an equal abundance of genotypes we measure the time to the first loss of a genotype. When a genotype has died the population becomes monoclonal and oscillations are no longer possible. With the fixed population size in the Evo ^{+} process oscillations survive longer than in the EcoEvo ^{+} process with a changing population size. The evolutionary dynamics are similar and defined through the relative abundance of the types, but the population size change can speed up the frequency of event occurrences and increase the probability of extinction through the bottleneck effect when population sizes are low.
Intraspecific competition stabilises negative frequencydependent selection
The equations that define the stochastic process consist of a deterministic selection term and a noise term and represent the mean and variance of many individual simulations. Therefore, it is impossible to understand the stochastic model without making the deterministic dynamics clear. Furthermore, when population size is large, the stochastic process approaches the more manageable deterministic dynamics (details in the Additional file 1). The deterministic equations for all models from Table 2 have an internal coexistence fixed point, where both genotypes exist in a fixed ratio, which does not change over time. This point is only attractive, if starting with suitable initial compositions of genotypes the dynamics approach the state, in this case in the form of damped oscillations. The intraspecific competition (+) in the Evo ^{+} process and the EcoEvo ^{+} process result in such an attractive pull (Fig. 2). A second possibility is neutral stability, where genotype abundances oscillate with a constant amplitude and period, which depend on the initial abundances. These neutral cycles are produced by models where individuals only compete with other genotypes locally like in the Evo process with pairwise competition or the EcoEvo model with no intraspecific competition and the Hybrid model (Fig. 2).
In our models, the noise in the stochastic dynamics always leads to extinction, while deterministic dynamics never do. When population size is large enough to be impacted by the deterministic behaviour but stochastic noise still plays a role, the global competition models (+) show persisting Red Queen oscillations. The deterministic ‘pull’ and the stochastic ‘push’ balance [66], prolonging extinction times. For models with neutral oscillations (NFDS) in the deterministic dynamics stochastic effects will on average increase the amplitudes and push the trajectories to the edges of the space towards a faster extinction of genotypes.
The single simulations (Fig. 1) are only a snapshot and one specific realisation of the stochastic processes. Ideally, we would analytically derive general extinction times depending on the parameters of the model. Yet, to derive an exact analytical solution for this problem is extremely challenging. In addition to simulations, we have calculated the numerical (but exact) extinction times for low population sizes and provide an approximative method based on the averaged noise (see Additional file 1 for further details). These methods are limited to a subset of the models and can thus not be used for a comparison of all models, but only to support the computationally costly simulations which provide the now following main result.
The strength of random effects depends on the model properties
We simulate 1000 replicates for a set of parameter combinations of the models with two genotypes in each population and record the time it takes until one genotype has died out. As a general trend, the more constrained a population size is, the longer oscillations survive (higher extinction times in Fig. 3). This holds true for small to intermediate population sizes – note a similar vertical order of extinction times to the ordering of models by population size constraint in Table 2. When population sizes become larger and the deterministic model properties gain influence, models with competition terms (+, stasis, compare with Fig. 2) have higher extinction times and therefore more stable Red Queen oscillations.
By design, the discrete time (dt) processes have much higher extinction times and are thus not directly comparable to the continuous time simulations. A scaling would be possible for equal population sizes, but with different extinction routes and N_{H}≠N_{P} no such factor can be derived. The dtEvo ^{+} and dtEvo extinction times in Fig. 3 can therefore only be compared between them. For growing N_{P}, the dtEvo ^{+} process has an increased extinction time because of the stabilising attractive fixed point. This trend is even more pronounced in the approximate analytic solution (solid lines), inspired by Claussen [67, 68] (see Additional file 1). The error of the analytical approach cannot be neglected, but the qualitative trend is clearly visible and the result is fully analytical.
Due to the challenges of employing an exact analytical approach, we cannot perfectly tune the models for the same amplitudes, fluctuations and frequencies/periods of oscillations. The specific choice of the parameters is not necessarily directly comparable, but we have made an effort to choose them in a meaningful way, such that the fixed points are exactly the same and amplitudes comparable. We choose strong selection for the parasite w_{P}=1 and weaker selection for the host w_{H}=0.5 in the models derived from game theory, because the EcoEvo ^{+} model is built in a similar way: Parasite birth can only occur through the antagonistic interaction, but host mortality is also influenced by the competition term. While the parasite is obligate and thus completely dependent on the host, the host suffers, but does not always die from an infection.
The impact of selection intensities on the extinction times is further explored in the Additional file 1. We find that strongly diverging host and parasite selection intensities can counterintuitively lead to more stable dynamics in the Evo process than in the Evo ^{+} process.
Diversity inflow results in sequential negative frequencydependent selection dynamics and arms race dynamics
So far we have compared models with two genotypes in each species. We now provide an outlook of how diversity changes for many genotypes. We simulate one possible example with an initial uniform distribution of twenty genotypes in each species (see Additional file 1). Diversity, simply defined as the number of genotypes present in the population, declines exponentially with time at a constant rate. The manual reintroduction of an extinct, but temporarily best adapted parasite genotype can result in a selective sweep that leaves the parasite population monoclonal.
In reality, our genotypes are not as static in their traits as described here, but one of our ‘genotypes’ can actually be seen as an average of several individuals with slightly different traits. We now add a form of mutation or recombination to the model so that reproduction does not necessarily result in a clonal daughter, but a new individual with different traits. For example, parasites could evolve quickly by allowing beneficial mutations to produce other, even extinct, genotypes. Depending on the model system, a sexually reproducing host could also store genetic material to revive long extinct phenotypes by recombination. We abstract this by inserting a conversion rate μ from one genotype to the neighbouring genotype. For example with five predefined genotypes we have \(H_{1} \xrightarrow {\mu /2} H_{5}\) and \(H_{1} \xrightarrow {\mu /2} H_{2}\) and so on. The dynamics we observe now (Fig. 4) are not pure negative frequencydependent selection dynamics, but a mixture of oscillations and arms race dynamics, where selective sweeps can make a population monoclonal in a very short time, but a reintroduction of extinct genotypes allows for oscillations to reemerge.
Discussion
In this paper we compare evolutionary models from evolutionary game theory to ecoevolutionary models from theoretical ecology to understand the impact of ecology and other model properties on the long term coevolutionary Red Queen oscillations of host and parasite genotypes. The models are individualbased and intrinsically stochastic, thereby allowing genetic drift and the loss of genotypes from a population. Starting with an initially uniform distribution of genotypes, we define the extinction time as the first time that any genotype is lost from any of the two populations, and use this extinction time to measure the robustness of the Red Queen cycles and therefore, the maintenance of diversity. Our main result is that including ecology in models, in the form of a changing population size, leads to a faster loss of genotypes, when stochastic dynamics are considered. This result is similar to the simulation results by Gokhale et al. [28], where ecological dynamics were artificially removed from the simulations, in an attempt to make a straightforward comparison of ecoevo and evo dynamics. In contrast, we compare two modelling frameworks with historically developed differences between them. The models presented here are all based on the same widely used biological assumptions – haploid wellmixed host and parasite genotypes that interact through the matchingalleles infection matrix – but with differences in their mathematical properties: discrete and continuous time models with attractive or neutral deterministic dynamics. The models are further intrinsically stochastic, since they are derived from interactions between individuals. This inherent noise, genetic drift, also impacts the models within a given framework. The mean outward pull by noise that increases amplitudes and thus makes extinction more probable can be counteracted by intraspecific competition that pulls the dynamics back, decreases amplitudes and thus stabilises negative frequencydependent selection dynamics, resulting in longer extinction times. Finally, we provide a snapshot of what happens when standing genetic variation is large initially. If no inflow of genotypes via mutation or migration is provided, the number of genotypes in a population will decline exponentially. However, when conversions between neighbouring types are allowed with a small mutation rate, negative frequencydependent selection dynamics and selective sweeps can occur sequentially.
Previous theoretical studies have similarly examined the persistence of Red Queen oscillations under ecological feedbacks. For example Goméz et al. [24] found fluctuating selection and arms race dynamics in an epidemic model (hostfocussed) with explicitly modelled parasite populations. MacPherson and Otto [27] also combined epidemiological and neutrally stable hostparasite dynamics and showed that this can dampen allele frequency oscillations, which leads to stasis in their deterministic model but would return to oscillations under stochasticity (see Table 2). Recently, the game theoretical replicator dynamics were mathematically tuned for population size influence using a single parameter [18] resulting in damped oscillations and thus stable polymorphism for both matchingalleles and geneforgene infection matrices. While we argue that ecoevolutionary feedbacks increase oscillation amplitudes, Ashby et al. argue that oscillation amplitudes are decreased over time. However, the population dynamics in their model were dampened by a maximal value which resembles our intraspecific competition leading to stasis (Fig. 2). Furthermore, their models are deterministic which closer resembles our models when population size is large, where stabilising effects have a larger influence. In the more theoretical literature, it is now well established that assumptions such as population size fluctuations and stochasticity can result in more rapid extinction ([69, 70], and many more).
The stabilising property of intraspecific competition is documented in the literature [71]. Intraspecific competition (+) enters in our evolutionary models as part of a genotype’s fitness effect that is compared to the focal population’s average fitness, whereas in the ecoevolutionary models it is implemented as an ecological intraspecific competition term. Both the evolutionary and the ecological implementation of this intraspecific competition stabilise the dynamics and lead to stasis following damped oscillations. The more commonly used hostparasite coevolution models result in neutrally stable oscillations whereas damped oscillations are often seen as a termination of Red Queen dynamics. Yet, exactly this stasis shows similar oscillation patterns when stochasticity perturbs dynamics away from the stable fixed point (noise induced oscillations [66]). In a stochastic world, pure hostparasite dynamics therefore result in fast extinction, which would only be stabilised by intraspecific competition. For larger population sizes, when the stability of the fixed point gains in importance, the dynamics are pulled more towards the inner equilibrium state, making stochasticity less influential. Thus, only for organisms with large population sizes and good mixing, intraspecific competition would not be necessary for sustained Red Queen oscillations.
Although this study does not explicitly analyse modes of reproduction, our final result shows how reviving extinct genotypes can restore Red Queen dynamics. If parasites can evolve more quickly due to shorter generation times and larger numbers, then hosts are given an advantage by being able to “store” genotypes through recombination. Also, if clonal reproduction accumulates mutations (Muller’s ratchet), this could impact population sizes and sexual reproduction would be even more important [72, 73]. Ashby and King [8] devised a stochastic individual based susceptibleinfected model with diploid sexual hosts and showed that high diversity cannot maintain sexual reproduction when parasite transmission rates are low. Although our models are more abstract concerning reproduction, we do explicitly model parasites. If parasite populations are well mixed and diverse with high mutation rates, this can again select for higher diversity through sex, like in [24], where fluctuating selection dynamics, and thus high diversity, is more likely when hosts encounter a diverse parasite population and the disease load is high. Furthermore, our models can include global competition in both species or resource competition in the host, which stabilise the oscillating dynamics. More support for recombination during parasite infection was shown in [48], where hosts could optionally switch between two modes of reproduction. See also [4] for a comprehensive connection to the Red Queen Hypothesis for sexual reproduction.
We have shown that in the same setting and with the exact same parameters sequential occurrences of oscillating selection and arms race dynamics are possible. We show only a snapshot and we do not quantify dynamics as is done in [24], but we find it to be an interesting aspect that the dynamics can occur temporarily in the same simulation, with the same settings and assumptions. The more complete picture could include all possibilities discussed in the Red Queen literature: there can be constant extinction, as suggested by van Valen on a taxonomic level and there can be oscillations and arms race dynamics as suggested by hostparasite interactions and the resulting coevolution. With our preliminary results we might be going too far if we also justify sexual reproduction, yet, without recombination or mutation, diversity decline is inevitable theoretically.
Our models explore stochasticity under different restrictions of population size, while other modelling aspects are kept relatively plain. In the present work, the infection pattern is restricted to the matching alleles model, and the zerosum assumption, yet this is necessary for oscillations [21]. Other infectivity patterns that result in Red Queen dynamics have not been examined here. The geneforgene infection matrix could show similar results since the oscillations are also neutrally stable, yet including ecological dynamics changes the complexity of the cycles [25]. In general the robustness of those cycles under stochasticity would depend on the details of how different infection mechanisms are translated into mathematical equations. Further limitations are the haploidy of both hosts and parasites and thereby asexual reproduction, the lack of life history or infection history and there is no spatial structure and evolution in the values of resistance or infectiousness. We do, however, briefly explore the effects of including more genotypes and mutation as a means to revive genotypes. There is an increasing effort to openly discuss how verbal models and biological assumptions enter into models [27, 74]. Making the assumptions clear and readily available should be the standard for future publications. For stochastic processes the analogous deterministic dynamics should be stated to provide the reader with a more complete picture of stochastic dynamics.
The model predictions presented here although quite abstract may nevertheless apply to the real world. Bottlenecks are likely more common in natural hostparasite associations [75] than usually assumed and, therefore, the interaction dynamics are likely shaped by genetic drift and, thus, stochastic effects. Ecoevolutionary feedbacks have been confirmed to impact the form of coevolution in bacteriavirus experiments [76]. Increasing diversity in the parasite or higher exposure lead to a shift from negative frequencydependent selection to arms race dynamics in two bacteriaphage systems [24, 77]. Oscillations alongside incomplete selective sweeps were recently even documented in a nematodebacteria interaction [17]. It would now be of particular interest to assess the occurrence of bottlenecks, drift and competition in natural hostparasite associations and relate them to the resulting allele frequency dynamics. Such empirical data would help us to obtain a more general understanding of hostparasite coevolution and potentially question the importance of sustained Red Queen oscillations in this context.
Conclusions
We have shown that models equal in their verbal biological description can be quite different in their mathematical details, with great consequences for both deterministic and stochastic dynamics. The loss of genotypes is inevitable in stochastic models without mutation or immigration. This extinction is faster when ecological dynamics are considered in an evolutionary model. Competition between genotypes within a species stabilises the dynamics and slows down extinction thus sustaining Red Queen dynamics. The applicability of models to the real world thus depends on the system of interest and mathematical details should be carefully considered for each particular case study. When bottlenecks, drift and competition are observed, the model needs to be adapted accordingly.
Methods
The following method descriptions are short explanations of the stochastic processes used in this manuscript. The precise equations and methods of analysis can be found in the Additional file 1. The simulation code is provided at https://github.com/HannaSchenk/ShortLifeRQ.
The discrete time Evo^{+} process (dtEvo^{+}), also discrete time Moran process, is a stochastic birthdeath process, with a constant population size [78], often used in evolutionary game theory (see for example [79, 80] or [81]). Each birthdeath reaction has a reaction probability (or transition probability), depending on the state of the system in each discrete time step Δt=1. The original definition ensures that the probabilities sum up to one so that one reaction (also reactions where no transition happens  when birth and death event happen within the same genotype) takes place in each time step. In the Moran process, a ‘payoff’ π is what a genotype gains from interactions with others. The interaction matrix is
where α is the fitness gain for the parasite and the fitness loss for a host if the genotypes match, and β is the fitness gain or loss for mismatching pairs (here α=1 and β=0). For example, the probability of a P_{1} birth and thus a subsequent P_{2} death is proportional to \(\pi _{P_{1}}=\alpha h_{1} +\beta h_{2}\), where h_{1} and h_{2},p_{1} and p_{2} are relative abundances. How much this effects the socalled ‘fitness’ f is controlled with the selection intensity w_{P} (or w_{H} for the host) so that \(f_{P_{1}}=1w_{P}+w_{P} \, \pi _{P_{1}}\). This per capita ‘fitness’ is then normalised by a dynamically changing population average \(\overline {f_{P}}=p_{1} \, f_{P_{1}} + p_{2} \, f_{P_{2}}\) (this is the intraspecific competition step) and multiplied with the current abundance of the genotype. Thus \(p_{1} \, f_{P_{1}} / \overline {f_{P}}\) is then the birth probability of a genotype one parasite. The death probability is simply density dependent, thus the total probability of replacing a genotypetwo (death) by a genotypeone (birth) parasite is \(p_{2} \, p_{1} \, f_{P_{1}} / \overline {f_{P}}\). Since we are modelling two populations (host and parasite), we choose to update both populations simultaneously instead of sequentially, such that frequency changes of host genotypes and parasite genotypes can happen at once. The deterministic limit (population sizes N_{H},N_{P}→∞ and time steps Δt→0) of the Moran process in a single population is usually the differential equation of the replicator dynamics, however, in a twopopulation model the average fitness within each population is different and thus the adjusted replicator dynamics become the deterministic analogue [64, 82]. The adjusted replicator dynamics for hostparasite interactions have a globally attractive inner fixed point, in the symmetric matching alleles case this is the equal abundance of all genotypes.
The discrete time Evo process (dtEvo) [64, called pairwise comparison process or local update process in evolutionary game theory], is another birth death process, nearly equivalent to the Moran process but here competition is strictly local and pairwise, not normalised by a global average fitness. What is \(f_{P_{1}}/\overline {f_{P}}\) in the Moran process is here \(0.5+0.5 \, (f_{P_{1}}f_{P_{2}})/ max(\Delta \pi _{P})\). The ‘fitness’ of parasite 1 only depends on the difference in ‘fitness’ to parasite 2 which depends on the abundances of host genotypes (see equation for \(f_{P_{1}}\)), but not, as when normalising with \(\overline {f_{P}}\), on the relative abundances of the parasite genotypes. Thus, in the pairwise comparison process, the antagonist influences globally (since there is no spatial structure), but within a species the competition is local. This results in the recovery of the replicator dynamics with neutral cycles in the deterministic limit.
A Gillespie algorithm [83] can be employed to simulate the above stochastic processes. In this case reaction rates (not probabilities) are calculated for each species and, using random numbers, the shortest waiting time for each reaction is determined under the assumption of exponential waiting times. The reaction with the shortest time takes place and time is updated accordingly. This makes time continuous and time steps unequal. In contrast to the discrete time models, the Gillespie algorithm only updates one species at a time. The now following processes are also implemented using a Gillespie algorithm.
The EcoEvo process uses independent reactions of host birth, parasite death and hostparasite interactions similar to the individualbased equivalent of the Lotka Volterra equations. This results in a microscopic process often believed to be a more natural approach because the reactions describe individual and independent events on the ‘microscopic’ level rather than population dynamics on the ‘macroscopic’ level. Host birth reactions are density dependent with constant rate b_{H}, parasite death is density dependent with constant rate d_{P} and a densitydependent interaction of matching hostparasite pairs can result in the death of the host or the birth of a parasite with constant rate λ. The population size has no restrictions in this case and freely follows the evolutionary dynamics. The deterministic analogue has a neutrally stable fixed point.
The EcoEvo^{+} process are like the EcoEvo independent reactions, but with additional competition in the host population. Densitydependent interactions of two host individuals, independent of the genotype, result in the death of one of the individuals with constant rate μ. This model, when reduced to the deterministic limit, is an antagonistic interaction model with logistic growth in the host (carrying capacity K=b_{H}/μ) and an attractive inner fixed point.
The Hybrid model is a process with selfcontrolled population size. It is built on the EcoEvo model but with constrained birth and death rates adapted from the Evo model and dynamically varied to balance birth and death events on average. Thus, the population size is tightly controlled, yet it is not strictly constant. Building on the stochastic processes from evolutionary game theory above, one can set up a process that utilises the infection matrix for death events in the host and birth events in the parasite in explicit individual reactions. For example the dynamic reaction rate of a death event of a host genotype one is \(d_{H_{1}}=1w_{H}+w_{H} \, \frac {\alpha \, P_{1}+ \beta \, P_{2}}{N_{P}}\). The birth rates for the host and the death rates for the parasite are then dynamically adjusted to equal the total rates of host death and parasite birth. The deterministic limit is the replicator dynamics with a neutrally stable fixed point, as in the pairwise comparison process.
Availability of data and materials
The code that produces the simulated data is available at https://github.com/HannaSchenk/ShortLifeRQ
Abbreviations
 AD:

Adaptive dynamics
 ARD:

Arms race dynamics
 CSS:

Coevolutionary stable strategy
 FSD:

Fluctuating selection dynamics
 GfG:

Geneforgene
 HP:

Hostparasite
 IBM:

Individual based model
 MA:

Matching alleles
 NFSD:

Negative frequencydependent selection
 ODE:

Ordinary differential equation
 PDE:

Partial differential equation
 RE:

Recursion equation
 RK:

Red King
 RQ:

Red Queen
 SDE:

Stochastic differential equation
 SI:

Susceptibleinfected
References
 1
Van Valen L. A new evolutionary law. Evol Theory. 1973; 1:1–30.
 2
Salathé M, Kouyos RD, Bonhoeffer S. The state of affairs in the kingdom of the Red Queen. Trends Ecol Evol. 2008; 23(8):439–45.
 3
Brockhurst MA, Chapman T, King KC, Mank JE, Paterson S, Hurst GDD. Running with the Red Queen: the role of biotic conflicts in evolution. Proc Royal SocB: Biol Sci. 2014; 281(1797):1797.
 4
Neiman M, Lively CM, Meirmans S. Why sex?A pluralist approach revisited. Trends Ecol Evol. 2017; 32(8):589–600.
 5
Strotz LC, Simões M, Girard MG, Breitkreuz L, Kimmig J, Lieberman BS. Getting somewhere with the Red Queen: chasing a biologically modern definition of the hypothesis. Biol Lett. 2018; 5:14. https://doi.org/10.1098/rsbl.2017.0734.
 6
da Silva J. Red Queen Theory. eLS. 2018;:1–7. http://doi.org/10.1002/9780470015902.a0028127.
 7
Lively CM. A Review of Red Queen Models for the Persistence of Obligate Sexual Reproduction. J Hered. 2010; 101(suppl 1):S13–20.
 8
Ashby B, King KC. Diversity and the maintenance of sex by parasites. J Evol Biol. 2015; 28(3):511–20.
 9
West SA, Lively CM, Read AF. A pluralist approach to sex and recombination. J Evol Biol. 1999; 12(6):1003–12.
 10
Jaenike J. A hypothesis to account for the maintenance of sex within populations. Evol Theory. 1978; 3:191–4.
 11
Bell G. The Masterpiece of Nature: The Evolution and Genetics of Sexuality. CUP Archive. 1982.
 12
Woolhouse MEJ, Webster JP, Domingo E, Charlesworth B, Levin BR. Biological and biomedical implications of the coevolution of pathogens and their hosts. Nature Genet. 2002; 32(4):569–77.
 13
Buckling A, Rainey PB. Antagonistic coevolution between a bacterium and a bacteriophage. Proc Royal Soc B. 2002; 269:931–6.
 14
Blanford S, Thomas MB, Pugh C, Pell JK. Temperature checks the Red Queen?Resistance and virulence in a fluctuating environment. Ecol Lett. 2003; 6(1):2–5.
 15
Decaestecker E, Gaba S, Raeymaekers JAM, R Stoks LvK, Ebert D, Meester LD. Host–parasite ‘Red Queen’ dynamics archived in pond sediment. Nature. 2007; 450:870–3.
 16
Koskella B, Lively CM. Evidence for negative frequencydependent selection during experimental coevolution of a freshwater snail and sterlizing trematode. Evolution. 2009; 63:2213–21.
 17
Papkou A, Guzella T, Yang W, Koepper S, Pees B, Schalkowski R, et al. The genomic basis of Red Queen dynamics during rapid reciprocal hostpathogen coevolution. Proc Nat Acad Sci. 2019; 116(3):923–8. https://www.pnas.org/content/116/3/923.
 18
Ashby B, Iritani R, Best A, White A, Boots M. Understanding the role of ecoevolutionary feedbacks in hostparasite coevolution. J Theoret Biol. 2019; 464:115–25.
 19
Maynard Smith J, Price GR. The logic of animal conflict. Nature. 1973; 246:15–8.
 20
Kouyos RD, Salathé M, Bonhoeffer S. The Red Queen and the persistence of linkagedisequilibrium oscillations in finite and infinite populations. BMC Evol Biol. 2007; 7(1):211.
 21
Engelstädter J, Bonhoeffer S. Red Queen dynamics with nonstandard fitness interactions. PLoS Comput Biol. 2009; 5(8):e1000469.
 22
Wolinska J, King KC. Environment can alter selection in hostparasite interactions. Trends Parasitol. 2009; 25(5):236–44.
 23
Ashby B, Gupta S. Parasitic Castration Promotes Coevolutionary Cycling but Also Imposes a Cost on Sex. Evolution. 2014; 68(8):2234–44. http://onlinelibrary.wiley.com/doi/10.1111/evo.12425/abstract.
 24
Gómez P, Ashby B, Buckling A. Population mixing promotes arms race host–parasite coevolution. Proc Royal Soc London B: Biol Sci. 2015; 282(1798):20142297.
 25
Song Y, Gokhale CS, Papkou A, Schulenburg H, Traulsen A. Hostparasite coevolution in populations of constant and variable size. BMC Evol Biol. 2015; 15:212.
 26
Schenk H, Traulsen A, Gokhale CS. Chaotic provinces in the kingdom of the Red Queen. J Theoret Biol. 2017; 431:1–10.
 27
MacPherson A, Otto SP. Joint coevolutionary–epidemiological models dampen Red Queen cycles and alter conditions for epidemics. Theoret Population Biol. 2018; 122:137–48.
 28
Gokhale CS, Papkou A, Traulsen A, Schulenburg H. LotkaVolterra dynamics kills the Red Queen: population size fluctuations and associated stochasticity dramatically change hostparasite coevolution. BMC Evol Biol. 2013; 13:254.
 29
Carius HJ, Little TJ, Ebert D. Genetic variation in a hostparasite association: potential for coevolution and frequencydependent selection. Evolution. 2001; 55(6):1136–45.
 30
Schulte RD, Makus C, Hasert B, Michiels NK, Schulenburg H. Host–parasite local adaptation after experimental coevolution of Caenorhabditis elegans and its microparasite Bacillus thuringiensis. Proc Royal Soc London B: Biol Sci. 2011; 278(1719):2832–9.
 31
Schaffer WM, Rosenzweig ML. Homage to the red queen. I. Coevolution of predators and their victims. Theoret Population Biol. 1978; 14(1):135–57. http://www.sciencedirect.com/science/article/pii/0040580978900084.
 32
Seger J. Dynamics of Some Simple HostParasite Models with More than Two Genotypes in Each Species. Philosoph Trans Royal Soc London Ser B, Biol Sci. 1988; 319(1196):541–55.
 33
Nee S. Antagonistic coevolution and the evolution of genotypic randomization. J Theoret Biol. 1989; 140(4):499–518.
 34
Dybdahl MF, Lively CM. HostParasite Coevolution: Evidence for Rare Advantage and TimeLagged Selection in a Natural Population. Evolution. 1998; 52(4):1057–66.
 35
Boots M, Sasaki A. “Small worlds” and the evolution of virulence: infection occurs locally and at a distance. Proc Royal Soc London B: Biol Sci. 1999; 266(1432):1933–8. http://rspb.royalsocietypublishing.org/content/266/1432/1933.
 36
Peters A, Lively C. The Red Queen and fluctuating epistasis: a population genetic analysis of antagonistic coevolution. Am Naturalist. 1999; 154(4):393–405.
 37
Sasaki A. Hostparasite coevolution in a multilocus geneforgene system. Proc Royal Soc B: Biol Sci. 2000; 267(1458):2183–8.
 38
Agrawal AF, Lively CM. Parasites and the evolution of selffertilization. Evolution. 2001; 55(5):869–79.
 39
Agrawal AF, Lively CM. Infection genetics: geneforgene versus matchingalleles models and all points in between. Evol Ecol Res. 2002; 4:79–90.
 40
Gandon S. Local adaptation and the geometry of host–parasite coevolution. Ecol Lett. 2002; 5(2):246–56.
 41
Gandon S. Evolution of multihost parasites. Evolution. 2004; 58(3):455–69.
 42
Alizon S, van Baalen M. Multiple Infections, Immune Dynamics, and the Evolution of Virulence. Am Naturalist. 2008; 172(4):E150–68. PMID: 18702601. https://doi.org/10.1086/590958.
 43
Agrawal AF. Differences between selection on sex versus recombination in red queen models with diploid hosts. Evolution: International Journal of Organic Evolution. 2009; 63(8):2131–41.
 44
Best A, White A, Boots M. The Implications of Coevolutionary Dynamics to HostParasite Interactions. Am Naturalist. 2009; 173(6):779–91.
 45
Lively CM. An epidemiological model of hostparasite coevolution and sex. J Evol Biol. 2010; 23(7):1490–7.
 46
Greischar MA, Lively CM. Parasites can simplify hostpopulation dynamics and reduce extinction risk. Evol Ecol Res. 2011; 13:557–69.
 47
Gilman RT, Nuismer SL, Jhwueng DC. Coevolution in multidimensional trait space favours escape from parasites and pathogens. Nature. 2012; 483(7389):328.
 48
Mostowy R, Engelstädter J. Host–parasite coevolution induces selection for conditiondependent sex. J Evol Biol. 2012; 25(10):2033–46.
 49
Luijckx P, Fienberg H, Duneau D, Ebert D. A MatchingAllele Model Explains Host Resistance to Parasites. Curr Biol. 2013; 23(12):1085–8.
 50
Abou Chakra M, Hilbe C, Traulsen A. Plastic behaviors in hosts promote the emergence of retaliatory parasites. Sci Rep. 2014; 4:4251.
 51
Taylor BP, Cortez MH, Weitz JS. The virus of my virus is my friend: Ecological effects of virophage with alternative modes of coinfection. J Theoret Biol. 2014; 354:124–36. http://www.sciencedirect.com/science/article/pii/S002251931400143X.
 52
Engelstädter J. HostParasite Coevolutionary Dynamics with Generalized Success/Failure Infection Genetics. Am Naturalist. 2015; 185:E117–29.
 53
Rabajante JF, Tubay JM, Uehara T, Morita S, Ebert D, Yoshimura J. Red Queen dynamics in multihost and multiparasite interaction systems. Sci Rep. 2015; 5:10004.
 54
Hesse E, Best A, Boots M, Hall AR, Buckling A. Spatial heterogeneity lowers rather than increases host–parasite specialization. J Evol Biol. 2015; 28(9):1682–90. https://onlinelibrary.wiley.com/doi/abs/10.1111/jeb.12689.
 55
Rabajante JF, Tubay JM, Ito H, Uehara T, Kakishima S, Morita S, et al. Hostparasite Red Queen dynamics with phaselocked rare genotypes. Sci Adv. 2016; 2(3):e1501548.
 56
Nordbotten JM, Stenseth NC. Asymmetric ecological conditions favor RedQueen type of continued evolution over stasis. Proc Nat Acad Sci USA. 2016; 113:1847–52.
 57
Best A, Ashby B, White A, Bowers R, Buckling A, Koskella B, et al. Host–parasite fluctuating selection in the absence of specificity. Proc R Soc B. 2017; 284(1866):20171615.
 58
Bonachela JA, Wortel MT, Stenseth NC. Ecoevolutionary Red Queen dynamics regulate biodiversity in a metabolitedriven microbial system. Sci Rep. 2017; 7(1):17655.
 59
Greenspoon PB, Mideo N. Parasite transmission among relatives halts Red Queen dynamics. Evolution. 2017; 71(3):747–55. https://onlinelibrary.wiley.com/doi/abs/10.1111/evo.13157.
 60
Lively CM. Habitat Heterogeneity, Host Population Structure, and Parasite Local Adaptation. J Hered. 2017; 109(1):29–37. doi:10.1093/jhered/esx100.
 61
Nuismer SL. Rethinking Conventional Wisdom: Are Locally Adapted Parasites Ahead in the Coevolutionary Race?Am Naturalist. 2017; 190(4):584–93.
 62
Veller C, Hayward LK, Hilbe C, Nowak MA. The Red Queen and King in finite populations. Proc Nat Acad Sci USA. 2017; 114(27):E5396–405.
 63
Ashby B, Boots M. Multimode fluctuating selection in host–parasite coevolution. Ecol Lett. 2017; 20(3):357–65.
 64
Traulsen A, Claussen JC, Hauert C. Coevolutionary dynamics: From finite to infinite populations. Phys Rev Lett. 2005; 95:238701.
 65
Claussen JC. Discrete stochastic processes, replicator and FokkerPlanck equations of coevolutionary dynamics in finite and infinite populations. Banach Center Publ. 2008; 80:17–31.
 66
McKane AJ, Newman TJ. PredatorPrey Cycles from Resonant Amplification of Demographic Stochasticity. Phys Rev Lett. 2005; 94:218102.
 67
Claussen JC. Drift reversal in asymmetric coevolutionary conflicts: influence of microscopic processes and population size. Eur Phys J B. 2007; 60:391–9.
 68
Claussen JC, Traulsen A. Cyclic Dominance and Biodiversity in WellMixed Populations. Phys Rev Lett. 2008; 100:058104.
 69
Kamenev A, Meerson B. Extinction of an infectious disease: A large fluctuation in a nonequilibrium system. Phys Rev E. 2008; 77:061107.
 70
Xue C, Goldenfeld N. Coevolution Maintains Diversity in the Stochastic “Kill the Winner” Model. Phys Rev Lett. 2017; 119(26):268101.
 71
Chesson P. Mechanisms of maintenance of species diversity. Ann Rev Ecol Syst. 2000; 31:343–66.
 72
Howard RS, Lively CM. Parasitism, mutation accumulation and the maintenance of sex. Nature. 1994; 367:554–7.
 73
Howard RS, Lively CM. The ratchet and the Red Queen: the maintenance of sex in parasites. J Evol Biol. 2002; 15(4):648–56.
 74
Yamamichi M, Klauschies T, Miner BE, van Velzen E. Modelling inducible defences in predator–prey interactions: assumptions and dynamical consequences of three distinct approaches. Ecol Lett. 2019; 22(2):390–404. https://onlinelibrary.wiley.com/doi/abs/10.1111/ele.13183.
 75
Papkou A, Gokhale CS, Traulsen A, Schulenburg H. Hostparasite coevolution: why changing population size matters. Zoology. 2016; 119:330–8.
 76
Frickel J, Sieber M, Becks L. Ecoevolutionary dynamics in a coevolving host–virus system. Ecol Lett. 2016; 19(4):450–9.
 77
Betts A, Gray C, Zelek M, MacLean RC, King KC. High parasite diversity accelerates host adaptation and diversification. Science. 2018; 360(6391):907–11. https://science.sciencemag.org/content/360/6391/907.
 78
Moran PAP. Random processes in genetics. Proc Cambridge Philosoph Soc. 1958; 54:60–71.
 79
Nowak MA, Sasaki A, Taylor C, Fudenberg D. Emergence of cooperation and evolutionary stability in finite populations. Nature. 2004; 428:646–50.
 80
Taylor C, Fudenberg D, Sasaki A, Nowak MA. Evolutionary game dynamics in finite populations. Bull Math Biol. 2004; 66:1621–44.
 81
Traulsen A, Hauert C. Stochastic evolutionary game dynamics In: Schuster HG, editor. Reviews of Nonlinear Dynamics and Complexity II. Weinheim: WileyVCH: 2009. p. 25–61. Reviews of Nonlinear Dynamics and Complexity II, page 2561, in Heinz Georg Schuster, editor. WileyVCH, Weinheim, 2009.
 82
Maynard Smith J. Evolution and the Theory of Games. Cambridge: Cambridge University Press; 1982.
 83
Gillespie D. A general method for numerically simulating the stochastic time evolution of coupled chemical reactions. J Comput Phys. 1976; 22:403–34.
Acknowledgements
We want to thank Peter Czuppon, Jens Christian Claussen and Andreas Rößler for detailed and enlightening discussions on methods, approximations and stochastic differential equations. We would like to thank Michael Raatz, HyeJin Park, Stefano Giaimo and Yuriy Pichugin for helpful comments on the manuscript.
Funding
We gratefully acknowledge generous funding by the Max Planck Society, which takes no part in the design, analysis or publication of research.
Author information
Affiliations
Contributions
H.Schulenburg and AT designed the research question. H.Schenk and AT developed and adapted the models. H.Schenk conducted the analysis. All authors discussed and interpreted the results. H.Schenk wrote the initial draft. All authors revisited the manuscript critically and approved the final version. All authors read and approved the final manuscript.
Corresponding author
Correspondence to Hanna Schenk.
Ethics declarations
Ethics approval and consent to participate
Not applicable
Consent for publication
Not applicable
Competing interests
Arne Traulsen is an editorial board member of BMC Evolutionary Biology but he was not involved in handling this manuscript.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Schenk, H., Schulenburg, H. & Traulsen, A. How long do Red Queen dynamics survive under genetic drift? A comparative analysis of evolutionary and ecoevolutionary models. BMC Evol Biol 20, 8 (2020). https://doi.org/10.1186/s1286201915625
Received:
Accepted:
Published:
Keywords
 Hostparasite
 Red queen
 Ecoevolutionary dynamics
 Stochastic models
 Extinction time
 Stability