# A Model with Darwinian Dynamics on a Rugged Landscape

## Abstract

We discuss the population dynamics with selection and random diffusion, keeping the total population constant, in a fitness landscape associated with Constraint Satisfaction, a paradigm for difficult optimization problems. We obtain a phase diagram in terms of the size of the population and the diffusion rate, with a glass phase inside which the dynamics keeps searching for better configurations, and outside which deleterious ‘mutations’ spoil the performance. The phase diagram is analogous to that of dense active matter in terms of temperature and drive.

### Keywords

Population dynamics Glasses Genetic algorithm Glassy landscape## 1 Introduction

Optimization problems—finding the minima of complicated functions—are ubiquitous in science. Statistical Mechanics has proved to be an extremely powerful tool to analyze such problems and the associated algorithms. This is based on the recognition that the energy function of glassy systems are archetypical rugged landscapes, and that the annealing and aging of real glasses are nature’s way to minimize the energy. Simulating the annealing procedure for artificial optimization problems is a robust and quite effective method [1, 2].

Darwinian dynamics may be viewed as an alternative method to optimize a function—in this case maximizing the ‘fitness’—clearly also widespread in nature. This has been long recognized, and the literature on artificial ‘Genetic Algorithms’ is vast [3]. The principle is rather different from that of annealing: instead of the algorithm searching actively for a better situation (a ‘Lamarckian’ strategy), it just produces ‘clones’ that mutate randomly and are later selected according to their fitness. Because the connection between Darwinian dynamics and physical evolution is less obvious than in the case of annealing, the implications of physics to such problems has been much less studied. Although there have been statistical mechanical models of evolution (see, for example [4, 5, 6, 7, 8, 9]) using the knowledge of universal glassy features has been much less exploited [10].

*N*is finite, and for simplicity is kept fixed by randomly decimating the population: a Moran process.

The dynamics of a population of *N* individuals reproducing and undergoing random mutations and selection has long been recognized to bear a resemblance with a system driven by a ‘fitness potential’, with an element of ‘noise’ given by random fluctuations that are larger, the smaller the total population (see, e.g. [14, 15]). However, the stochastic dynamics of a system in contact with a thermal bath satisfy the relation of ‘detailed-balance’—the condition that the bath is itself in thermal equilibrium—obviously not applicable in general to an evolutionary dynamics with mutation and selection. A known exception happens when the population is dominated by a single mutant at any time, whose identity changes in rare and rapid ‘sweeps’ in which a new mutant fixes [16, 17], see Fig. 1c. It turns out that in that special case [16, 18, 19, 20], there is a correspondence that we shall exploit to understand some features of the phase diagram.

In this paper we shall study the Darwinian dynamics in an archetypical constraint optimization problem (Satisfiability: KSAT and XORSAT). Our purpose is not to propose this model as a relevant metaphor for biology (there are many references on this, see for example [4, 5, 6, 7, 8, 9]), but rather to work out the details in a nontrivial case. A complete analytic solution for the population dynamics in these models is perhaps possible, but seems like a daunting task.

## 2 The Model

We shall consider a population of individuals assumed to be independent, their internal states being denoted \(a=1,...,2^L\)’. Each has on average \(\lambda _{a}\) offspring per unit time. The total number *N* is kept constant—or in some cases slowly varying—by decimating or ‘cloning’ randomly chosen individuals at the necessary rate, a Moran process [21]. The probability of mutation per generation a state *a* to a state *b* is \(\mu _{ab}\), so that mutation times are random with average \(\tau _{ab}=1/\left( \lambda _{a}\mu _{ab}\right) \). In the literature, either the probabilities \(\mu _{ab}\) or the times \(\tau _{ab}\) are often taken identical for all allowed mutations. We shall adopt here \(\tau _{ab}=\tau _0 \; \forall {ab} \)

*N*individuals. We represent the internal state of an individual using Boolean variables: \(\mathbf{s}^a=\{s_{1}^a,...,s_{L}^a\}\) taking values \(s_i^a=0,1\). The fitness functions we use are standard spin-glass benchmarks, whose landscape properties have been extensively studied [22]. It is constructed as follows: there are \(\alpha L\) clauses \(\nu \) with \(K=3\) variables, of the form \((s_{i_{1}^{\nu }}\vee \overline{s_{i_{2}^{\nu }}}\vee s_{i_{3}^{\nu }})\) where both the \((i_{1}^{\nu },i_{2}^{\nu },i_{3}^{\nu })\) chosen for each clause—and the fact that the variable is negated or not—are decided at random once and for all. The Random K-SAT and Random Xor-SAT take the form, for example:

*E*

*N*individuals, each identified by a vector \(\mathbf{s}^{individual}\) is obtained by flipping randomly one of their components, in other words it is a diffusion on

*L*dimensional hypercube (Fig. 2), where they reproduce or die according to the SAT or XORSAT fitness rule.

## 3 A Brief Digression: The House of Cards Model

*ab*with \(\mu =\sum _{b}\mu _{ab}\), so an individual may jump between any two states. The evolution of this system is depicted in Fig. 3, where initially each individual is chosen randomly. The system traverses through several regimes:

- (i)
*Continuous population*essentially all the population is in states*a*such that \(1\ll n_{a}\ll N\), for all other states \(n_{a}=0\). One may treat the problem in terms of a continuous approximation \(\rho (\lambda )\) corresponding to the fraction of individuals having \(\lambda _{a}\) between contained between \(\lambda \) and \(\lambda +d\lambda \), using the Replicator equation [26]. We have:where \(\mu \) is the total probablity of mutating out of the interval \([\lambda ,\lambda +d\lambda ]\), and \(\bar{p}(\lambda )\) is the density of states.$$\begin{aligned} \dot{\rho }(\lambda )=[\lambda \left( 1-\mu \right) -\langle \lambda \rangle ]\rho (\lambda )+\mu \lambda \bar{p}(\lambda )with\langle \lambda \rangle =\int d\lambda \,\rho (\lambda )\,\lambda \end{aligned}$$(3) - (ii)
*Concurrent mutations regime*[27] Finite population size effects cannot be neglected even if the population starts at regime (i), because they begin to show up at times of order \(\ln N\). Here a finite fraction of all individuals are concentrated in a finite number of types (Fig. 3) competing for domination (strong clonal interference). (See [27, 28, 29, 30], especially Refs. [24, 25, 26, 27, 28, 29, 30] therein). - (iii)
*Successional mutation regime*The system settles into a regime in which the majority of individuals belong to a single type. Some of these individuals mutate, most often deleteriously, and die, accounting for a constantly renewed population ‘cloud’ of order \(\mu N\) outside the dominant sub-population. Every now and then, an individual mutates to a state that is more fit, in which case it may spread in the population until completely taking over (fixation). There are, in addition, events in which the entire population may get fixated to a mutation that is (slightly) less fit: these extinction events are exponentially rare in*N*. In this regime, it is easy to compute the probability for a new mutation to appear and fix in an interval of time \(\delta t\), large with respect to the fixation time, small compared to the time between successive fixations [14]:where$$\begin{aligned} P({\text{ fixed } \text{ in } }a\rightarrow {\text{ fixed } \text{ in } }b)&\equiv P(a\rightarrow b) =N\lambda _{a}\mu _{ab}\delta t\;\frac{\frac{\lambda _{a}}{\lambda _{b}}-1}{(\frac{\lambda _{a}}{\lambda _{b}})^{N}-1}\nonumber \\&=N\lambda _{a}\mu _{ab}\delta t\; \frac{e^{\left( E_{b}-E_{i}\right) /L}-1}{e^{\left( E_{b}-E_{a}\right) N/L}-1}. \end{aligned}$$(4)$$\begin{aligned} E_{a}\equiv -L\ln \lambda _{a}\,, \end{aligned}$$(5)

^{1}

**Emergence of Detailed Balance in the Successional Regime**

*for the successional mutation regime*the ‘meta-dynamics’ of the dominant sub-population, considered as a single entity, neglecting the relatively short times in which the system is not concentrated into a single type (the fixation processess). Since in this example \(\mu _{ab}=\mu _{ba}\):

*N*, and dropping \(O\left( 1/N\right) \) corrections we write \(T=\frac{L}{N}\). At very long times, the system will reach a distribution

*equilibrium*Random Energy Model [25], (see [24, 32]). In particular, we conclude that, depending on the value of

*N*, at very long times the system will equilibrate to either a ‘liquid’ phase (for \(N/L<\ln 2\)) or a ‘frozen’ phase (for \(N/L>\ln 2\)), see Fig. 4: in the former random extinction events stop the system from converging to the optimum level of fitness, while in the latter this level is at long times reached. A feature we find here, and is a general fact, is that even if the dynamics satisfy detailed balance, and are hence able in principle to equilibrate, this takes place at unrealistically long times.

The qualitative features of the population at different times has long been known, the similarity of the role played by fluctuations due to finite population size with thermal fluctuations has also been noted long ago [14, 15]. Here the analogy becomes an identity, and the effect of accumulation of deleterious mutations becomes just the question of an ordinary order-disorder phase transition. Similarly, the effect of population bottlenecks becomes the same as a spike in temperature.

## 4 The Phase Diagram of the Darwinian SAT Models

The example of the House of Cards model suggest that we consider a phase diagram with variables \(\propto 1/N\) and \( 1/\tau _0\). To obtain a meaningful phase diagram (Fig. 5), the scalings with growing *N* must be consistently defined. It is easy to see that for this one must keep *N* / *L* constant, and numerics further show that mutation times must scale as \(\tau \equiv \tau _0/ N\), where \(\tau \) is constant for a given value of *N* / *L*.^{2} Barrier-crossing mechanisms for the entire population are expected for both high and flat and wide barriers [35, 36].

### 4.1 The Thermal Correspondence at Low Mutation Rate

Let us first discuss the line \(1/\tau =0\). In this regime, mutations are very rarely proposed, and the system eventually falls in the successional regime (\(N/\left( \lambda \tau \right) \ll 1\)). As we have seen above, detailed balance is then expected to hold with temperature \(T=\frac{L}{N}\). Indeed, in Fig. 6 we show the results of a simulated annealing performed with an ordinary Monte Carlo program on a single sample, superposed with a ‘populational annealing’ performed by slowly increasing the population of a set of individuals performing diffusion and reproducing according to the fitness in Eq. (1). The coincidence of both curves in terms of \(T=L/N\) is reassuring.

In the limit \(\tau \rightarrow \infty \), when the population behaves like a thermal realization of a SAT system at temperature \(T=\frac{L}{N}\), the situation is well understood for the XorSAT and the SAT problems [37]: there is a (dynamic) glass transition at a certain temperature \(T_{d}\) below which the phase-space breaks into components, and the dynamics become slow, rendering the optimization very hard. This transition happens before the thermodynamic one, which itself is closely analogous to the freezing one of the REM.

We may locate the dynamic transition by plotting the autocorrelation functions \(C(t,t')=\frac{4}{L}\sum _{i}(s_{i}\left( t\right) -\frac{1}{2})(s_{i}\left( t'\right) -\frac{1}{2})\) at decreasing temperatures. As \(T_{d}\) is approached from above, the correlation decays in a two-time process, a fast relaxation to a plateau followed by a much slower ‘\(\alpha \)-relaxation’ (in the glass terminology), taking a time \(t_{\alpha }\). As \(T_{d}\) is reached, \(t_{\alpha }\) diverges. Below \(T_{d}\), the system *ages*: the time \(t_{\alpha }\) now keeps increasing with time, \(C(t,t')\) decays in a time \((t-t')_{decay}\sim t_{\alpha }(t')\) with \(t_{\alpha }(t')\) an increasing function of \(t'\). What we have described is the ‘Random First Order Transition’ [38]. Nothing new here, as the system is equivalent to a thermal system, known to exhibit such a transition.

*N*individuals are fully clustered in a configuration at most times, and

*N*no longer has an obvious thermal meaning. We approach the transition by increasing

*N*at fixed \(\tau \), and also by decreasing \(\tau \) at fixed

*N*. The correlation curves obtained are shown in Fig. 7: the nature of the transition remains the same, but the transition value of critical

*N*shifts with \(\tau \). All in all, we obtain

^{3}the phase diagram of Fig. 5.

### 4.2 Thermal Properties of Slow Dynamics in the Region with Slow Dynamics (large \(t_\alpha \))

We know that the \(\tau ^{-1}=0\) axis is just equivalent to the thermal problem, with temperature *L* / *N*. What can we say about clustering for smaller \(\tau \)? Figure 1 shows the contributions of different configurations (the top uniform color corresponds to contributions smaller than 10 % each). We see that for all but the highest \(\tau \), the system is in the *concurrent mutation regime* [27], and the thermal correspondence, applied naively, breaks down. Considering several examples with timescale separation, we have conjectured [40] that whenever the \(\alpha \) relaxation time is large (near and below the transition), the correspondence with a thermal system may still hold, but taken for quantities that are averaged over a time of several \(t_\alpha \), and considering two situations at time-separations much larger than \(t_{\alpha }\).

*above*the transition (so that a stationary distribution might be reached) but not far from it (so that \(t_{\alpha }\) is large), the circled numbers in Fig. 5. We start with a system in equilibrium at time \(t=0\), and we switch on a field \(E\rightarrow E-hA\), where

*A*is any observable, in our case we choose \(A=\sum _{i}s_{i}\). After an arbitrary time

*t*we measure again the value of

*A*and check the equation (see [40])

*fails*. Instead, when we compute the differences as \(\Delta =\bar{A}(t)-\bar{A}(t=0)\), with \(\bar{A}\) the average of

*A*within a window comparable to the time to reach a plateau—the

*‘equilibration within a valley’*time \(\ll t_{\alpha }\)—and \(t\sim 3t_{\alpha }\), the relation (9) for the averaged values \(\Delta =\bar{A}(t)-\bar{A}(0)\) works perfectly, without fitting parameters. We interpret this as meaning that the jumps between valleys, taking a long time, are indeed governed by a temperature

*L*/

*N*, although the diffusion inside a valley is not.

### 4.3 An Analogy with Dense Active Matter

Active matter is composed by particles that have a source of energy other than a simple thermal bath. This may be the case of bacteria, of specially defined propulsed particles, and of randomly shaken particles. A way to model this situation, is to consider a source of energy in the high frequencies (for example a thermal bath at high temperature \(T_{fast}\) with white noise) and a mechanism of dissipation, such as a thermal bath with low-frequency correlated noise at a lower temperature \(T_{slow}\) (see Ref. [42]). The high frequency source need not be a true thermal bath, it could consist for example of random ‘kicks’. In any case, the system is not truly in equilibrium, because for this one needs a same temperature at all timescales. If one considers a situation with high density and a suitable energy balance situation, the particle system approaches a glass transition just as an ordinary thermal one. Timescale-separation appears, the fast motion is essentially ruled by the ‘fast’ excitation, while the slow motion by the ‘slow’ bath. One expects then detailed balance at temperature \(T_{slow}\) to hold for slow evolution and time-averaged quantities, while it does not hold inside a state. The situation is quite close to the one discussed in this paper, the role of the ‘fast’ excitation being played here by the fluctuations due to the Darwinian nature of the dynamics within a state. The parameter \(1/\tau \) plays the role of the input energy rate at high frequencies, while \(\frac{L}{N}\) plays the role of \(T_{slow}\). Our phase diagram Fig. 5 is strikingly similar the one of the active matter model in Ref [42].

### 4.4 Changing Environments: Connection to Glassy Rheology

A system as the one we are considering, which is achieving better fitness by slowly adapting to a complex landscape, is extremely sensitive to changes in this landscape. This effect has been discussed in [31], although in a slightly different form, and also in [43]. The counterpart in glass physics of this fact has long been known. Consider the situation [44] of a plastic bar prepared a time \(t_{w}\) ago from a melt. The polymers constituting the bar slowly rearrange—ever more slowly—to energetically better and better configurations, and this process is known to go on at least for decades. The bar is out of equilibrium, a fact that we may recognize by testing its response to stress, which measurably depends on \(t_{w}\). Now suppose that we apply a large, fixed deformation to the bar, for example applying a strong torsion one way and the other. The new constraints change the problem of optimization the polymers are ‘solving’: we expect evolution to restart to a certain extent, and the apparent ‘age’ of the bar to become smaller than \(t_{w}\). This is indeed what happens [44], a phenomenon called ‘rejuvenation’. Rejuvenation brings about an acceleration in the dynamics. If the changes are continuous and different, instead of aging (growth of \(t_{\alpha }\)), the system settles in a value of \(t_{\alpha }\) that depends on—adapts to—the speed of change of the energy landscape. Note that this property of evolution speed, *as measured from the changes in the population* adapting to landscape change speed, that is sometimes attributed to a form of criticality [22], here appears as a universal property of aging systems.

## 5 Conclusions

We have studied the dynamics of a population of random walkers reproducing with a rate corresponding to a ‘rugged’ fitness function. The natural phase variables are the size of the system and the diffusion constant (the mutation rate). We obtain a glassy region in this phase diagram where the systems ‘ages’, slowly evolving to ever better fitnesses. We have found evidence that even for relatively high diffusion constants, the size of the system may be interpreted as an inverse temperature, provided one considers time-averaged quantities. The phase diagram bears a striking resemblance to the one that would be obtained for the same system in contact with a bath at temperature *T* driven simultaneously at the high frequencies, the intensity of the latter playing the role of the inverse mutation rate \(1/\tau \).

If one allows for many mutations to exist, while still having a single dominant population at almost all times, a somewhat different regime is obtained [27]. Then Eq. (4) no longer holds due to the population ‘cloud’ of deleterious mutations. If these mutants do not reproduce (\(\lambda =0\)), Eq. (4) may be mended by considering an effective \(N_{eff}=N-N_{cloud}\), but for more general deleterious mutations a simple prescription is hard to give. However, this correction is small when mutation rates are low \(\mu \ll 1\) (but not necessarily very low \(\mu N\ll 1\)). More precisely, Eq. (4) holds when the fraction of deleterious mutations is small, \(\frac{\mu \lambda }{\lambda -\lambda _{del}}\ll 1\), where \(\lambda \) is the fitness of the dominant population and \(\lambda _{del}\) is a typical fitness of deleterious mutations.

This entails that the width of the fitness distribution in the population at a given time is \(\sigma _{\lambda }^{2}\sim \frac{N}{L^{2}\tau _0}\sim \frac{1}{N^{2}}\) (following arguments as in [33, 34] ). The time-scale for a given spin flip is on average \(\tau _{point}=L\tau _0\). which corresponds also to the time-scale for an individual to shuffle its entire configuration.

## Acknowledgments

We would like to thank JP Bouchaud, and D.A. Kessler for helpful discussions.