Analysis of Multilevel Replicator Dynamics for General Two-Strategy Social Dilemma

Abstract

Here, we consider a game-theoretic model of multilevel selection in which individuals compete based on their payoff and groups also compete based on the average payoff of group members. Our focus is on multilevel social dilemmas: games in which individuals are best off cheating, while groups of individuals do best when composed of many cooperators. We analyze the dynamics of the two-level replicator dynamics, a nonlocal hyperbolic PDE describing deterministic birth–death dynamics for both individuals and groups. While past work on such multilevel dynamics has restricted attention to scenarios with exactly solvable within-group dynamics, we use comparison principles and an invariant property of the tail of the population distribution to extend our analysis to all possible two-player, two-strategy social dilemmas. In the Stag–Hunt and similar games with coordination thresholds, we show that any amount of between-group competition allows for fixation of cooperation in the population. For the prisoners’ dilemma and Hawk–Dove game, we characterize the threshold level of between-group selection dividing a regime in which the population converges to a delta function at the equilibrium of the within-group dynamics from a regime in which between-group competition facilitates the existence of steady-state densities supporting greater levels of cooperation. In particular, we see that the threshold selection strength and average payoff at steady state depend on a tug-of-war between the individual-level incentive to be a defector in a many-cooperator group and the group-level incentive to have many cooperators over many defectors. We also find that lower-level selection casts a long shadow: If groups are best off with a mix of cooperators and defectors, then there will always be fewer cooperators than optimal at steady state, even in the limit of infinitely strong competition between groups.

Appendices

Well-Posedness for Measure-Valued Formulation

In this section, we will demonstrate that our representation of $\mu _t(\mathrm{d}x)$ in terms of the push-forward measure of $\mu _0(\mathrm{d}x)$ is well posed, justifying our use of the push-forward representation in proving preservation of the Hölder exponents and in characterizing convergence to delta functions at within-group equilibrium below the threshold for existence of steady state. Our strategy for proving existence of solutions to our measure-valued equations involves a contraction mapping approach often used for hyperbolic PDEs with nonlocal terms, which often arise in models of populations structured by age or size (Dawidowicz and Łoskot 1986), as well as in models of collective motion of animal groups (Eftimie et al. 2009; Canizo et al. 2011) or bacterial chemotaxis (Hillen and Stevens 2000). This approach has also been considered in the context of measure-valued solutions for transport equations (Evers et al. 2015; Evers 2016), models of collective motion, and models with genetic or age structure (Canizo et al. 2011; Cañizo et al. 2013).

To discuss the existence of solutions, we can consider the following slight generalization of the measure-valued dynamics of Eq. 13

$$\begin{aligned} \displaystyle \frac{\mathrm{d} \displaystyle }{\mathrm{d}t}\int _0^1 \Psi (x) \mu _t(\mathrm{d}x) = \displaystyle \int _0^1 \left\{ j(x) \displaystyle \frac{\partial \Psi (x)}{\partial x} + \lambda \Psi (x) \left[ G(x) - \left( \int _0^1 G(y) \mu _t(\mathrm{d}y) \right) \right] \right\} \mu _t(\mathrm{d}x) \end{aligned}$$

(A1)

where the within-group dynamics j(x), the group payoff G(x), and the test function $\Psi (x)$ are defined on [0, 1] and are continuously differentiable in x. We also assume that $j(0) = j(1) = 0$ to capture the feature that all-cooperator and all-defector groups are steady states of the within-group dynamics. To understand the solutions of Eq. A1, we consider an associated linear PDE. Given an arbitrary $h(t) \in C([0,T])$, we define the linear PDE

$$\begin{aligned} \displaystyle \frac{\mathrm{d} \displaystyle }{\mathrm{d}t}\int _0^1 \Psi (x) \mu ^h_t(\mathrm{d}x)&= \displaystyle \int _0^1 \left\{ j(x) \displaystyle \frac{\partial \Psi (x)}{\partial x} + \lambda \Psi (x) \left[ G(x) - h(t)\right] \right\} \mu ^h_t(\mathrm{d}x) \end{aligned}$$

(A2a)

$$\begin{aligned} \mu _0^{h}(\mathrm{d}x)&= \mu _0(\mathrm{d}x) \end{aligned}$$

(A2b)

where $\mu _t^h(\mathrm{d}x)$ denotes our solution of Eq. A2 for given h(t). Because j(x) is Lipschitz and $j(0) = j(1) = 0$, we see that the characteristic curves $x(t,x_0)$ satisfying $\displaystyle \frac{\mathrm{d} }{\mathrm{d}t} x(t,x_0) = j(x(t))$ and $x(0,x_0) = x_0$ exist globally in time.

Our strategy will be to use solutions of Eq. A2 in order to establish the existence of a solution to Eq. A1. We can do this by considering an arbitrary forcing function $h^0(t)$, finding the corresponding solution $\mu _t^{h^0}(\mathrm{d}x)$, and then finding a new forcing function $h^1(t) := \int _0^1 G(x) \mu _t^{h^0}(\mathrm{d}x)$. By iterating this process, we hope to construct a sequence of forcing functions $h^j$ converging to a fixed point. In other words, if we define our iteration function as

$$\begin{aligned} H(h(t)) := \int _0^1 G(x) \mu _t^h(\mathrm{d}x), \end{aligned}$$

(A3)

then we are looking to find a fixed point $h_{\flat }(t)$ satisfying $H(h_{\flat }(t)) = h_{\flat }(t)$, in which case $\mu _t^{h_{\flat }}(\mathrm{d}x)$ satisfies Eq. A1. For our subsequent analysis, we denote $G^* := \max _{x \in [0,1]} G(x)$, $(G')^* = \max _{x \in [0,1]} |G'(x)|$, $j^* := \max _{x \in [0,1]} j(x)$, and $||h(t)||_T := \sup _{s \in [0,t]} |h(s)|$. Using the norm $||\cdot ||_T$, we aim to show that H(h(t)) is a contraction on $C\left( [0,T] \right)$. To discuss contraction mappings, we will look to estimate $||H(h(t)) - H({\tilde{h}}(t))||_T$ for any two given functions $h(t), {\tilde{h}}(t)$ in our function space. In describing the measure-valued dynamics, it is helpful to use the shorthand notation

$$\begin{aligned} \langle \Psi , \mu _t \rangle := \int _0^1 \Psi (x) \mu _t(\mathrm{d}x). \end{aligned}$$

(A4)

For example, this allows us to describe the dynamics of the auxiliary linear problem of Eq. A2 as follows:

$$\begin{aligned} \displaystyle \frac{\mathrm{d} }{\mathrm{d}t} \langle \Psi , \mu ^h_t \rangle&= \left\langle j(x) \frac{\partial \Psi }{\partial x} , \mu ^h_t \right\rangle + \lambda \langle \Psi \left[ G(x) - h(t)\right] , \mu ^h_t \rangle \end{aligned}$$

(A5a)

$$\begin{aligned} \mu _0^h&= \mu _0. \end{aligned}$$

(A5b)

Now, we will present the results on well-posedness for the two-level PDE of Eq. A1. First, we have two lemmas dealing with the linear auxiliary problem from Eq. A2. In Lemma 3, we show that there is a variation of constants formula which must be satisfied by any solution $\mu ^h_t(\mathrm{d}x)$ of Eq. A2, which serves as a useful tool for proving uniqueness and computing contraction-mapping estimates. In Lemma 4, we show that there exists a unique solution to Eq. A2, which has an explicit representation formula reminiscent of the implicit formula of Eq. 16 for the nonlinear problem. In Proposition 10, we use the results of the two lemmas and a contraction-mapping argument to show that Eq. A1 has a unique solution and that this solution $\mu _t(\mathrm{d}x)$ satisfies the implicit representation formula of Eq. 16.

Lemma 3

If $\mu ^h_t(\mathrm{d}x)$ is a measure-valued solution to Eq. A2, then it also satisfies the variation of constants formula

$$\begin{aligned} \langle \Psi , \mu ^h_t \rangle = \langle P_t \Psi , \mu ^h_0 \rangle + \lambda \int _0^t \langle P_{t-s} \left( G(x) - h(t) \right) , \mu _s^h \rangle \mathrm{d}s \end{aligned}$$

(A6)

where $P_t (f(x)) = f(\phi _t(x))$ denotes evaluating functions along characteristic curves $\phi _t(x)$.

Remark 5

This is a generalization of the variation of constants formula described in Lemma 11 of Luo and Mattingly (2017), which described the special case in which $j(x) = -sx(1-x)$ and $G(x) = x$. The derivation of the formula is analogous for the generalized formula, so we will omit the proof.

Lemma 4

Given $T > 0$, the flow of measures $\mu ^h_t(\mathrm{d}x)$ given by the formula

$$\begin{aligned} \mu ^h_t(\mathrm{d}x)&= w^h_t(x) (\mu _0 \circ \phi _t^{-1})(\mathrm{d}x) \end{aligned}$$

(A7a)

$$\begin{aligned} w^h_t(\phi _t(x_0))&= \exp \left( \lambda \int _0^t \left[ G(\phi _s(x_0)) - h(s) \right] \mathrm{d}s \right) \end{aligned}$$

(A7b)

is the unique solution of Eq. A2 for each $t \in [0,T]$.

Proof

We see that Eq. A7 solves Eq. A2 by differentiating with respect to time, obtaining

$$\begin{aligned} \displaystyle \frac{\mathrm{d} }{\mathrm{d} t} \displaystyle \int _0^1 \Psi (x) \mu ^h_t(\mathrm{d}x)&= \displaystyle \frac{\mathrm{d} }{\mathrm{d} t} \displaystyle \int _0^1 \Psi (x) w^h(x) (\mu ^h_0 \circ \phi _t^{-1})(\mathrm{d}x) = \displaystyle \frac{\mathrm{d} }{\mathrm{d} t} \int _0^1 \Psi (\phi _t(x)) w^h_t(\phi _t(x)) \mu ^h_0(\mathrm{d}x) \\&= \displaystyle \int _0^1 \frac{\partial \Psi (\phi _t(x))}{\partial \phi _t(x)} \left[ \frac{\partial \phi _t(x)}{\partial t} \right] w^h_t(\phi _t(x)) \mu ^h_0(\mathrm{d}x) + \displaystyle \int _0^1 \Psi (\phi _t(x)) \frac{\partial w^h_t(\phi _t(x))}{\partial t} \mu ^h_0(\mathrm{d}x) \\&= \displaystyle \int _0^1 \frac{\partial \Psi (\phi _t(x))}{\partial \phi _t(x)} j(\phi _t(x)) w^h_t(\phi _t(x)) \mu ^h_0(\mathrm{d}x) \\&\quad +\,\lambda \displaystyle \int _0^1 \Psi (\phi _t(x)) \left[ G(\phi _t(x)) - h(t) \right] w^h_t(\phi _t(x)) \mu ^h_0(\mathrm{d}x) \\&= \displaystyle \int _0^1 \left\{ \frac{\partial \Psi (x)}{\partial x} j(x) w^h_t(x) + \lambda \Psi (x) \left[ G(x) - h(t) \right] w^h_t(x) \right\} \left[ \mu ^h_0 \circ \phi _t^{-1} \right] (\mathrm{d}x) \\&= \displaystyle \int _0^1 \left\{ \frac{\partial \Psi (x)}{\partial x} j(x) + \lambda \Psi (x) \left[ G(x) - h(t) \right] \right\} \mu _t(\mathrm{d}x) \end{aligned}$$

where we used that $\phi _t(x)$ solves $\frac{\partial }{\partial t}\phi _t(x_0) = j(\phi _t(x_0))$, that $\frac{\partial }{\partial t}w^h_t(\phi _t(x_0)) = \lambda \left( G(\phi _t(x_0)) - h(t) \right) w^h_t(\phi _t(x_0))$. Putting together the first line and last line, we can conclude that $\mu ^h_t(\mathrm{d}x)$ given by Eq. A7a is a solution to Eq. A2.

Next, we see that $\mu _t^h(\mathrm{d}x)$ is unique. Suppose there were two solutions $\mu _t^h(\mathrm{d}x)$ and $\nu _t^h(\mathrm{d}x)$ to Eq. A2 (where notably the initial conditions agree upon $\mu ^h_0(\mathrm{d}x) = \nu _0^h(\mathrm{d}x) = \mu _0(\mathrm{d}x)$. Using the variation of constants formula, we have that

$$\begin{aligned}&\langle \Psi , \mu _t^h \rangle \, = \, \langle P_t \Psi , \mu _0 \rangle + \lambda \int _0^t \langle \left( P_{t-s} \Psi \right) \left( G(x) - h(t) \right) , \mu _s^h \rangle \mathrm{d}s \\&\langle \Psi , \nu _t^h \rangle \, = \, \langle P_t \Psi , \mu _0 \rangle + \lambda \int _0^t \langle \left( P_{t-s} \Psi \right) \left( G(x) - h(t) \right) , \nu _s ^h\rangle \mathrm{d}s. \end{aligned}$$

We see that

$$\begin{aligned} | \langle \Psi , \mu _t^h - \nu _t^h \rangle | \le \lambda \left( G^* + ||h||_t \right) \int _0^t \langle P_{t-s} \Psi , \mu _s^h - \nu _s^h \rangle \mathrm{d}s. \end{aligned}$$

Using the total variation norm $||\mu _t - \nu _t||_{TV} := \sup _{||\Psi || \le 1} \langle \Psi , \mu _t - \nu _t \rangle$ and that $||P_{t-s} \Psi ||_{\infty } \le ||\Psi _{\infty }||$ (because $P_{t-s} \Psi := \Psi (\phi _{t-s}(x_0))$), we can further see, for test functions satisfying $||\Psi ||_{\infty } \le 1$, that

$$\begin{aligned} ||\mu _t^h - \nu _t^h||_{TV} \le \lambda \left( G^* + ||h||_t \right) \int _0^t ||\mu _t^h - \nu _t^h ||_{TV} \mathrm{d}s \end{aligned}$$

and Grönwall’s inequality lets us deduce that

$$\begin{aligned} ||\mu _t^h - \nu _t^h||_{TV} \le ||\mu _0^h - \nu _0^h||_{TV} \exp \left( \lambda \left[ G^* + ||h||_t \right] T \right) = 0 \, \, {\mathrm {because}} \, \, \mu _0^h = \nu _t^h, \end{aligned}$$

which allows us to conclude that $\mu _t^h(\mathrm{d}x) = \nu _t^h(\mathrm{d}x)$ for all $t \in [0,T]$ and solutions of Eq. A2 are unique. Together with our explicit representation formula from $\mu _t^h(\mathrm{d}x)$, we can conclude that there exists a unique solution to Eq. A2 for given function h(t). We now need to construct an iteration scheme and use solutions from Eq. A2 to produce a solution of Eq. A1. $\square$

Proposition 10

Assume that j(x), G(x), and $\psi (x)$ are $C^1$ on [0, 1]. Given initial probability measure $\mu _0(\mathrm{d}x)$, there exists a unique solution $\mu _t(\mathrm{d}x)$ to Eq. A1 for all time $t \ge 0$. Furthermore, the solution $\mu _t(\mathrm{d}x)$ satisfies the implicit representation formula from Eq. 16

$$\begin{aligned} \mu _t(\mathrm{d}x) = w_t(x) \left( \mu _0 \circ \phi _t^{-1}\right) (\mathrm{d}x). \end{aligned}$$

Proof

From the push-forward representation of $\mu _t^h(\mathrm{d}x)$ and the fact that $\mu _0(\mathrm{d}x)$ is a probability measure, we see that $\mu _t^h$ satisfies the following a priori estimate:

$$\begin{aligned}&\int _0^1 \Psi (x) \mu _t^h(\mathrm{d}x) = \int _0^1 \Psi (\phi _t(x)) w^h_t(\phi _t(x)) \mu _0(\mathrm {d} x) \le ||\Psi ||_{\infty } \exp \left( \lambda \left[ G^* + ||h||_T \right] T \right) \int _0^1 \mu _0(\mathrm{d}x) \nonumber \\&\quad = ||\Psi ||_{\infty } \exp \left( \lambda \left[ G^* + ||h||_T \right] T \right) , \end{aligned}$$

(A8)

so for $T < \infty$ and given h(t), there exists $M^T_{h} < \infty$ such that $\langle \Psi , \mu _t^h \rangle \le M^T_h$.

To apply the Banach fixed-point theorem, we now need to show that

• (i) H(h(t)) is a contraction: $\exists \eta \in (0,1)$ such that $\forall h(t), {\tilde{h}}(t) \in C[0,T]$, $||H(h(t)) - H({\tilde{h}}(t))||_T \le \eta ||h(t) - {\tilde{h}}(t) ||_T$

• (ii) There exists sufficiently large R such that the closed ball B(R, 0), centered at 0 with radius R, is mapped to itself by H(h(t)).

First, we show that H(h(t)) is a contraction. Because G(x) is an admissible test function, we can use Eq. A2 to compute

$$\begin{aligned}&\displaystyle \frac{\partial }{\partial t} \bigg | \int _0^1 G(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | \\&\quad \le \bigg | \displaystyle \frac{\partial }{\partial t} \int _0^1 G(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | \\&\quad \le \bigg | - \int _0^1 G'(x) j(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | \\&\qquad +\, \lambda \bigg | \int _0^1 G(x) \left[ G(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) +\left( h(t) \mu _t^{h}(\mathrm {d} x) - \tilde{h} (t) \mu _t^{\tilde{h}}(\mathrm {d} x)\right) \right] \bigg | \\&\quad \le (G')^* j^* \bigg | \int _0^1 \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | + \lambda G^* \bigg | \int _0^1 G(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | \\&\qquad +\, \lambda \bigg | \int _0^1 G(x) h(t) \left( \mu _t^h (\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | + \lambda \bigg | \int _0^1 G(x) \left( h(t) - {\tilde{h}}(t) \right) \mu _t^{{\tilde{h}}}(\mathrm{d}x) \bigg |. \end{aligned}$$

Because $h(t) \in C([0,T])$, we know that there exists a bound $B_{h} < \infty$ such that $h(t) \le ||h(t)||_T \le B_h$. From our a priori estimate on $\mu _t^{{\tilde{h}}}(\mathrm{d}x)$, we know that there is $M^T_{{\tilde{h}}}$ such that $\int _0^1 \mu _t^{{\tilde{h}}} (\mathrm{d}x) \le M^T_{{\tilde{h}}}$ for $t \in [0,T]$ and a corresponding bound $M^T_h$ for $\mu _t^h(\mathrm{d}x)$. Using these, we can now say that

$$\begin{aligned}&\displaystyle \frac{\partial }{\partial t} \bigg | \int _0^1 G(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg |\\&\quad \le (G')^* j^* \left( M^T_{h} + M^T_{{\tilde{h}}} \right) + \lambda \left( G^* + B_{h} \right) \bigg | \int _0^1 G(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | \\&\qquad +\, \lambda G^* M^T_{{\tilde{h}}} ||h(t) - {\tilde{h}}(t)||_T. \end{aligned}$$

If $||h(t) - {\tilde{h}}(t)||_T = 0$, we know that $h(t) = \tilde{h(t)}$ and $\mu _t^h(\mathrm{d}x) = \mu _t^{{\tilde{h}}}(\mathrm{d}x)$ by the uniqueness of solutions to Eq. A2, and we can correspondingly conclude $\int _0^1 G(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) = 0$ for $t \in [0,T]$. In the alternate case that $||h(t) - {\tilde{h}}(t)||_T > 0$, we know that there exists $W < \infty$ such that $(G')^* j^* \left( M^T_{h} + M^T_{{\tilde{h}}} \right) \le W ||h(t) - {\tilde{h}}(t)||_T$, and we can write that

$$\begin{aligned}&\displaystyle \frac{\partial }{\partial t} \bigg | \int _0^1 G(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | \\&\quad \le \lambda \left( G^* + B_{h} \right) \bigg | \int _0^1 G(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | \\&\quad +\, \left( W + \lambda G^* M^T_{{\tilde{h}}} \right) ||h(t) - {\tilde{h}}(t)||_T. \end{aligned}$$

By Grönwall’s inequality, we have that

$$\begin{aligned} \bigg | \int _0^1 G(x) \left( \mu _t^{h}(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | \le \left( \frac{W + \lambda G^* M_{{\tilde{h}}}^T}{\lambda \left( G^* + B_h \right) } \right) \left( e^{\lambda \left( G^* + ||h||_T \right) T} - 1\right) ||h(t) - {\tilde{h}}(t)||_T. \end{aligned}$$

For an $\eta \in (0,1)$, we can choose $T(\eta )$ close enough to 0 guarantees that

$$\begin{aligned} ||H(h(t)) - H({\tilde{h}}(t))||_T(\eta ):= & {} \sup _{t \in [0,T(\eta )]} \bigg |\int _0^1 G(x) \left( \mu _t^h(\mathrm{d}x) - \mu _t^{{\tilde{h}}}(\mathrm{d}x) \right) \bigg | \\\le & {} \eta ||h(t) - {\tilde{h}}(t)||_{T(\eta )}, \end{aligned}$$

which tells us that $H(h(t)) : C[0,T] \rightarrow C[0,T]$ is a contraction. Now, we show that H(h(t)) is a maps closed balls $B_R$ of radius R to itself. We compute

$$\begin{aligned} |H(h(t))| =\bigg | \int _0^1 G(x) \mu _t^h(\mathrm{d}x) \bigg | \le G* | \langle 1, \mu ^h_t \rangle | \le \sup _{||\Psi |_{\infty } \le 1} \langle \Psi , \mu _t^h \rangle := ||\mu _t^h||_{TV} \end{aligned}$$

Therefore, to estimate |H(h(t))|, it suffices to estimate $||\mu _t^h||_{TV}$. We use the variation of constants formula to compute that

$$\begin{aligned}&|\langle \Psi , \mu _t^h \rangle | = \bigg |\langle \Psi , \mu _0^h \rangle + \lambda \int _0^t \langle \left( P_{t-s}\Psi \right) (G(x) - h(t)), \mu _s^h \rangle \mathrm{d}s \bigg | \\&\quad \le ||\mu _0^h||_{TV} + \lambda \left( G^* + ||h||_T \right) \int _0^t \langle P_{t-s} \Psi , \mu _s^h \rangle \mathrm{d}s \end{aligned}$$

where we used that $P_{t} \Psi (x) = \Psi (\phi _t(x))$ to deduce that $||P_t \Psi ||_{\infty } \le ||\Psi ||_{\infty } \le 1$ and that $\langle P_t \Psi , \mu _0^h \rangle \le ||\mu _0^h||_{TV}$. Noting further that $\langle P_{t-s} \Psi , \mu _s \rangle \le ||P_{t-s} \Psi ||_{\infty } \int _0^1 \mu _s^h(\mathrm{d}x) \le ||\Psi ||_{\infty } M^T_{h} \le M^T_{h}$ for $||\Psi ||_{\infty } \le 1$, we find that

$$\begin{aligned} | \langle \Psi , \mu _t \rangle | \le ||\mu _0^h||_{TV} + \lambda \left( G^* + ||h||_T \right) M^T_{h} T. \end{aligned}$$

Because this is true for all test functions $\Psi$ satisfying $||\Psi ||_{\infty } \le 1$, we can combine our inequalities above and the definition of the $||\cdot ||_T$ norm to conclude that

$$\begin{aligned} ||H(h(t))||_T \le ||\mu _0^h||_{TV} + \lambda \left( G^* + ||h||_T \right) M^T_{h} T. \end{aligned}$$

Choosing $T_{\epsilon } < \left( \lambda \left( G^* + ||h||_T \right) M^T_{h} \right) ^{-1} \epsilon$ gives us that

$$\begin{aligned} ||H(h(t))||_{T_{\epsilon }} \le ||\mu _0||_{TV} + \epsilon , \end{aligned}$$

so we see that choosing a ball with radius $R \ge ||\mu _0||_{TV} + \epsilon$, then we see that H(h(t)) maps that closed ball $\{h(t) | ||h(t)||_T \le R\}$ to itself.

Combining the facts that H(h(t)) maps C[0, T] into C[0, T], it is a contraction mapping, and it maps sufficiently large closed balls $B_R$ to themselves, and that C[0, T] is complete with respect to the $||\cdot ||_T$ norm, we can apply the Banach fixed-point theorem to show that there exists a unique fixed point $h_{\flat }$ such that $H(h_{\flat }(t)) = \int _0^1 G(x) \mu _t^{h_{\flat }}(\mathrm{d}x) = h_{\flat }(t)$, which further means that $\mu _t^{h_{\flat }}(\mathrm{d}x)$ solves Eq. A1 and is unique. Because there exists a $T > 0$ such that the solution $\mu _t(\mathrm{d}x)$ exists and is unique, we can use the a priori estimate of Eq. A8 to applying a similar argument existence argument starting with an initial population at $\mu _{\frac{T}{2}}(\mathrm{d}x)$ and function h(t) defined for $t \in [\frac{T}{2}.\frac{3T}{2}]$ to demonstrate the existence of solutions on the time interval $[\frac{T}{2},\frac{3T}{2}]$, and we can continue this iteration to establish the existence of a unique solution $\mu _t(\mathrm{d}x)$ to Eq. A1 for any time $t \ge 0$.

Furthermore, because solutions to Eq. A2 satisfy the representation formula

$$\begin{aligned} \mu ^h_t(\mathrm{d}x)= & {} w_t^h(x)( \mu _0 \circ \phi _t^{-1})(\mathrm{d}x) \, \, {\mathrm {or}} \, \, \int _0^1 \Psi (x) \mu ^h_t(\mathrm{d}x) \\= & {} \int _0^1 \Psi (\phi _t(x)) \exp \left( \lambda \int _0^t \left[ G(\phi _s(x_0)) - h(s) \right] \mathrm{d}s \right) \mu _0(\mathrm{d}x), \end{aligned}$$

we can choose $h(t) = h_{\flat }(t)$ and use the fixed point relation $h_{\flat }(t) = H(h_{\flat }(t)) = \langle G(\cdot ) \rangle _{\mu ^{h_{\flat }}_t}$ to find that

$$\begin{aligned} \int _0^1 \Psi (x) \mu _t(\mathrm{d}x) = \int _0^1 \Psi (\phi _t(x)) \exp \left( \lambda \int _0^t \left[ G(\phi _s(x_0)) - \langle G(\cdot ) \rangle _{\mu ^{h_{\flat }}_s} \right] \mathrm{d}s \right) \mu _0(\mathrm{d}x). \end{aligned}$$

Because $\mu _t^{h_{\flat }}(\mathrm{d}x)$ is the unique solution to Eq. A1, we now see that the solution of Eq. A1 satisfies the implicit representation formula $\mu _t^{h_{\flat }}(\mathrm{d}x) = w_t(x)( \mu _0 \circ \phi _t^{-1})(\mathrm{d}x)$ of Eq. 16. $\square$

Integrals Along Simplified Characteristics

In this section, we show how to compute the integrals of solutions along the simplified characteristic curves $\Psi _t(k;x_0)$ from the PD and $\Xi _t(k;x_0)$ and $\Pi _t(k;x_0)$ from the HD game.

1.1 PD Integrals

We start with $\Psi _t(k;x)$, the solution for the logistic family serving as the faster and slower characteristics for the within-group replicator dynamics.

$$\begin{aligned} \int _0^t \Psi _s(k;x_0) \mathrm{d}s&= \int _0^t \left[ \frac{x_0}{x_0 + \left( 1 - x_0 \right) e^{k s}} \right] \mathrm{d}s \\&= x_0 \int _1^{x_0 + \left( 1 - x_0\right) e^{k t}} \frac{\mathrm{d}u}{ k u\left( u - x_0\right) } \, \, \text {(where } u = x_0 + \left( 1 - x_0\right) e^{k s }) \\&= \frac{1}{k} \int _1^{x_0 + \left( 1-x_0\right) e^{k t}} \left( - \frac{1}{u} + \frac{1}{u - x_0} \right) \mathrm{d}u \\&= \frac{1}{ k} \left[ -\log \left( u \right) + \log \left( u - x_0\right) \right] \bigg |_1^{x_0 + \left( 1 - x_0\right) e^{k t}} \\&=\left[ t - \frac{1}{k} \log \left( x_0 + \left( 1 - x_0\right) e^{ k t} \right) \right] . \end{aligned}$$

Knowing that $x_0$ can be written as $x_0 = \frac{x}{x + (1-x) e^{-kt}}$, we can plug in for $x_0$ above and conclude that

$$\begin{aligned} \int _0^t \Psi _s(k;x_0) \mathrm{d}s= & {} t - \frac{1}{k} \log \left( \frac{x}{x + (1-x)e^{-kt}} + \left[ \frac{(1-x)e^{-kt}}{x + (1-x)e^{-kt}}\right] e^{kt} \right) \\= & {} t + \frac{1}{k} \log \left( x + (1-x)e^{-kt} \right) . \end{aligned}$$

For the logistic ODE, we can also compute the integral of $\Psi _t(k\cdot x)$ as

$$\begin{aligned} \int _0^t \Psi _s(k;x_0)^2 \mathrm{d}s&= x_0^2 \int _0^t \frac{\mathrm{d}s}{\left( x_0 + \left( 1 - x_0 \right) e^{k s}\right) ^2} = \frac{x_0^2}{k} \int _1^{x_0 + \left( 1 - x_0\right) e^{k t}} \frac{\mathrm{d}u}{ u^2 \left( u - x_0\right) } \\&= \frac{1}{k} \int _1^{x_0 + \left( 1 - x_0\right) e^{k t}} \left( \frac{1}{u-x_0} - \frac{1}{u} - \frac{x_0}{u^2} \right) \mathrm{d}u \\&= \frac{1}{k} \left[ \log (u - x_0) - \log (u) + \frac{x_0}{u} \right] \bigg |_{1}^{x_0 + (1 - x_0) e^{k t}} \\&= t - \frac{1}{k} \log (x_0 + \left( 1-x_0\right) e^{kt}) + \frac{x_0}{k} \left[ \frac{1}{x_0 + (1-x_0)e^{kt}} - 1\right] . \end{aligned}$$

Using the formula $x_0 = \frac{x}{x + (1-x) e^{-kt}}$ and our expression for $\int _0^t \Psi _t(k;x_0)dt$, we can further see that

$$\begin{aligned} \int _0^t \Psi _s(k;x_0)^2 \mathrm{d}s&= t + \frac{1}{k} \log \left( x + (1-x)e^{-kt} \right) \\&\quad +\, \frac{1}{k} \left[ \frac{x}{x + (1-x) e^{-kt}} \right] \left[ \frac{x + (1-x)e^{-kt}}{x + \left( (1-x) e^{-kt}\right) e^{kt}} - 1 \right] \\&= t + \frac{1}{k} \log \left( x + (1-x)e^{-kt} \right) + \frac{x}{k} \left[ 1 - \frac{1}{x + (1-x)e^{-kt}} \right] . \end{aligned}$$

1.2 HD Integrals

1.2.1 Dynamics Above Within-Group Equilibrium at $\frac{\beta }{|\alpha |}$

We first consider $\Xi _t(k,x)$, the faster and slower characteristic curves for the within-group replicator dynamics for the HD games when the level of cooperation exceeds the within-group equilibrium. Because we are interested in dynamics in the interval $[\frac{\beta }{|\alpha |},1]$, we can choose a rescaled state variable $X := \frac{|\alpha | x - \beta }{|\alpha | - \beta }$. In terms of our new variable, the ODE from Eq. 31 takes the following form:

$$\begin{aligned} \displaystyle \frac{\mathrm{d} X(t)}{\mathrm{d} t} = - \left( |\alpha | - \beta \right) k X \left( 1 - X \right) \, \,, \, \, X(0) = X_0 := \frac{|\alpha | x_0 - \beta }{|\alpha | - \beta } \end{aligned}$$

(B1)

whose solution is given by $\Psi _t\left( \left( |\alpha | - \beta \right) k;X_0\right)$. Because $\Xi _t(k;x_0)$ describes the evolution of x(t) and $\Psi _t\left( \left( |\alpha | - \beta \right) k;X_0\right)$ describes the evolution of $X(t) = \frac{|\alpha | x(t) - \beta }{|\alpha | - \beta }$, we see that we can relate the two named solutions by

$$\begin{aligned} \Xi _t(k;x_0) = \frac{\beta }{|\alpha |} + \left( \frac{|\alpha | - \beta }{|\alpha |}\right) \Psi _t\left( \left( |\alpha | - \beta \right) k;X_0 \right) \end{aligned}$$

(B2)

and we also have that

$$\begin{aligned} \Xi _t(k;x_0)^2 = \frac{\beta ^2}{|\alpha |^2} + 2 \frac{\beta }{|\alpha |} \left( \frac{|\alpha | - \beta }{|\alpha |}\right) \Psi _t\left( \left( |\alpha | - \beta \right) k;X_0 \right) + \left( \frac{|\alpha | - \beta }{|\alpha |}\right) ^2 \Psi _t\left( \left( |\alpha | - \beta \right) k;X_0 \right) ^2. \end{aligned}$$

(B3)

Using Eq. B2 and our result from Eq. 41, we can compute

$$\begin{aligned} \int _0^t \Xi _s\left( k;x\right) \mathrm{d}s&= \int _0^t \left[ \frac{\beta }{|\alpha |} \right. \\&\quad \left. + \left( \frac{|\alpha | - \beta }{|\alpha |} \right) \Psi _s\left( \left( |\alpha | - \beta \right) k;X_0\right) \right] \mathrm{d}s \\&= \left( \frac{\beta }{|\alpha |}\right) t \\&\quad +\, \left( \frac{|\alpha | - \beta }{|\alpha |} \right) \int _0^t \Psi _s\left( \left( |\alpha | - \beta \right) k;X_0\right) \mathrm{d}s \\&= t + \left( \frac{|\alpha | - \beta }{|\alpha |} \right) \left( \frac{1}{\left( |\alpha | - \beta \right) k} \right) \\&\quad \log \left( X + (1-X) e^{-\left( |\alpha | - \beta \right) t} \right) . \end{aligned}$$

Then, using $X = \frac{|\alpha |x - \beta }{|\alpha | - \beta }$, we can deduce that

$$\begin{aligned} \int _0^t \Xi _s\left( k;x\right) \mathrm{d}s&= t + \frac{1}{|\alpha | k} \log \left( \frac{|\alpha | x - \beta + |\alpha | \left( 1-x\right) e^{-kt}}{|\alpha | - \beta } \right) . \end{aligned}$$

Using Eq. B3 and the integrals calculated in Eqs. 41 and 42, we can also see that

$$\begin{aligned} \int _0^t \Xi _s\left( k;x\right) ^2 \mathrm{d}s&= \left( \frac{\beta ^2}{|\alpha |^2}\right) t + \frac{2\beta \left( |\alpha | - \beta \right) }{|\alpha |^2} \int _0^t \Psi _s\left( \left( |\alpha | - \beta \right) k; X_0 \right) \mathrm{d}s \\&\quad +\, \left( \frac{|\alpha | - \beta }{|\alpha |} \right) ^2 \int _0^t \Psi _s\left( \left( |\alpha | - \beta \right) k; X_0 \right) ^2 \mathrm{d}s \\&= t + \left( \frac{|\alpha | + \beta }{|\alpha |^2 k} \right) \log \left( X + (1-X) e^{-\left( |\alpha | - \beta \right) kt} \right) \\&\quad +\, \left( \frac{|\alpha | - \beta }{|\alpha |^2} \right) \frac{X}{k} \left[ 1 - \frac{1}{X + (1-X) e^{-\left( |\alpha | - \beta \right) k t}} \right] . \end{aligned}$$

Using that $X = \frac{|\alpha | x - \beta }{|\alpha | - \beta }$, we are able to see that

$$\begin{aligned}&\int _0^t \Xi _s\left( k;x\right) ^2 \mathrm{d}s = t + \left( \frac{|\alpha | + \beta }{|\alpha |^2 k} \right) \left[ \log \left( |\alpha | x - \beta + |\alpha | (1-x) e^{-\left( |\alpha | - \beta \right) k t}\right) - \log (|\alpha | - \beta ) \right] \\&\quad - \frac{1}{|\alpha | k} \left[ \frac{\left( 1-x\right) \left( |\alpha | x - \beta \right) \left( 1 - e^{- \left( |\alpha | - \beta \right) k t} \right) }{|\alpha | x - \beta + |\alpha | \left( 1 - x \right) e^{- \left( |\alpha | - \beta \right) k t}} \right] . \end{aligned}$$

1.2.2 Dynamics Below $x^{\mathrm{eq}} = \frac{\beta }{|\alpha |}$

For the HD dynamics below the within-group dynamics, we choose rescaled state variable $X = \frac{|\alpha |}{\beta }x$. In terms of the new variable, the within-group dynamics of Eq. 36 can be written as

$$\begin{aligned} \displaystyle \frac{\mathrm{d} X(t)}{\mathrm{d} t} = \beta k X\left( 1 - X\right) \, \, , \, \, X_0 = \frac{|\alpha |}{\beta } x_0. \end{aligned}$$

(B4)

Because Eq. B4 can be obtained from Eq. B1 by reversing time, we see that solutions to Eq. B4 are the backward-in-time solution to the logistic ODE, $\Psi _t^{-1}(k;X_0)$. Therefore, we have that $\Pi _t(k;x_0) = \Psi _T^{-1}(\beta k;X_0)$, so we see the equivalence of the characteristic curves

$$\begin{aligned} \Pi _t(k;x_0) = \left( \frac{\beta }{|\alpha |}\right) \Psi _t^{-1}\left( \beta k ; X_0 \right) \end{aligned}$$

(B5)

and, using Eq. 20, we can now compute the following integrals along the characteristic curves $\Pi _t(k;x_0)$

$$\begin{aligned} \int _0^t \Pi _s(k;x_0) \mathrm{d}s&= \frac{\beta }{|\alpha |} \int _0^t \Psi _s^{-1}\left( \beta k ; X_0 \right) \mathrm{d}s \\&= \frac{\beta }{|\alpha |} \int _0^t \left[ \frac{X_0}{X_0 + \left( 1 - X_0 \right) e^{-\beta k s}} \right] \mathrm{d}s \\&= - \frac{\beta X_0}{\alpha } \int _1^{X_0 + \left( 1 - X_0\right) e^{\beta k t}} \frac{dU}{\beta k U\left( U - X_0\right) } \, \, \text {(where } U = X_0 + \left( 1 - X_0\right) e^{\beta k s }) \\&= \frac{1}{|\alpha | k} \int _1^{X_0 + \left( 1-X_0\right) e^{\beta k t}} \left( \frac{1}{U} - \frac{1}{U - X_0} \right) dU \\&= \frac{1}{|\alpha | k} \left[ \log \left( U \right) - \log \left( U - X_0\right) \right] \bigg |_1^{X_0 + \left( 1 - X_0\right) e^{-\beta k t}} \\&= \frac{1}{|\alpha |} \left[ \beta t + \frac{1}{k} \log \left( X_0 + \left( 1 - X_0\right) e^{-\beta k t} \right) \right] . \end{aligned}$$

Now, we can write $X_0$ in terms of t and X using $X_0 = \Psi _t(\beta k ;X) = \frac{X}{X + \left( 1-X\right) e^{\beta k t}}$ and use $X = \frac{|\alpha |}{\beta } x$ to tell us that

$$\begin{aligned} \int _0^t \Pi _s(k;x_0) \mathrm{d}s&= \frac{1}{|\alpha |} \left[ \beta t + \frac{1}{k} \log \left( \frac{X}{X + \left( 1-X\right) e^{\beta k t}} + \left( \frac{\left( 1-X\right) e^{\beta k t}}{X + \left( 1-X\right) e^{\beta k t}} \right) e^{-\beta k t} \right) \right] \\&= \frac{1}{|\alpha |} \left[ \beta t - \frac{1}{k} \log \left( X + \left( 1-X\right) e^{\beta k t} \right) \right] \\&-\frac{1}{|\alpha | k} \log \left( \left( 1-X\right) + Xe^{-\beta k t} \right) \\&= - \frac{1}{|\alpha | k} \left[ \log \left( \left( \beta -|\alpha |\right) + |\alpha |e^{-\beta k t} \right) - \log (\beta ) \right] . \end{aligned}$$

We also want to calculate

$$\begin{aligned} \int _0^t \Pi _s(k;x_0)^2 \mathrm{d}s&= \frac{\beta ^2}{|\alpha |^2} \int _0^t \Psi _s^{-1}\left( \beta k ; X_0 \right) ^2 \mathrm{d}s \\&= \frac{\beta ^2}{|\alpha |^2} \int _0^t \left[ \frac{X_0^2}{\left( X_0 + \left( 1 - X_0 \right) e^{-\beta k s}\right) ^2} \right] \mathrm{d}s \\&= - \frac{\beta ^2 X_0^2}{|\alpha |^2} \int _1^{X_0 + \left( 1 - X_0\right) e^{\beta k t}} \frac{dU}{\beta k U^2\left( U - X_0\right) } \, \, \text {(where } U = X_0 + \left( 1 - X_0\right) e^{\beta k s }) \\&= \frac{\beta }{|\alpha |^2 k} \int _1^{X_0 + \left( 1 - X_0\right) e^{-\beta k t}} \left( \frac{X_0}{U^2} + \frac{1}{U} - \frac{1}{U-X_0} \right) dU \\&= \frac{\beta }{|\alpha |^2 k} \left[ \frac{X_0}{U} + \log (U) - \log (U - X_0) \right] \bigg |_1^{X_0 + \left( 1-X_0\right) e^{-\beta k t}} \\&= \frac{\beta }{|\alpha |^2 k} \left[ \beta k t + \log \left( X_0 + \left( 1-X_0\right) e^{-\beta k t} \right) + X_0 \left( 1 - \frac{1}{X_0 + \left( 1 - X_0 \right) e^{- \beta k t} } \right) \right] . \end{aligned}$$

From Eq. B5, we can also express $X_0$ in terms of t and X as $X_0 = \frac{X}{X+ (1-X)e^{\beta k t}}$. Plugging this in for $X_0$ above lets us write that

$$\begin{aligned} \int _0^t \Pi _s(k;x_0)^2 \mathrm{d}s&= \frac{\beta }{|\alpha |^2 k} \left[ \beta k t - \log \left( X + \left( 1-X\right) e^{\beta k t} \right) + \frac{X(1-X)\left( 1 - e^{\beta k t}\right) }{X + (1-X) e^{\beta k t}} \right] \\&= \frac{\beta }{|\alpha |^2 k} \left[ - \log \left( (1-X) + Xe^{-\beta k t} \right) + \frac{X(1-X)\left( 1 - e^{\beta k t}\right) }{X + (1-X) e^{\beta k t}} \right] . \end{aligned}$$

Then, using $X = \frac{|\alpha |}{\beta }x$, we can finally see that

$$\begin{aligned} \int _0^t \Pi _s(k;x_0)^2 \mathrm{d}s= & {} - \frac{\beta }{|\alpha |^2 k} \left[ \log \left( \beta - |\alpha | x + |\alpha | x e^{- \beta k t} \right) - \log (\beta ) \right] \nonumber \\&- \frac{1}{|\alpha | k} \left[ \frac{x \left( \beta - |\alpha | x \right) \left( e^{\beta k t} - 1 \right) }{|\alpha | x + \left( \beta - |\alpha | x \right) e^{\beta k t}} \right] . \end{aligned}$$

(B6)

1.3 Comparison Principles for Average Group Payoff

Here, we use the comparison principles from Sect. 3.1 to describe the average payoff $G(\phi _t(x))$ of a group with a fraction of cooperators at time t given by $\phi _t(x)$. This will allow us to understand how the solutions along the exact characteristics can be related to the group payoff along trajectories on simpler time-dependent curves with explicit solution formulas. For the various cases of the PD, we use the signs of $\gamma$ and $\alpha$ and the rankings of characteristic curves given by Eqs. 23 and 28 to find the following bounds for $G(\phi _t(x_0))$ along our known simpler solution curves:

$$\begin{aligned} \begin{aligned} \gamma \Psi _t\left( |\beta | + |\alpha |;x_0 \right) - |\alpha | \Psi _t(|\beta |;x_0)^2 \le G(\phi _t(x_0))&\le \gamma \Psi _t\left( |\beta |;x_0 \right) - |\alpha | \Psi _t(|\beta | + |\alpha |;x_0)^2 \text {: (Case I PD)} \\ \gamma \Psi _t\left( |\beta |;x_0 \right) + \alpha \Psi _t(|\beta |;x_0)^2 \le G(\phi _t(x_0))&\le \gamma \Psi _t\left( |\beta | - \alpha ;x_0 \right) + \alpha \Psi _t(|\beta | - \alpha ;x_0)^2 \text {: (Case III PD)} \\ -|\gamma | \Psi _t\left( |\beta |-\alpha ;x_0 \right) + \alpha \Psi _t(|\beta |;x_0)^2 \le G(\phi _t(x_0))&\le -|\gamma | \Psi _t\left( |\beta |;x_0 \right) + \alpha \Psi _t(|\beta |-\alpha ;x_0)^2 \text {: (Case IV PD)}. \end{aligned} \end{aligned}$$

In Fig. 14, we illustrate example trajectories for $G(\phi _t(x_0))$ and corresponding upper and lower bounds for the Case I (left) and Case III (right) PDs. In the example for the Case I PD, we can see that $G(\phi _t(x_0))$ and its bounds are non-monotonic in time, corresponding to the characteristic curves traversing the intermediate optimum composition for average group payoff.

Fig. 14

Comparison of group payoff along characteristic curves $G(\phi _t(x_0))$ with expressions along exactly solvable curves for the PD game. Parameters correspond to Case I (left) and Case III (right) (color figure online)

We can similarly bound $G(\phi _t(x_0))$ for the HD game using the ranking of characteristic curves from Eqs. 34 and 39 and the fact that $\gamma > 0$ and $\alpha < 0$ for all HD games. We find that

$$\begin{aligned} \gamma \Pi _t\left( 1 - \tfrac{\beta }{|\alpha |};x_0\right) - |\alpha | \Pi _t(1;x_0)^2&\le G(\phi _t(x_0)) \le \gamma \Pi _t\left( 1;x_0 \right) - |\alpha | \Pi _t\left( 1 - \tfrac{\beta }{|\alpha |};x_0\right) ^2 : \, \, x_0 < \tfrac{\beta }{|\alpha |} \\ \gamma \Xi _t\left( 1;x_0\right) - |\alpha | \Xi _t\left( 1- \tfrac{\beta }{|\alpha |};x_0\right) ^2&\le G(\phi _t(x_0)) \le \gamma \Pi _t\left( 1- \tfrac{\beta }{|\alpha |};x_0 \right) - |\alpha | \Pi _t\left( 1 ;x_0\right) ^2 : \, \, x_0 > \tfrac{\beta }{|\alpha |} \end{aligned}$$

In Fig. 15, we illustrate example trajectories for $G(\phi _t(x_0))$ and corresponding upper and lower bounds for the group payoff both above the within-group equilibrium (left) and below the within-group equilibrium (right).

Fig. 15

Comparison of group payoff along characteristic curves $G(\phi _t(x_0))$ with expressions for upper and lower bounds of group payoff in terms of exactly solvable curves for the HD game. Dynamics described above within-group equilibrium (left) and below the within-group equilibrium (right). Long-time behavior for $G(\phi _t(x_0))$ and the bounds agree upon group payoff $G(x^{\mathrm{eq}})$ at within-group equilibrium for HD game (color figure online)