Metastability of Logit Dynamics for Coordination Games

Abstract

Logit dynamics (Blume in Games Econ Behav 5:387–424, 1993) are randomized best response dynamics for strategic games: at every time step a player is selected uniformly at random and she chooses a new strategy according to a probability distribution biased toward strategies promising higher payoffs. This process defines an ergodic Markov chain, over the set of strategy profiles of the game, whose unique stationary distribution is the long-term equilibrium concept for the game. However, when the mixing time of the chain is large (e.g., exponential in the number of players), the stationary distribution loses its appeal as equilibrium concept, and the transient phase of the Markov chain becomes important. It can happen that the chain is “metastable”, i.e., on a time-scale shorter than the mixing time, it stays close to some probability distribution over the state space, while in a time-scale multiple of the mixing time it jumps from one distribution to another. In this paper we give a quantitative definition of “metastable probability distributions” for a Markov chain and we study the metastability of the logit dynamics for some classes of coordination games. We first consider a pure n-player coordination game that highlights the distinctive features of our metastability notion based on distributions. Then, we study coordination games on the clique without a risk-dominant strategy (which are equivalent to the well-known Glauber dynamics for the Curie–Weiss model) and coordination games on a ring (both with and without risk-dominant strategy).

Appendices

Appendix A: Markov Chain Summary

In this section we recall some basic facts about Markov chains. For a more detailed treatment, we refer the reader to [29].

1.1 Total Variation Distance

The total variation distance $\left\| \mu - \nu \right\| _\mathrm{TV}$ between two probability distributions $\mu$ and $\nu$ on $\Omega$ is defined as

$$\begin{aligned} \left\| \mu -\nu \right\| _\mathrm{TV}:=\max _{A\subset \Omega } |\mu (A)-\nu (A)| = \frac{1}{2} \sum _{x\in \Omega } |\mu (x)-\nu (x)|. \end{aligned}$$

The total variation distance is actually a distance and, in particular, the triangle inequality holds. That is, the following simple fact holds.

Fact 1

For distributions $\mu _1,\mu _2,$ and $\mu _3$, it holds that

$$\begin{aligned} \left\| \mu _1-\mu _3\right\| _\mathrm{TV}\leqslant \left\| \mu _1-\mu _2\right\| _\mathrm{TV}+\left\| \mu _2-\mu _3\right\| _\mathrm{TV}. \end{aligned}$$

A stochastic matrix P over $\Omega$ is a non-negative matrix in which rows and columns are indexed by elements of $\Omega$ and such that, for all $x\in \Omega$, it holds that

$$\begin{aligned} \sum _{y\in \Omega } P(x,y)=1. \end{aligned}$$

Fact 2

For all distributions $\mu$ and $\nu$ on $\Omega$ and all stochastic matrices P it holds that

$$\begin{aligned} \left\| \mu P-\nu P\right\| _\mathrm{TV}\leqslant \left\| \mu -\nu \right\| _\mathrm{TV}. \end{aligned}$$

1.2 Mixing Time

Consider a Markov chain $\mathcal {M}=\{X_t\}$ with finite state space $\Omega$ and transition matrix P. We stress that P is stochastic matrix over $\Omega$ and we will often identify $\mathcal {M}$ with P. It is a classical result that if $\mathcal {M}$ is irreducible and aperiodic ² (also called ergodic) there exists an unique stationary distribution; that is, a distribution $\pi$ on $\Omega$ such that $\pi \cdot P=\pi$.

An ergodic Markov chain $\mathcal {M}$ converges to its stationary distribution $\pi$; specifically, there exist constants C and $0<\alpha <1$ such that

$$\begin{aligned} d(t)\leqslant C\cdot \alpha ^t, \end{aligned}$$

where

$$\begin{aligned} d(t)= \max _{x\in \Omega } \left\| P^t(x,\cdot ) - \pi \right\| _\mathrm{TV} \end{aligned}$$

and $P^t(x,\cdot )$ is the distribution at time t of the Markov chain starting at x. For $0 < \varepsilon \leqslant 1$, the mixing time is defined as

$$\begin{aligned} t_{\text {mix}}(\varepsilon ) = \min \{t\in \mathbb {N}:d(t)\leqslant \varepsilon \}. \end{aligned}$$

It is usual to set $\varepsilon = 1/4$ and to write ${t_\mathrm{mix}}$ for ${t_\mathrm{mix}}(1/4)$.

Fact 3

For $0<\varepsilon \leqslant 1$,

$$\begin{aligned} {t_\mathrm{mix}}(\varepsilon )\leqslant \lceil \log _2\varepsilon ^{-1}\rceil \cdot {t_\mathrm{mix}}. \end{aligned}$$

1.3 Coupling

A coupling of two probability distributions $\mu$ and $\nu$ on $\Omega$ is a pair of random variables (X, Y) defined on $\Omega \times \Omega$ such that the marginal distribution of X is $\mu$ and the marginal distribution of Y is $\nu$. A well-known property of coupling is given by the following theorem (see, e.g., [29, Proposition 4.7]).

Theorem A.1

Let $\mu$ and $\nu$ two probability distributions on $\Omega$. Then

$$\begin{aligned} \left\| \mu - \nu \right\| _\mathrm{TV} = \inf \left\{ \mathbf {P}_{} \left( X \ne Y \right) :\left( X, Y\right) ~\text {is a coupling of}~\mu ~\text {and}~\nu \right\} \end{aligned}$$

Appendix B: Biased Birth-and-Death Chains

In this section we consider birth-and-death chains with state space $\Omega = \{0,1, \dots , n\}$ (see Chapter 2.5 in [29] for a detailed description of such chains). For $k \in \{1, \dots , n-1 \}$ let $p_k = \mathbf {P}_{k} \left( X_1 = k+1 \right)$, $q_k = \mathbf {P}_{k} \left( X_1 = k-1 \right)$, and $r_k = 1 - p_k - q_k = \mathbf {P}_{k} \left( X_1 = k \right)$. We will be interested in the probability that the chain starting at some state $h \in \Omega$ hits state n before state 0, namely $\mathbf {P}_{k} \left( X_{\tau _{0,n}} = n \right)$ where $\tau _{0,n} = \min \{ t \in \mathbb {N} :X_t \in \{0,n\} \}$.

We start by giving an exact formula for such probability for the case when $p_k$ and $q_k$ do not depend on k.

Lemma B.1

Suppose for all $k \in \{ 1, \dots , n-1 \}$ it holds that $p_k = \varepsilon$ and $q_k = \delta$, for some $\varepsilon$ and $\delta$ with $\varepsilon + \delta \leqslant 1$. Then the probability the chain hits state n before state 0 starting from state $h \in \Omega$ is

$$\begin{aligned} \mathbf {P}_{h} \left( X_{\tau _{0,n}} = n \right) = \frac{1 - \left( \delta / \varepsilon \right) ^h}{1 - \left( \delta / \varepsilon \right) ^n}. \end{aligned}$$

The proof of this result, that is showed below for sake of completeness, uses standard arguments (see, e.g., [38, p.159]).

Proof

Let $\alpha _k$ be the probability to reach state n before state 0 starting from state k, i.e.

$$\begin{aligned} \alpha _k = \mathbf {P}_{k} \left( X_{\tau _{0,n}} = n \right) . \end{aligned}$$

Observe that for $k = 1, \dots , n-1$ we have

$$\begin{aligned} \alpha _k = \delta \cdot \alpha _{k-1} + \varepsilon \cdot \alpha _{k+1} + \left( 1 - (\delta + \varepsilon ) \right) \alpha _k. \end{aligned}$$

(13)

Hence

$$\begin{aligned} \varepsilon \cdot \alpha _k - \delta \cdot \alpha _{k-1} = \varepsilon \cdot \alpha _{k+1} - \delta \cdot \alpha _k \end{aligned}$$

with boundary conditions $\alpha _0 = 0$ and $\alpha _n = 1$. If we name $\Delta _k = \varepsilon \cdot \alpha _k - \delta \cdot \alpha _{k-1}$ we have $\Delta _k = \Delta _{k+1}$ for all k. By simple calculation and using that $\alpha _0 = 0$ it follows that

$$\begin{aligned} \alpha _k = \frac{\Delta }{\varepsilon } \sum _{i = 0}^{k-1}\left( \frac{\delta }{\varepsilon } \right) ^i = \frac{\Delta }{\varepsilon - \delta } \left( 1 - (\delta / \varepsilon )^k \right) . \end{aligned}$$

From $\alpha _n = 1$ we get

$$\begin{aligned} \Delta = \frac{\varepsilon - \delta }{\left( 1 - (\delta / \varepsilon )^n \right) }. \end{aligned}$$

Hence

$$\begin{aligned} \alpha _k = \frac{1 - \left( \delta / \varepsilon \right) ^k}{1 - \left( \delta / \varepsilon \right) ^n}. \end{aligned}$$

$\square$

Lemma B.2

Suppose for all $k \in \{ 1, \dots , n-1 \}$ it holds that $p_k \ge \varepsilon$ and $q_k \leqslant \delta$, for some $\varepsilon$ and $\delta$ with $\varepsilon + \delta \leqslant 1$. Then the probability to hit state n before state 0 starting from state $h \in \Omega$ is

$$\begin{aligned} \mathbf {P}_{h} \left( X_{\tau _{0,n}} = n \right) \ge \frac{1 - \left( \delta / \varepsilon \right) ^h}{1 - \left( \delta / \varepsilon \right) ^n}. \end{aligned}$$

Proof

Let $\{Y_t\}$ be a birth-and-death chain with the same state space as $\{X_t\}$ but transition rates

$$\begin{aligned} \mathbf {P}_{k} \left( Y_1 = k-1 \right) = \delta ; \qquad \mathbf {P}_{k} \left( Y_1 = k+1 \right) = \varepsilon . \end{aligned}$$

Fig. 1

Partition for the coupling in Lemma B.2

Consider the following coupling of $X_t$ and $Y_t$: When $(X_t,Y_t)$ is at state (k, h), consider the two [0, 1] intervals, each one partitioned in three subintervals as in Fig. 1. Let U be a uniform random variable over the interval [0, 1] and choose the update for the two chains according to position of U in the two intervals. Observe that, since $p_k \ge \varepsilon$ and $q_k \leqslant \delta$, if the two chains start at the same state $h \in \Omega$, i.e. $(X_0, Y_0) =(h,h)$, then at every time t it holds that $X_t \ge Y_t$. Hence if chain $Y_t$ hits state n before state 0, then chain $X_t$ hits state n before state 0 as well. More formally, let $\tau _{0,n}$ and $\hat{\tau }_{0,n}$ be the random variables indicating the first time chains $X_t$ and $Y_t$ respectively hit state 0 or n; then

$$\begin{aligned} \left\{ Y_{\hat{\tau }_{0,n}} = n \right\} \Rightarrow \left\{ X_{\tau _{0,n}} = n \right\} . \end{aligned}$$

Thus

$$\begin{aligned} \mathbf {P}_{h} \left( X_{\tau _{0,n}} = n \right) \ge \mathbf {P}_{h} \left( Y_{\hat{\tau }_{0,n}} = n \right) \ge \frac{1 - \left( \delta / \varepsilon \right) ^h}{1 - \left( \delta / \varepsilon \right) ^n}, \end{aligned}$$

where in the last inequality we used Lemma B.1. $\square$

Lemma B.3

Suppose for all $k \in \{ 1, \dots , n-1 \}$ it holds that $q_k / p_k \leqslant \alpha$, for some $\alpha < 1$. Then the probability to hit state 0 before state n starting from state $h \in \Omega$ is

$$\begin{aligned} \mathbf {P}_{h} \left( X_{\tau _{0,n}} = 0 \right) \leqslant \alpha ^h. \end{aligned}$$

Proof

Let $\hat{p}_k = \frac{p_k}{p_k + q_k}$ and $\hat{q}_k = \frac{q_k}{p_k + q_k}$ and let $\{Y_t\}$ be the birth-and-death chain with transition rates $\hat{p}_k$ and $\hat{q}_k$.

Let $\{U_t\}$ be an array of random variables such that $U_t = - 1$ with probability $q_{{pg{\tiny } Y_t}}$, $U_t = +1$ with probability $p_{{ Y_t}}$ and $U_t = 0$ with remaining probability. We will use $U_t$ to update chains $X_t$ and $Y_t$ at different time steps. Specifically, we denote with u the index of the first variables $U_t$ not used for updating $X_t$ (thus, at the beginning $u = 1$) and : set $Y_{t + 1} = Y_t + U_t$; for chain $X_t$, we toss a coin that gives head with probability $p_{{ X_t}} + q_{{ X_t}}$ and if it gives tail we set $X_{t+1} = X_t$, otherwise we set $X_{t+1} = X_t + U_u$. Roughly speaking, we have that the chain $X_t$ follows the path traced by chain $Y_t$: indeed, it is easy to see that, if they start at the same place, the sequence of states visited by the two chains is the same and in the same order. Hence chain $X_t$ hits state 0 before state n if and only if chain $Y_t$ hits state 0 before state n and thus $\mathbf {P}_{k} \left( X_{\tau ^X_{0,n}} = 0 \right) = \mathbf {P}_{k} \left( Y_{\tau ^Y_{0,n}} = 0 \right)$.

Finally, observe that $\frac{\hat{q}_k}{\hat{p}_k} = \frac{q_k}{p_k} \leqslant \alpha$ and $\hat{p}_k + \hat{q}_k = 1$. Hence, for every $k \in \{1, \ldots , n-1\}$, we have that $\hat{p}_k \ge \frac{1}{1+\alpha }$ and $\hat{q}_k \leqslant \frac{\alpha }{1+\alpha }$. This implies, from Lemma B.2, that, from any state $h \in \Omega$,

$$\begin{aligned} \mathbf {P}_{h} \left( Y_{\tau ^Y_{0,n}} = n \right) \ge \frac{1 - \alpha ^h}{1 - \alpha ^n} \ge 1 - \alpha ^h. \end{aligned}$$

The lemma follows. $\square$

Appendix C: Ehrenfest Urns

The Ehrenfest urn is the Markov chain with state space $\Omega = \{ 0,1,\dots , n\}$ that, when at state k, moves to state $k-1$ or $k+1$ with probability k / n and $(n-k)/n$ respectively (see, for example, Section 2.3 in [29] for a detailed description). The next lemma gives an upper bound on the probability that the Ehrenfest urn starting at state k hits state 0 or n within time step t.

Lemma C.1

Let $\{Z_t \}$ be the Ehrenfest urn over $\{0,1, \dots , n\}$ and let $\tau _{0,n}$ be the first time the chain hits state 0 or state n. Then for every $k \ge 1$ it holds that

$$\begin{aligned} \mathbf {P}_{k} \left( \tau _{0,n} < n \log n + cn \right) \leqslant \frac{c'}{n}. \end{aligned}$$

for suitable positive constants c and $c'$.

Proof

First observe that for any $t \ge 3$ the probability of hitting 0 or n before time t for the chain starting at 1 is only ${\mathcal O}(1/n)$ larger than for the chain starting at 2, which in turn is only ${\mathcal O}(1/n)$ larger than for the chain starting at 3. Indeed, by conditioning on the first step of the chain, we have

$$\begin{aligned} \mathbf {P}_{1} \left( \tau _{0,n}< t \right)&= \mathbf {P}_{1} \left( \tau _{0,n}< t \mid Z_1 = 0 \right) \mathbf {P}_{1} \left( Z_1 = 0 \right) \\&\quad \,\, + \mathbf {P}_{1} \left( \tau _{0,n}< t \mid Z_1 = 2 \right) \mathbf {P}_{1} \left( Z_1 = 2 \right) \\&= \frac{1}{n} + \frac{n-1}{n} \mathbf {P}_{2} \left( \tau _{0,n}< t-1 \right) \leqslant \frac{1}{n} + \mathbf {P}_{2} \left( \tau _{0,n} < t \right) \end{aligned}$$

and

$$\begin{aligned} \mathbf {P}_{2} \left( \tau _{0,n}< t \right)&= \mathbf {P}_{2} \left( \tau _{0,n}< t \mid Z_1 = 1 \right) \mathbf {P}_{2} \left( Z_1 = 1 \right) \\&\quad \, + \mathbf {P}_{2} \left( \tau _{0,n}< t \mid Z_1 = 3 \right) \mathbf {P}_{2} \left( Z_1 = 3 \right) \\&= \frac{2}{n} \mathbf {P}_{1} \left( \tau _{0,n}< t-1 \right) + \frac{n-2}{n} \mathbf {P}_{3} \left( \tau _{0,n}< t-1 \right) \\&\leqslant \frac{2}{n} \left( \frac{1}{n} + \mathbf {P}_{2} \left( \tau _{0,n}< t \right) \right) + \frac{n-2}{n} \mathbf {P}_{3} \left( \tau _{0,n} < t \right) . \end{aligned}$$

Hence,

$$\begin{aligned} \mathbf {P}_{2} \left( \tau _{0,n}< t \right)&\leqslant \frac{2}{n-2} + \mathbf {P}_{3} \left( \tau _{0,n}< t \right) \leqslant \frac{3}{n} + \mathbf {P}_{3} \left( \tau _{0,n}< t \right) ; \\ \mathbf {P}_{1} \left( \tau _{0,n}< t \right)&\leqslant \frac{4}{n} + \mathbf {P}_{3} \left( \tau _{0,n} < t \right) . \end{aligned}$$

Moreover observe that the probability that the chain starting at k hits state 0 or n before time t is symmetric, i.e. $\mathbf {P}_{k} \left( \tau _{0,n}< t \right) = \mathbf {P}_{n-k} \left( \tau _{0,n} < t \right)$, and it is decreasing for k that goes from 0 to n / 2. In particular, for every k such that $3 \leqslant k \leqslant n-3$ it holds that $\mathbf {P}_{k} \left( \tau _{0,n}< t \right) \leqslant \mathbf {P}_{3} \left( \tau _{0,n} < t \right)$. Now we show that $\mathbf {P}_{3} \left( \tau _{0,n} < n \log n + cn \right) = {\mathcal O}(1/n)$ and this will complete the proof.

First observe that

$$\begin{aligned} \mathbf {P}_{3} \left( \tau _{0,n}< t \right) = \mathbf {P}_{3} \left( \tau _{0,n}< t \wedge \tau _0< \tau _n \right) + \mathbf {P}_{3} \left( \tau _{0,n}< t \wedge \tau _n < \tau _0 \right) , \end{aligned}$$

where $\tau _0$ and $\tau _n$ are the first time the chain hits state 0 and state n, respectively. Then

$$\begin{aligned} \mathbf {P}_{3} \left( \tau _{0,n}< t \right)\leqslant & {} \mathbf {P}_{3} \left( \tau _0< t \right) + \mathbf {P}_{3} \left( \tau _n< t \right) \leqslant \mathbf {P}_{3} \left( \tau _0< t \right) + \mathbf {P}_{n-3} \left( \tau _n< t \right) \\= & {} 2 \cdot \mathbf {P}_{3} \left( \tau _0 < t \right) , \end{aligned}$$

where the equality follows from the symmetry of the Ehrenfest urn Markov chain.

Let us now consider a path $\mathcal {P}$ of length t starting at state 3 and ending at state 0. Observe that any such path must contain the sub-path going from state 3 to state 0 whose probability is $6/n^3$. Moreover, for all the other $t-3$ moves we have that if the chain crosses an edge $(i,i+1)$ from left to right then it must cross the same edge from right to left (and vice-versa). The probability for any such pair of moves is

$$\begin{aligned} \frac{n-i}{n} \cdot \frac{i+1}{n} \leqslant \frac{e^{2/n}}{4}, \end{aligned}$$

for every i. Hence, for any path $\mathcal {P}$ of length t going from 3 to 0, the probability that the chain follows exactly path $\mathcal {P}$ is³

$$\begin{aligned} \mathbf {P}_{3} \left( (X_1, \dots , X_t) = \mathcal {P} \right) \leqslant \frac{6}{n^3}\cdot \left( \frac{e^{2/n}}{4}\right) ^{(t-3)/2} = \frac{6}{n^3} \cdot \frac{2^3}{e^{3/n}} \cdot \left( \frac{e^{1/n}}{2}\right) ^t \leqslant \frac{48}{n^3} \cdot \left( \frac{e^{1/n}}{2}\right) ^t. \end{aligned}$$

Let $\ell$ and r be the number of left and right moves respectively in path $\mathcal {P}$ then $\ell + r = t$ and $\ell - r = 3$. Hence the total number of paths of length t going from 3 to 0 is less than

$$\begin{aligned} \left( {\begin{array}{c}t\\ \ell \end{array}}\right) = \left( {\begin{array}{c}t\\ \frac{t - 3}{2}\end{array}}\right) \leqslant 2^t. \end{aligned}$$

Thus, the probability that starting from 3 the chain hits 0 for the first time exactly at time t is

$$\begin{aligned} \mathbf {P}_{3} \left( \tau _0 = t \right) \leqslant \left( {\begin{array}{c}t\\ \frac{t-3}{2}\end{array}}\right) \frac{48}{n^3} \cdot \left( \frac{e^{1/n}}{2}\right) ^t \leqslant \frac{48}{n^3} e^{t/n}. \end{aligned}$$

Finally, the probability that the hitting time of 0 is less than t is

$$\begin{aligned} \mathbf {P}_{3} \left( \tau _0 < t \right)&\leqslant \sum _{i = 3}^{t-1} \mathbf {P}_{3} \left( \tau _0 = i \right) \\&\leqslant \frac{48}{n^3} \sum _{i = 3}^{t-1} e^{i/n} = \frac{48}{n^3} \cdot \frac{e^{t/n} - 1}{e^{1/n} - 1} \leqslant \frac{48 e^c}{n}. \end{aligned}$$

In the last inequality we used that $e^{1/n} - 1 \ge 1/n$ and $t = n \log n + c n$. $\square$

In the proof of Lemma 4.2 we deal with the lazy version of the Ehrenfest urn. The next lemma, which is folklore, allows us to use the bound we achieved in Lemma C.1 for the non-lazy chain.

Lemma C.2

Let $\{X_t\}$ be an irreducible Markov chain with finite state space $\Omega$ and transition matrix P and let $\{\hat{X}_t\}$ be its lazy version, i.e. the Markov chain with the same state space and transition matrix $\hat{P} = \frac{P + I}{2}$ where I is the $\Omega \times \Omega$ identity matrix. Let $\tau _A$ and $\hat{\tau }_A$ be the hitting time of the subset of states $A \subseteq \Omega$ in chains $\{X_t\}$ and $\{\hat{X}_t\}$ respectively. Then, for every starting state $b \in \Omega$ and for every time $t \in \mathbb {N}$ it holds that

$$\begin{aligned} \mathbf {P}_{b} \left( \hat{\tau }_A \leqslant t \right) \leqslant \mathbf {P}_{b} \left( \tau _A \leqslant t \right) . \end{aligned}$$

Appendix D: Proofs from Sect. 5.2

Throughout this section we denote the players with $0, 1, \ldots , n-1$ so that the neighbors of i are $i\pm 1\bmod n$.

1.1 Hitting ${\mathbf p}$ Starting from R when $\Delta > \delta$

In this section we prove Lemma 5.7 that provides an upper bound on the hitting time of state ${\mathbf p}$ when starting from a state ${\mathbf {x}}\in R$.

Lemma 5.7. For all ${\mathbf {x}}\in R$ and $\varepsilon >0$, if $\Delta > \delta$, $\beta =\omega (\log n)$ and n is sufficiently large

$$\begin{aligned} P_{{\mathbf {x}}}\left( \tau _{\mathbf p}>\frac{8-\varepsilon }{\varepsilon }\cdot n^2\right) \leqslant \frac{\varepsilon }{4}. \end{aligned}$$

Remind that states in R have at least two neighbors both playing $+1$ and without loss of generality we assume that $x_0=x_1=+1$. Intuitively, for $\beta =\omega (\log n)$, each of player 0 and 1 changes her strategy with very low probability. Moreover, player 2, when selected for update, updates her strategy to $+1$ with high probability. Similarly, after player 2 has played $+1$, we have that each of player 0, 1 and 2 changes her strategy with very low probability and player 3, when selected for update, plays $+1$ with high probability. This process repeats until every player is playing $+1$. In the following, we estimate the number of steps sufficient to have all players playing strategy $+1$ with high probability.

For sake of compactness, we will denote the strategy of player i at time step t by $X_t^i$. We start with a simple observation that lower bounds the probability that a player picks strategy $+1$ when selected for update, given that at least one of their neighbors is playing $+1$.

Observation D.1

If player i is selected for update at time t then, for $b\in \{-1,+1\}$

$$\begin{aligned} \mathbf {P}_{} \left( X_t^i = +1 \mid X_{t-1}^{i+b} = +1 \right) \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) . \end{aligned}$$

We start by lower bounding the probability that players $2, \ldots , n-1$ are selected for update at least once in this order before a given number t of steps. Set $\rho _1 = 0$ and, for $i=2,\dots ,n-1$, let $\rho _i$ be the first time player i is selected for update after time step $\rho _{i-1}$. Thus, at time $\rho _i$ player i is selected for update and players $2,\dots , i - 1$ have already been selected at least once in this order. In particular, $\rho _{n-1}$ is the first time step at which every player i, $i \ge 2$, has been selected at least once after his left neighbor. Obviously, $\rho _i > \rho _{i-1}$ for $i=2,\ldots ,n-1$. The next lemma lower bounds the probability that $\rho _{n-1} \leqslant t$.

Lemma D.2

For every ${\mathbf {x}}\in R$ and every $t>0$, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \rho _{n-1} \leqslant t \right) \ge 1 - \frac{n^2}{t}. \end{aligned}$$

Proof

Every player i has probability $\frac{1}{n}$ of being selected at any given time step. Therefore, $\mathbf {E}_{} \left[ \rho _2 \right] = \mathbf {E}_{} \left[ \rho _2 - \rho _1 \right] = n$ and $\mathbf {E}_{} \left[ \rho _i - \rho _{i - 1} \right] = n$, for $i=3, \dots , n-1$. Thus, by linearity of expectation,

$$\begin{aligned} \mathbf {E}_{} \left[ \rho _{n-1} \right] = \sum _{i=2}^{n-1} \mathbf {E}_{} \left[ \rho _i - \rho _{i - 1} \right] \leqslant n^2. \end{aligned}$$

The lemma follows from the Markov inequality. $\square$

The next lemma gives a lower bound to the probability that $X_t^i=+1$ for all players i and for $t\ge \rho _{n-1}$.

Lemma D.3

For every ${\mathbf {x}}\in R$, for every player i and for every $t > 0$, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( X_t^i = +1 \mid \rho _{n - 1} \leqslant t \right) \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^{t}. \end{aligned}$$

We prove the lemma first for $i\ge 2$ and then we will deal with players 0 and 1. Fix player $i\ge 2$, time step t and set $s_{i+1}=t$. Starting from time step t and going backward to time step 0, we identify the sequence of time steps $s_i>s_{i-1}>\ldots> s_2 >0$ such that, for $j=i,i-1,\ldots ,2$, $s_j$ is the last time player j has been selected before time $s_{j+1}$. We remark that, since $t \ge \rho _{n-1} > \rho _i$ we have that players $2, \dots , i$ are selected at least once in this order and thus all the $s_j$ are well defined. Strictly speaking, the sequence $s_i,\ldots ,s_2$ depends on i and t and thus a more precise, and more cumbersome, notation would have been $s_{i,j,t}$. Since player i and time step t will be clear from the context, we drop i and t.

In order to lower bound the probability that $X_t^i = +1$ for $i \ge 2$, we first bound it in terms of the probability that player 2 plays $+1$ at time $s_2$ and then we evaluate this last quantity. The next lemma is the first step of our proof.

Lemma D.4

For every ${\mathbf {x}}\in R$, every player $2\leqslant i \leqslant n-1$ and every $t \ge \rho _{n-1}$, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( X_{t}^i = +1 \mid \rho _{n-1} \leqslant t \right) \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^{i-2} \mathbf {P}_{{\mathbf {x}}} \left( X_{s_2}^2 = +1 \mid \rho _{n-1} \leqslant t \right) . \end{aligned}$$

Proof

For every i, $s_i$ is the last time the player i is selected for update before t and thus $X_t^i = X_{s_i}^i$. Hence, for $i=2$ the lemma obviously holds. For $i>2$ and $j=3, \ldots , i$, we observe that, since $t \ge \rho _i$, $s_{j-1}$ is the last time that player $j-1$ has been selected for update before time $s_{j}$ and thus $X_{s_j - 1}^{j-1} = X_{s_{j-1}}^{j-1}$. Then, from Observation D.1, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( X_{s_j}^j = +1 \mid \rho _{n - 1} \leqslant t \right)&\ge \mathbf {P}_{{\mathbf {x}}} \left( X_{s_j}^j = +1 \mid X_{s_j-1}^{j-1} = +1, \rho _{n - 1} \leqslant t \right) \\&\quad \,\,\cdot \mathbf {P}_{{\mathbf {x}}} \left( X_{s_j-1}^{j-1} = +1 \mid \rho _{n - 1} \leqslant t \right) \\&\ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) \mathbf {P}_{{\mathbf {x}}} \left( X_{s_{j-1}}^{j-1} = +1 \mid \rho _{n - 1} \leqslant t \right) . \end{aligned}$$

We obtain the lemma by iteratively applying the same argument to $\mathbf {P}_{{\mathbf {x}}} \left( X_{s_{j-1}}^{j-1} = +1 \mid \rho _{n - 1} \leqslant t \right)$, for $j = i-1, i-2, \ldots , 3$. $\square$

We now bound the probability that player 2 plays $+1$ at time step $s_2$. If player 1 has not been selected for update before time $s_2$, then $X_{s_2-1}^1 = X_0^1 = +1$, and, from Observation D.1, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( X_{s_2}^2 = +1 \mid \rho _{n - 1} \leqslant t \right)\ge & {} \mathbf {P}_{{\mathbf {x}}} \left( X_{s_2}^2 = +1 \mid X_{s_2-1}^1 = +1, \rho _{n - 1} \leqslant t \right) \\\ge & {} \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) . \end{aligned}$$

It remains to consider the case when player 1 has been selected for update at least once before time $s_2$. For any fixed time step t, we define a new sequence of time steps $r_0>r_1,\ldots >0$ in the following way. We set $r_0=s_2$ and let $r_j$, for $j>0$, be the last time player $j\mod 2$ has been selected before time $r_{j-1}$. For the last element in the sequence, $r_k$, it holds that player $(k+1)\mod 2$ is not selected before time step $r_k$.

Since at time $s_2$ player 2 has been selected for update and since $r_1$ is the last time step player 1 has been selected for update before $r_0 = s_2$, we have $X_{s_2-1}^1 = X_{r_1}^1$ and, by Observation D.1,

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( X_{s_2}^2 = +1 \mid \rho _{n - 1} \leqslant t \right)\ge & {} \mathbf {P}_{{\mathbf {x}}} \left( X_{s_2}^2 = +1 \mid X_{s_2-1}^1 = +1, \rho _{n - 1} \leqslant t \right) \nonumber \\&\cdot \mathbf {P}_{{\mathbf {x}}} \left( X_{s_2-1}^1 = +1 \mid \rho _{n - 1} \leqslant t \right) \nonumber \\\ge & {} \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) \cdot \mathbf {P}_{{\mathbf {x}}} \left( X_{r_1}^1 = +1 \mid \rho _{n - 1} \leqslant t \right) .\nonumber \\ \end{aligned}$$

(14)

Finally, we bound $\mathbf {P}_{{\mathbf {x}}} \left( X_{r_1}^1 = +1 \mid \rho _{n - 1} \leqslant t \right)$.

Lemma D.5

For every ${\mathbf {x}}\in R$, time step t, and player i, let $r_0, \ldots , r_k$ be defined as above. If $k>0$, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( X_{r_1}^{1} = +1 \mid \rho _{n-1} \leqslant t \right) \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^{k}. \end{aligned}$$

Proof

For sake of compactness, in this proof we denote the parity of integer a with $\mathcal {P}(a) = a \mod 2$. Thus, the definition of sequence $r_j$ gives that player $\mathcal {P}(j)$ has been selected for update at time $r_j$ and

$$\begin{aligned}&\mathbf {P}_{{\mathbf {x}}} \left( X_{r_j}^{\mathcal {P}(j)} = +1 \mid \rho _{n - 1} \leqslant t \right) \\&\quad \ge \mathbf {P}_{{\mathbf {x}}} \left( X_{r_j}^{\mathcal {P}(j)} = +1 \mid X_{r_j-1}^{\mathcal {P}(j+1)} = +1, \rho _{n - 1} \leqslant t \right) \mathbf {P}_{{\mathbf {x}}} \left( X_{r_j-1}^{\mathcal {P}(j+1)} = +1 \mid \rho _{n - 1} \leqslant t \right) . \end{aligned}$$

If $j \ne k$ player $\mathcal {P}(j+1)$ has not been selected for update between time $r_{j + 1} \leqslant r_j-1$ and time $r_j$ and, by Observation D.1, we obtain

$$\begin{aligned}&\mathbf {P}_{{\mathbf {x}}} \left( X_{r_j}^{\mathcal {P}(j)} = +1 \mid \rho _{n - 1} \leqslant t \right) \\&\quad \ge \mathbf {P}_{{\mathbf {x}}} \left( X_{r_j}^{\mathcal {P}(j)} = +1 \mid X_{r_j-1}^{\mathcal {P}(j+1)} = +1, \rho _{n - 1} \leqslant t \right) \mathbf {P}_{{\mathbf {x}}} \left( X_{r_j-1}^{\mathcal {P}(j+1)} = +1 \mid \rho _{n - 1} \leqslant t \right) \\&\quad \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) \mathbf {P}_{{\mathbf {x}}} \left( X_{r_{j+1}}^{\mathcal {P}(j+1)} = +1 \mid \rho _{n - 1} \leqslant t \right) . \end{aligned}$$

If $j = k$, instead, player $\mathcal {P}(k+1)$ has not been selected for update before time $r_k$ and thus $X_{r_k - 1}^{\mathcal {P}(k+1)} = X_0^{\mathcal {P}(k+1)} = +1$. By Observation D.1, we have

$$\begin{aligned}&\mathbf {P}_{{\mathbf {x}}} \left( X_{r_k}^{\mathcal {P}(k)} = +1 \mid \rho _{n - 1} \leqslant t \right) \\&\quad \ge \mathbf {P}_{{\mathbf {x}}} \left( X_{r_k}^{\mathcal {P}(k)} = +1 \mid X_{r_k-1}^{\mathcal {P}(k+1)} = +1, \rho _{n - 1} \leqslant t \right) \mathbf {P}_{{\mathbf {x}}} \left( X_{r_k-1}^{\mathcal {P}(k+1)} = +1 \mid \rho _{n - 1} \leqslant t \right) \\&\quad \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) . \end{aligned}$$

$\square$

Proof

(of Lemma D.3) For every player $i \ge 2$ and $t>0$, we have

$$\begin{aligned}&\mathbf {P}_{{\mathbf {x}}} \left( X_t^i = +1 \mid \rho _{n - 1} \leqslant t \right) \\&\quad \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^{i - 2} \mathbf {P}_{{\mathbf {x}}} \left( X_{s_2}^2 = +1 \mid \rho _{n-1} \leqslant t \right)&(\text {from Lemma}~\hbox {D.4})\\&\quad \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^{i - 1} \mathbf {P}_{{\mathbf {x}}} \left( X_{r_1}^1 = +1 \mid \rho _{n-1} \leqslant t \right)&(\text {from Equation}~14)\\&\quad \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^{i - 1 + k}&(\text {from Lemma}~\hbox {D.5})\\&\quad \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^t, \end{aligned}$$

where k is the index of the last term in the sequence $r_0, r_1, \ldots$ previously defined and where the last inequality follows from $i - 1 + k \leqslant t$, since the sequence of updates we are considering cannot be longer than t.

The lemma for players 0 and 1 can be proved in a similar way. Clearly, if player $i=0,1$ has never been selected for update before time t, we have that $X_t^i = +1$ with probability 1. If player i has been selected at least once we have to distinguish the cases $i=0$ and $i=1$. If $i=1$, we define $r_0 = t+1$ and we identify a sequence of time step $r_1> r_2> \ldots > 0$ as above: we have that $X_t^1 = X_{r_1}^1$ and from Lemma D.5 follows that

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( X_t^i = +1 \mid \rho _{n - 1} \leqslant t \right) \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^k \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^t, \end{aligned}$$

where k is the last index of the sequence $r_1, r_2, \dots$. Finally, the probability that player 0 plays the strategy $+1$ at time t, given that she was selected for update at least once, can be handled similarly to the probability that player 2 plays the strategy $+1$ at time $s_2$. $\square$

The following lemma gives the probability that the hitting time of the profile ${\mathbf p}$ is less or equal to t, given that $\rho _{n-1} \leqslant t$.

Lemma D.6

For every ${\mathbf {x}}\in R$ and every $t > 0$, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{{\mathbf p}} \leqslant t \mid \rho _{n - 1} \leqslant t \right) \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^{nt}. \end{aligned}$$

Proof

Let f be the permutation that sorts the players according to the order of last selection for update; i.e., f(0) is the last player that is selected for update, f(1) is the next to last one, and so on. We have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{{\mathbf p}} \leqslant t \mid \rho _{n - 1} \leqslant t \right)&\ge \mathbf {P}_{{\mathbf {x}}} \left( \bigwedge _{j=0}^{n-1} X_t^{f(j)} = +1 \mid \rho _{n - 1} \leqslant t \right) \\&= \prod _{j=0}^{n - 1} \mathbf {P}_{{\mathbf {x}}} \left( X_t^{f(j)} = +1 \mid \bigwedge _{i=j+1}^{n-1} X_t^{f(i)} = +1, \rho _{n - 1} \leqslant t \right) \\&\ge \prod _{j=0}^{n - 1} \mathbf {P}_{{\mathbf {x}}} \left( X_t^{f(j)} = +1 \mid \rho _{n - 1} \leqslant t \right) \\&\ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^{nt}, \end{aligned}$$

where for the second inequality we used that the probability of selecting $+1$ can only increase when there are other players playing this strategy, whereas the last inequality follows from Lemma D.3. $\square$

Now we are ready to prove Lemma 5.7.

Proof

(of Lemma 5.7) From Lemmas D.2 and  D.6, we have that for every ${\mathbf {x}}\in R$ and every $t > 0$

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{\mathbf p}\leqslant t \right)&\ge \mathbf {P}_{{\mathbf {x}}} \left( \rho _{n-1} \leqslant t \right) \cdot \mathbf {P}_{{\mathbf {x}}} \left( \tau _{{\mathbf p}} \leqslant t \mid \rho _{n - 1} \leqslant t \right) \\&\ge \left( 1 - \frac{n^2}{t}\right) \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) ^{nt}. \end{aligned}$$

For sufficiently large n we have that, since $\beta =\omega (\log n)$,

$$\begin{aligned} e^{(\Delta - \delta )\beta } \ge \frac{8-\varepsilon }{\varepsilon } \cdot \frac{n^3}{\log \frac{8}{8-\varepsilon }}. \end{aligned}$$

By setting $t=\frac{8-\varepsilon }{\varepsilon }\cdot n^2$, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{\mathbf p}\leqslant t \right)&\ge \left( 1 - \frac{\varepsilon }{8 - \varepsilon }\right) \left( 1 - \frac{1}{1 + \frac{8-\varepsilon }{\varepsilon }\cdot \frac{n^3}{\log \frac{8}{8-\varepsilon }}}\right) ^{\frac{8-\varepsilon }{\varepsilon }\cdot n^3}\\&\ge \frac{8\left( 1 - \frac{\varepsilon }{4}\right) }{8 - \varepsilon } \frac{8 - \varepsilon }{8} = 1 - \frac{\varepsilon }{4}, \end{aligned}$$

where the second inequality follows from the well known approximation $1 - a \ge e^{-\frac{a}{1 - a}}$. $\square$

1.2 D.2 Hitting ${\mathbf p}$ Starting from $S_d$ when $\Delta > \delta$

In this section we prove Lemma 5.8 that provides an upper bound on the hitting time of state ${\mathbf p}$ when starting from a state ${\mathbf {x}}\in S_d$.

Lemma 5.8. For all $d>0$, ${\mathbf {x}}\in S_d$ and $\varepsilon >0$, if $\Delta > \delta$, $\beta =\omega (\log n)$ and n is sufficiently large

$$\begin{aligned} P_{{\mathbf {x}}}\left( \tau _{\mathbf p}>\frac{8}{\varepsilon }\cdot n^2\right) \leqslant \frac{1}{2^d + 1}+\frac{\varepsilon }{4}. \end{aligned}$$

In fact, we show that from any ${\mathbf {x}}\in S_d$ the dynamics hit after a polynomial number of steps a profile in R with high probability. Then, the aimed result easily follows by Lemma 5.7.

Specifically, let us denote by $\mathsf{negl}\left( n\right)$ a function in n that is smaller than the inverse of every polynomial in n. Then the following upper bound on the hitting time of R from the set $S_d$ holds.

Lemma D.7

For every $d>0$ and ${\mathbf {x}}\in S_d$, if $\Delta > \delta$, $\beta = \omega \left( {\log n}\right)$ and n is sufficiently large

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _R \leqslant n^2 \right) \ge \frac{2^d}{2^d+1} \left( 1 - \mathsf{negl}\left( n\right) \right) . \end{aligned}$$

Before proving this lemma, we show how Lemma 5.8 easily follows from Lemmas D.7 and  5.7.

Proof

(of Lemma 5.8) By Lemmas 5.7 and D.7, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{{\mathbf p}} \leqslant \frac{8n^2}{\varepsilon } \right)&\ge \mathbf {P}_{{\mathbf {x}}} \left( \tau _{{\mathbf p}} \leqslant \frac{8n^2}{\varepsilon } \mid \tau _R \leqslant n^2 \right) \mathbf {P}_{{\mathbf {x}}} \left( \tau _R \leqslant n^2 \right) \\&\ge \mathbf {P}_{X_{\tau _R}} \left( \tau _{{\mathbf p}} \leqslant \frac{(8 - \varepsilon )n^2}{\varepsilon } \right) \mathbf {P}_{{\mathbf {x}}} \left( \tau _R \leqslant n^2 \right) \\&\ge \left( 1 - \frac{\varepsilon }{4}\right) \frac{2^d}{2^d + 1} \left( 1 - \mathsf{negl}\left( n\right) \right) \ge \frac{2^d}{2^d+1} - \frac{\varepsilon }{4}. \end{aligned}$$

$\square$

We now prove Lemma D.7. Specifically we will bound $\mathbf {P}_{{\mathbf {x}}} \left( \tau _{R} \leqslant t \wedge \tau _R \leqslant \tau _{\mathbf m} \right)$ and will use the fact that

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _R \leqslant t \right) \ge \mathbf {P}_{{\mathbf {x}}} \left( \tau _{R} \leqslant t \wedge \tau _R \leqslant \tau _{\mathbf m} \right) . \end{aligned}$$

(15)

Let $\theta ^\star$ be the first time at which all players have been selected at least once. We define players playing $+1$ in profile ${\mathbf {x}}$ as the plus-players of ${\mathbf {x}}$ and their neighbors as border-players. Consider the event E that “a border-player is selected for update before at least one of her neighboring plus-players” and denote by $\overline{E}$ its complement. Observe that

$$\begin{aligned}&\mathbf {P}_{{\mathbf {x}}} \left( \tau _R \leqslant t \wedge \tau _R \leqslant \tau _{\mathbf m} \right) \nonumber \\&\quad \ge \mathbf {P}_{{\mathbf {x}}} \left( \tau _R \leqslant t \wedge \tau _R \leqslant \tau _{\mathbf m}\mid E \wedge \theta ^\star \leqslant t \right) \mathbf {P}_{{\mathbf {x}}} \left( E \wedge \theta ^\star \leqslant t \right) \nonumber \\&\quad = \mathbf {P}_{{\mathbf {x}}} \left( \tau _R \leqslant t \wedge \tau _R \leqslant \tau _{\mathbf m}\mid E \wedge \theta ^\star \leqslant t \right) \mathbf {P}_{{\mathbf {x}}} \left( E \mid \theta ^\star \leqslant t \right) \mathbf {P}_{{\mathbf {x}}} \left( \theta ^\star \leqslant t \right) . \end{aligned}$$

(16)

We next bound the three components appearing in the last line of (16). A bound on the probability that $\theta ^\star \leqslant t$ directly follows from the coupon collector argument; we include a proof of this bound for completeness.

Lemma D.8

For every $t > 0$,

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \theta ^\star \leqslant t \right) \ge 1 - n e^{-t/n}. \end{aligned}$$

Proof

The logit dynamics at each time step select a player for update uniformly and independently of the previous selections. Thus, the probability that i players are never selected for update in t steps is $\left( 1 - \frac{i}{n}\right) ^t$ and

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \theta ^\star > t \right) \leqslant \sum _{i = 1}^{n - 1} \left( 1 - \frac{i}{n}\right) ^t \leqslant \sum _{i = 1}^{n - 1} e^{-\frac{it}{n}} \leqslant n e^{-t/n}. \end{aligned}$$

$\square$

Next we bound the probability of the event E given that all players have been selected at least once.

Lemma D.9

For every ${\mathbf {x}}\in S_d$, $d > 0$ and every $t > 0$

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( E \mid \theta ^\star \leqslant t \right) \ge \frac{2^d}{2^d+1}. \end{aligned}$$

Proof

Let us define $\ell ({\mathbf {x}})$ as the number of border-players in ${\mathbf {x}}$. Note that, since there are no adjacent plus-players in ${\mathbf {x}}$, $\ell ({\mathbf {x}}) \ge d$.

The proof proceeds by induction on d. Let $d=1$ and denote by i the plus-player. Since we are conditioning on $\theta ^\star \leqslant t$, all players are selected at least once by time t and thus the probability that one of the two neighbors of i is selected for update before i is selected is $\frac{2}{3} = \frac{2^d}{2^d+1}$.

Suppose now that the claim holds for $d-1$ and consider ${\mathbf {x}}\in S_d$. Denote by $T_{\mathbf {x}}$ the set of all the plus-players in ${\mathbf {x}}$ and their border-players and let i be the first player in $T_{\mathbf {x}}$ to be selected for update (notice that i is well defined since $\theta ^\star \leqslant t$). Observe that, if i is a border-player, then the event E occur and this happens with probability $\frac{l({\mathbf {x}})}{l({\mathbf {x}})+d}$. If i is a plus-player, we consider the subset $\overline{T}_{\mathbf {x}}\subset T_{\mathbf {x}}$ of the remaining $d-1$ plus-players and their border-players. The event E will occur if and only if at least one border-player in $\overline{T}_{\mathbf {x}}$ is selected before one of its neighboring plus-players. Notice though that $\overline{T}_{\mathbf {x}}= T_{\mathbf y}$, for ${\mathbf y}\in S_{d-1}\setminus R$ such that $y_i = -1$ and ${\mathbf y}_{-i} = {\mathbf {x}}_{-i}$. Thus, by inductive hypothesis, $\mathbf {P}_{{\mathbf y}} \left( E \mid \theta ^\star \leqslant t \right) \ge \frac{2^{d-1}}{2^{d-1}+1} = \frac{2^{d}}{2^{d}+2}$. Thus,

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( E \mid \theta ^\star \leqslant t \right)&= \frac{l({\mathbf {x}})}{l({\mathbf {x}})+d}+\frac{d}{l({\mathbf {x}})+d} \cdot \mathbf {P}_{{\mathbf y}} \left( E \mid \theta ^\star \leqslant t \right) \\&\ge \frac{l({\mathbf {x}})}{l({\mathbf {x}})+d}+\frac{d}{l({\mathbf {x}})+d} \cdot \frac{2^d}{2^d+2}\\&= 1 - \frac{2d}{(l({\mathbf {x}})+d)(2^d + 2)}\\&\ge 1 - \frac{1}{2^d + 2}, \end{aligned}$$

where the last inequality follows from $l({\mathbf {x}})\ge d$. The claim follows since $1 - \frac{1}{2^d + 2} \ge \frac{2^d}{2^d+1}$. $\square$

Finally suppose that the event E occurs, i.e., there is a border-player i selected for update before than at least one of the neighboring plus-players, and $\theta ^\star \leqslant t$. Let $\tau$ be the time step at which E occurs. Note that, since all players have been selected at least once before t, then $\tau < t$. Moreover, at time $\tau$, the player i has at least one neighbor playing strategy $+1$, and thus she will play this strategy with probability at least $\left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right)$. Then we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _R \leqslant t \wedge \tau _R \leqslant \tau _{\mathbf m}\mid E \wedge \theta ^\star \leqslant t \right)\ge & {} \mathbf {P}_{{\mathbf {x}}} \left( X_{\tau }^i = +1 \mid E \wedge \theta ^\star \leqslant t \right) \nonumber \\\ge & {} \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) , \end{aligned}$$

(17)

where $X_{\tau }^i$ denotes the strategy of i in the profile $X_\tau$.

Now we are ready to prove Lemma D.7.

Proof

(of Lemma D.7) From (15), (16), (17), Lemmas D.8 and  D.9 we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _R \leqslant n^2 \right) \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) \cdot \frac{2d-1}{2d} \cdot \left( 1 - n e^{-n}\right) = \frac{2d-1}{2d} \cdot (1 - \mathsf{negl}\left( n\right) ), \end{aligned}$$

where the last equality holds for n sufficiently large since $\beta = \omega (\log n)$. $\square$

1.3 D.3 Hitting $\{{\mathbf p}, {\mathbf m}\}$ Starting from $S_d$ when $\Delta > \delta$

In this section we prove Lemma 5.9 that provides an upper bound on the hitting time of the set $\{{\mathbf p},{\mathbf m}\}$ when starting from a state ${\mathbf {x}}\in S_d$.

Lemma 5.9. For all ${\mathbf {x}}\in \{\pm 1\}^n$ and $\varepsilon >0$, if $\Delta > \delta$, $\beta =\omega (\log n)$ and n is sufficiently large

$$\begin{aligned} P_{{\mathbf {x}}}\left( \tau _{\{{\mathbf p},{\mathbf m}\}}>\frac{8}{\varepsilon }\cdot n^2\right) \leqslant \frac{\varepsilon }{2}. \end{aligned}$$

The proof is similar to the proof of Lemma 5.8, except that now we need a bound on the probability to hit R or ${\mathbf m}$ in polynomial time when starting from a state in $S_d$. Next lemma gives us such a bound.

Lemma D.10

For all ${\mathbf {x}}\in \{\pm 1\}^n$, if $\Delta > \delta$, $\beta = \omega \left( {\log n}\right)$ and n is sufficiently large

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{R \cup \{{\mathbf m}\}} \leqslant n^2 \right) \ge 1 - \mathsf{negl}\left( n\right) . \end{aligned}$$

Lemma 5.9 then immediately follows.

Proof

(of Lemma 5.9) By Lemmas 5.7 and  D.10, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{\{{\mathbf p},{\mathbf m}\}} \leqslant \frac{8n^2}{\varepsilon } \right)\ge & {} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{\{{\mathbf p},{\mathbf m}\}} \leqslant \frac{8n^2}{\varepsilon } \mid \tau _{R \cup \{{\mathbf m}\}} \leqslant n^2 \right) \mathbf {P}_{{\mathbf {x}}} \left( \tau _{R \cup \{{\mathbf m}\}} \leqslant n^2 \right) \\\ge & {} \mathbf {P}_{X_{\tau _{R \cup \{{\mathbf m}\}}}} \left( \tau _{\{{\mathbf p},{\mathbf m}\}} \leqslant \frac{(8 - \varepsilon )n^2}{\varepsilon } \right) \mathbf {P}_{{\mathbf {x}}} \left( \tau _{R \cup \{{\mathbf m}\}} \leqslant n^2 \right) \\\ge & {} \left( 1 - \frac{\varepsilon }{4}\right) \left( 1 - \mathsf{negl}\left( n\right) \right) \ge 1 - \frac{\varepsilon }{2}. \end{aligned}$$

$\square$

Let us now prove Lemma D.10. First of all, we note that

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{R \cup \{{\mathbf m}\}} \leqslant t \right) = \mathbf {P}_{{\mathbf {x}}} \left( \tau _{R} \leqslant t \wedge \tau _R \leqslant \tau _{\mathbf m} \right) + \mathbf {P}_{{\mathbf {x}}} \left( \tau _{{\mathbf m}} \leqslant t \wedge \tau _{\mathbf m}\leqslant \tau _R \right) . \end{aligned}$$

(18)

A lower bound on the first term on the right hand side of the above equation was given in previous section. In fact, from (16), (17) and Lemma D.8, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{R} \leqslant t \wedge \tau _R \leqslant \tau _{\mathbf m} \right) \ge \left( 1 - \frac{1}{1 + e^{(\Delta - \delta )\beta }}\right) \cdot \left( 1 - n e^{-t/n}\right) \cdot \mathbf {P}_{{\mathbf {x}}} \left( E \mid \theta ^\star \leqslant t \right) . \end{aligned}$$

(19)

Similarly,

$$\begin{aligned}&\mathbf {P}_{{\mathbf {x}}} \left( \tau _{\mathbf m}\leqslant t \wedge \tau _{\mathbf m}\leqslant \tau _R \right) \nonumber \\&\quad \ge \mathbf {P}_{{\mathbf {x}}} \left( \tau _{\mathbf m}\leqslant t \wedge \tau _{\mathbf m}\leqslant \tau _R \mid \overline{E} \wedge \theta ^\star \leqslant t \right) \mathbf {P}_{{\mathbf {x}}} \left( \overline{E} \wedge \theta ^\star \leqslant t \right) \nonumber \\&\quad = \mathbf {P}_{{\mathbf {x}}} \left( \tau _{\mathbf m}\leqslant t \wedge \tau _{\mathbf m}\leqslant \tau _R \mid \overline{E} \wedge \theta ^\star \leqslant t \right) \mathbf {P}_{{\mathbf {x}}} \left( \overline{E} \mid \theta ^\star \leqslant t \right) \mathbf {P}_{{\mathbf {x}}} \left( \theta ^\star \leqslant t \right) \nonumber \\&\quad \ge \left( 1 - n e^{-t/n}\right) \cdot \mathbf {P}_{{\mathbf {x}}} \left( \tau _{\mathbf m}\leqslant t \wedge \tau _{\mathbf m}\leqslant \tau _R \mid \overline{E} \wedge \theta ^\star \leqslant t \right) \mathbf {P}_{{\mathbf {x}}} \left( \overline{E} \mid \theta ^\star \leqslant t \right) \qquad \quad \end{aligned}$$

(20)

where the last inequality follows from Lemma D.8.

Next lemma bounds $\mathbf {P}_{{\mathbf {x}}} \left( \tau _{\mathbf m}\leqslant t \wedge \tau _{\mathbf m}\leqslant \tau _R \mid \overline{E} \wedge \theta ^\star \leqslant t \right)$.

Lemma D.11

For every $d \ge 0$, for every ${\mathbf {x}}\in S_d$ and every $t > 0$, we have

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{\mathbf m}\leqslant t \wedge \tau _{\mathbf m}\leqslant \tau _R \mid \overline{E} \wedge \theta ^\star \leqslant t \right) \ge 1 - \frac{t}{e^{2\delta \beta }}. \end{aligned}$$

Proof

Suppose that the event E does not occur, i.e., all plus-players are selected before their neighboring border-players, and $\theta ^\star \leqslant t$. Let $\tau$ be the time step at which the last plus-player is selected. Note that, since all players have been selected at least once before t, then $\tau < t$. Thus, if in the first $\tau$ time steps the selected player adopts strategy $-1$, then $\tau _{{\mathbf m}} \leqslant \tau \leqslant t$ and $\tau _{\mathbf m}\leqslant \tau _R$, i.e., by denoting the player selected for update at time step j as i(j),

$$\begin{aligned}&\mathbf {P}_{{\mathbf {x}}} \left( \tau _{{\mathbf m}} \leqslant t \wedge \tau _{\mathbf m}\leqslant \tau _R \mid \overline{E} \wedge \theta ^\star \leqslant t \right) \ge \mathbf {P}_{{\mathbf {x}}} \left( \bigwedge _{j=1}^\tau X_j^{i(j)} = -1 \mid \overline{E} \wedge \theta ^\star \leqslant t \right) \\&\quad = \prod _{j=1}^{\tau } \mathbf {P}_{{\mathbf {x}}} \left( X_j^{i(j)} = -1 \mid \overline{E} \wedge \theta ^\star \leqslant t \wedge \bigwedge _{k=1}^j X_k^{i(k)} = -1 \right) . \end{aligned}$$

Given that no border-player is selected before the corresponding plus-player and that at each previous time step the selected player has adopted strategy $-1$, we have that, for every j, player i(j) has both neighbors playing $-1$. Hence, i(j) adopts strategy $-1$ with probability $\left( 1 - \frac{1}{1 + e^{2\delta \beta }}\right)$. Thus,

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{{\mathbf m}} \leqslant t \wedge \tau _{\mathbf m}\leqslant \tau _R \mid \overline{E} \wedge \theta ^\star \leqslant t \right)\ge & {} \left( 1 - \frac{1}{1 + e^{2\delta \beta }}\right) ^\tau \ge \left( 1 - \frac{1}{1 + e^{2\delta \beta }}\right) ^t \\\ge & {} \left( 1 - \frac{t}{e^{2\delta \beta }}\right) , \end{aligned}$$

where the last inequality follows from the approximations $1 - a \leqslant e^{-a}$ and $1 - a \ge e^{-\frac{a}{1 - a}}$ for $0 \leqslant a \leqslant 1$. $\square$

Now we are ready to prove Lemma D.10.

Proof

(of Lemma D.10) From (19), (20) and Lemma D.11, for n sufficiently large and $\beta = \omega (\log n)$ we have both

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{R} \leqslant n^2 \wedge \tau _R \leqslant \tau _{\mathbf m} \right) \ge \left( 1 - \mathsf{negl}\left( n\right) \right) \cdot \mathbf {P}_{{\mathbf {x}}} \left( E \mid \theta ^\star \leqslant n^2 \right) . \end{aligned}$$

and

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{\mathbf m}\leqslant n^2 \wedge \tau _{\mathbf m}\leqslant \tau _R \right) \ge \left( 1 - \mathsf{negl}\left( n\right) \right) \cdot \mathbf {P}_{{\mathbf {x}}} \left( \overline{E} \mid \theta ^\star \leqslant n^2 \right) . \end{aligned}$$

Then, from (18) it follows that

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{R \cup \{{\mathbf m}\}} \leqslant n^2 \right)\ge & {} \left( 1 - \mathsf{negl}\left( n\right) \right) \left( \mathbf {P}_{{\mathbf {x}}} \left( E \mid \theta ^\star \leqslant n^2 \right) + \mathbf {P}_{{\mathbf {x}}} \left( \overline{E} \mid \theta ^\star \leqslant n^2 \right) \right) \\= & {} 1 - \mathsf{negl}\left( n\right) . \end{aligned}$$

$\square$

1.4 D.4 Hitting $\{{\mathbf p}, {\mathbf m}\}$ when $\Delta = \delta$

In this section we prove Lemma 5.10 that provides an upper bound on the hitting time of the set $\{{\mathbf p},{\mathbf m}\}$ when $\Delta = \delta$.

Lemma 5.10. For all ${\mathbf {x}}\in \{\pm 1\}^n$ and $\varepsilon >0$, if $\Delta = \delta$, $\beta =\omega (\log n)$ and n is sufficiently large

$$\begin{aligned} \mathbf {P}_{{\mathbf {x}}} \left( \tau _{\{{\mathbf p},{\mathbf m}\}} > n^5 \right) \leqslant o(1). \end{aligned}$$

Let us start by introducing some useful notation. We say that the profile ${\mathbf {x}}$ has a plus-block of size l starting at player i if $x_i=x_{i+1}=\ldots =x_{i+l-1}=+1$ and $x_{i-1}=x_{i+l}=-1$ and players i and $i+l-1$ are the border players of the block. A similar definition is given for minus-blocks. Notice that every profile ${\mathbf {x}}\ne {\mathbf p},{\mathbf m}$ has the same number of plus-blocks and minus-blocks and this number is called the level of ${\mathbf {x}}$ and is denoted by $\ell ({\mathbf {x}})$. We set $\ell ({\mathbf p})=\ell ({\mathbf m})=0$. Moreover, for a profile ${\mathbf {x}}$, we defines $s_+({\mathbf {x}})$ as the number of plus-blocks of size 1, $s_-({\mathbf {x}})$ as the number of minus-blocks of size 1 and set $s({\mathbf {x}})=s_+({\mathbf {x}}) + s_-({\mathbf {x}})$.

We would like to study how long it takes to the logit dynamics to hit ${\mathbf p}$ or ${\mathbf m}$. Let ${\mathbf {x}}$ be the starting profile of the logit dynamics and assume that $\ell ({\mathbf {x}}) = \ell$. Since ${\mathbf p}$ and ${\mathbf m}$ have level 0, the dynamics have to go down $\ell$ levels before hitting one of these two target profiles. To reach a profile in a smaller level the dynamics have to reach a profile having a monochromatic block of size 1 is reached, select the unique player of this block for update and have this player changing her strategy to the same strategy as her neighbors. Let us denote with $\tau _i$ the hitting time of a profile at level i, i.e., $\tau _i = \tau _{\{{\mathbf {x}}:\ell ({\mathbf {x}}) = i\}}$. By linearity of expectation, we have that

$$\begin{aligned} \mathbf {E}_{{\mathbf {x}}} \left[ \tau _{\{{\mathbf p}, {\mathbf m}\}} \right] \leqslant \sum _{i=0}^{\ell -1} \max _{{\mathbf {x}}: \ell ({\mathbf {x}}) = i+1} \mathbf {E}_{{\mathbf {x}}} \left[ \tau _i \right] . \end{aligned}$$

(21)

Thus, to bound the hitting time of $\{{\mathbf p}, {\mathbf m}\}$ we have only to compute $\mathbf {E}_{{\mathbf {x}}} \left[ \tau _i \right]$ for any level $i = 0, 1, \ldots , \ell -1$ and for any profile ${\mathbf {x}}$ such that $\ell ({\mathbf {x}}) = i+1$.

Fix ${\mathbf {x}}$ be a profile of level $i+1$ and number arbitrarily its $2(i+1)$ monochromatic blocks. We denote by $k_j({\mathbf {x}})$ the size of the jth monochromatic block. For each $1 \leqslant j \leqslant 2(i+1)$, let us define the quantity $\theta _{j}$ as follows. Suppose that the dynamics starting from ${\mathbf {x}}$ hit for the first time a profile of level i after t steps and this happens because the jth monochromatic block disappears; then, we set $\theta _{j}=t$ and $\theta _{j'}=+\infty$ for all $j'\ne j$. Given that the starting profile ${\mathbf {x}}$ is at level $i+1$, we have $\tau _i = \min _j \theta _{j}$ and, thus,

$$\begin{aligned} \mathbf {E}_{{\mathbf {x}}} \left[ \tau _i \right] =\mathbf {E}_{{\mathbf {x}}} \left[ \min _j \theta _{j} \right] \leqslant \max _j \mathbf {E}_{{\mathbf {x}}} \left[ \theta _{j} \mid \theta _{j} < \theta _{j'} \text { for all } j'\ne j \right] . \end{aligned}$$

In order to have a more compact notation we define

$$\begin{aligned} \gamma _{i,l} = \max _{1 \leqslant j \leqslant 2(i + 1)} \max _{\begin{array}{c} {\mathbf {x}}:\ell ({\mathbf {x}})=i+1\\ k_j({\mathbf {x}})=l \end{array}} \mathbf {E}_{{\mathbf {x}}} \left[ \theta _{j} \mid \theta _{j} < \theta _{j'} \text {for all } j'\ne j \right] , \end{aligned}$$

and set $\gamma _i=\max _l\gamma _{i,l}$. Observe that

$$\begin{aligned} \mathbf {E}_{{\mathbf {x}}} \left[ \tau _i \right] \leqslant \gamma _i \end{aligned}$$

(22)

and it is not hard to see that $\gamma _{i,l}$ is non-decreasing with l. Next lemma gives a bound $\gamma _i$ in terms of $\gamma _{i+1}$. Using recursively this lemma we will be able to upper bound $\gamma _i$ for any $i \ge 0$.

Lemma D.12

For $0 \leqslant i < \lfloor n/2 \rfloor - 1$

$$\begin{aligned} \gamma _i \leqslant n^3 \left( 1 + \frac{1}{1+e^{2\Delta \beta }} \gamma _{i + 1}\right) . \end{aligned}$$

Moreover, $\gamma _{\lfloor n/2 \rfloor - 1} \leqslant n^3$.

Proof

We start by bounding $\gamma _{i,l}$ for any l. Let ${\mathbf {x}}$ and j be the profile and the monochromatic block that attain the maximum $\gamma _{i,l}$, respectively. We remark that we are conditioning on the event that $\theta _{i,j}<\theta _{i,j'}$ for all $j'\ne j$ (i.e., the jth block is the first one to disappear). To bound $\gamma _{i,l}$ we distinguish three cases, depending on the value of l.

$\underline{l=1}$: Let u be the unique player of the jth monochromatic block and let v be the player selected for update. Consider all the possible selections of v:

• If $u=v$ and v changes her strategy the block disappears and thus $\theta _j = 1$. This event occurs with probability $\frac{1}{n}\cdot \left( 1 -\frac{1}{1+e^{2\Delta \beta }}\right)$.

• Suppose that v is a neighbor of u and she changes her strategy. Observe that v cannot belong to a monochromatic block of size 1 since we are assuming that j is the first block to disappear. Thus, the two neighbors of v are playing different strategies and when v has to change her strategy she selects at random. We can conclude that the probability that v is selected and she changes her strategy is at most $1/2\cdot 2/n=1/n$. Moreover, after this update the dynamics reach a profile of level $i+1$ where the size of the jth block increases to 2.

• Suppose that v is not a border player of any monochromatic block and she changes her strategy. The players that are not at the border of a block are $n-4(i+1)+s({\mathbf {x}})$ and each of them changes her strategy with probability $\frac{1}{1+e^{2\Delta \beta }}$. We can conclude that the probability of such an event is $\frac{n-4(i+1)+s({\mathbf {x}})}{n(1+e^{2\Delta \beta }})$ and in this case the dynamics reach a profile of level $i + 2$.

• for all the remaining choices for v both the level of the reached profile and the length of the jth monochromatic block remain the same as in the starting profile.

Summing over all the possible choices for v and observing that $\gamma _{i,2}\ge \gamma _{i,1}$ we have

$$\begin{aligned} \gamma _{i,1}&\leqslant \frac{1}{n} \left( 1 - \frac{1}{1+e^{2\Delta \beta }}\right) + \frac{1}{n} (1 + \gamma _{i,2}) + \frac{n-4(i+1) + s({\mathbf {x}})}{n} \frac{1}{1+e^{2\Delta \beta }} (1 + \gamma _{i+1})\\&\quad + \left( \frac{n - 2}{n} - \frac{n - 4(i +1) + s({\mathbf {x}}) - 1}{n} \frac{1}{1 + e^{2\Delta \beta }}\right) (1 + \gamma _{i,1}). \end{aligned}$$

By simple calculations and using the fact that $n - 4(i+1) + s({\mathbf {x}}) \ge 0$, we obtain

$$\begin{aligned} \gamma _{i,1} \leqslant \left( \frac{1}{2} + \frac{1}{4e^{2\Delta \beta } + 2}\right) \left( n + \gamma _{i,2} + \frac{n - 4(i + 1) + s({\mathbf {x}})}{1+e^{2\Delta \beta }} \gamma _{i + 1}\right) . \end{aligned}$$

Finally, since $\left( \frac{1}{2} + \frac{1}{4e^{2\Delta \beta } + 2}\right) \leqslant \frac{2}{3}$ for $\beta =\omega (\log n)$, we can conclude that

$$\begin{aligned} \gamma _{i,1} \leqslant \frac{2}{3} (\gamma _{i,2} + b_i), \end{aligned}$$

(23)

where $b_i = n + \frac{\gamma _{i+1}}{1+e^{2\Delta \beta }} \cdot \max _{{\mathbf y}:\ell ({\mathbf y}) = i +1} (n - 4(i+1) + s({\mathbf y}))$.

$\underline{P1<l<n-2i-1}$: Let v be the player selected for update. Consider all the possible selections of v:

• suppose v is one of the two players at the border of the jth monochromatic block and she changes her strategy. Since the border players are two and their neighbors are playing different strategies, v selects her new strategy at random and thus the probability of this event is 1 / n. Moreover, the dynamics reach a profile ${\mathbf y}$ of level $i+1$ where the length of the jth monochromatic block decreases to $l-1$.

• suppose v does not belong to the jth monochromatic block but she is a neighbor of its border players and she changes her strategy. Then the dynamics reach a profile ${\mathbf y}$ of level $i+1$. Since v cannot belong to a monochromatic block of size 1 (otherwise her block would disappear before block j) we can state that the two neighbors of v are playing different strategies. Thus, when v updates her strategy she chooses each of the two alternatives with probability 1 / 2. Since there are two players adjacent to the border players of block j, this case happens with probability at most 1 / n.

• suppose v is a player that is not a broader player of any monochromatic block and she changes her strategy. Then, when v changes her strategy a monochromatic block is split and the reached profile has level $i+2$. Notice that there are $n - 4(i+1) + s({\mathbf {x}})$ such players and each of them has probability $\frac{1}{1+e^{2\Delta \beta }}$ to change her strategy.

• for all the remaining choices for v both the level of the reached profile and the length of the jth monochromatic block remain the same as in the starting profile.

Hence,

$$\begin{aligned} \gamma _{i,l}&\leqslant \frac{1}{n} (1 + \gamma _{i,l-1}) + \frac{1}{n} (1 + \gamma _{i,l+1}) + \frac{n - 4(i + 1) + s({\mathbf {x}})}{n} \frac{1}{1+e^{2\Delta \beta }} (1 + \gamma _{i+1})\\&+ \left( \frac{n - 2}{n} - \frac{n - 4(i +1) + s({\mathbf {x}})}{n} \frac{1}{1 + e^{2\Delta \beta }}\right) (1 + \gamma _{i,l}). \end{aligned}$$

By simple calculations, similar to the ones for the case $l=1$, we obtain

$$\begin{aligned} \gamma _{i,l} \leqslant \frac{1}{2} (\gamma _{i,l-1} + \gamma _{i,l+1} + b_i). \end{aligned}$$

From the previous inequality and Eq. 23, a simple induction on l shows that, for every $1\leqslant l< n-2i-1$, we have

$$\begin{aligned} \gamma _{i,l} \leqslant \frac{1}{l + 2} \left( (l + 1)\gamma _{i,l+1} + \frac{l(l + 3)}{2} b_i\right) . \end{aligned}$$

(24)

Moreover, from Eq. 24, we can use a simple inductive argument to show that, for every $h \ge 1$,

$$\begin{aligned} \gamma _{i,l}\leqslant & {} \frac{l + 1}{l + h + 1} \gamma _{i, l + h} + \frac{l+1}{2} b_i \sum _{j=l}^{l + h - 1} \frac{j(j+3)}{(j+1)(j+2)} \nonumber \\\leqslant & {} \frac{l + 1}{l + h + 1} \gamma _{i,l + h} + \frac{l+1}{2}hb_i. \end{aligned}$$

(25)

$\underline{l=n-2i-1}$: in this case all blocks other than the jth have size 1. Thus, every time one of these players is selected for update she doesn’t change her strategy otherwise there would be a monochromatic block disappearing before block j. This means that the size of the jth monochromatic block cannot increase. By using an argument similar to the one used in the previous cases, we obtain that

$$\begin{aligned} \gamma _{i,n - 2i - 1} \leqslant \gamma _{i,n - 2i - 2} + b_i. \end{aligned}$$

By using Eq. 24, we have

$$\begin{aligned} \gamma _{i,n - 2i - 1} \leqslant \frac{(n - 2i - 2)(n - 2i + 1) + 2(n - 2i)}{2} b_i \leqslant \frac{n^2}{2} b_i. \end{aligned}$$

Finally, for every $l\ge 1$, by using Eq. 25 with $h = n - 2i - 1 - l$, we have

$$\begin{aligned} \gamma _{i,l} \leqslant \frac{l + 1}{n - 2i} \gamma _{i,n - 2i - 1} + \frac{(l + 1)(n - 2i - 1 - l)}{2} b_i \leqslant n^2 b_i. \end{aligned}$$

The lemma finally follows by observing that $\max _{{\mathbf y}:\ell ({\mathbf y}) = i +1} (n - 4(i+1) + s({\mathbf y})) \leqslant n$ for $0 \leqslant i < \lfloor n/2 \rfloor - 1$ and it is exactly 0 for $i = \lfloor n/2 \rfloor - 1$. $\square$

Corollary D.13

If $\beta =\omega (\log n)$, then for every $i \ge 0$, $\gamma _i = {\mathcal O}(n^3).$

Proof

From Lemma D.12 we have $\gamma _{\lfloor n/2 \rfloor - 1} \leqslant n^3$. Instead, for $0 \leqslant i < \lfloor n/2 \rfloor - 1$, we have

$$\begin{aligned} \gamma _i \leqslant n^3 \left( 1 + \frac{1}{1+e^{2\Delta \beta }} \gamma _{i + 1}\right) \leqslant n^3 \left( 1 + \sum _{j=1}^{\lfloor n/2 \rfloor -i-1} \left( \frac{n^3}{1+e^{2\Delta \beta }}\right) ^j\right) . \end{aligned}$$

(26)

The corollary follows by observing that, if $\beta = \omega \left( \frac{\log n}{\Delta }\right)$, then the summation in Eq. 26 is o(1). $\square$

The above corollary gives a polynomial bound to the time that the dynamics take to go from a profile at level $i+1$ to a profile at level i. Lemma 5.10 easily follows.

Proof

(of Lemma 5.10) From (21) and (22), for every profile ${\mathbf {x}}$ at level $1\leqslant \ell \leqslant n/2$ we have

$$\begin{aligned} \mathbf {E}_{{\mathbf {x}}} \left[ \tau _{{\mathbf p},{\mathbf m}} \right] \leqslant \sum _{i=0}^{\ell -1} \gamma _i = {\mathcal O}(n^4), \end{aligned}$$

where the last bound follows from Corollary D.13. The lemma then follows from the Markov inequality. $\square$

1.5 D.5 Bounding $\alpha _{\mathbf {x}}$ when $\Delta = \delta$

Here we prove the following lemma.

Lemma D.14

For every $d\ge 0$, every profile ${\mathbf {x}}$ with exactly d players playing $+1$ and $\beta = \omega (\log n)$

$$\begin{aligned} \alpha _{\mathbf {x}}= \frac{d}{n} \pm o(1). \end{aligned}$$

Proof

Trivially, $\alpha _{\mathbf p}= 1$ and $\alpha _{\mathbf m}= 0$. We next show that for $\beta = \omega (\log n)$ and ${\mathbf {x}}$ with exactly d players playing $+1$, $\alpha _{\mathbf {x}}= \frac{d}{n} + \lambda _{\mathbf {x}}$, for some $\lambda _{\mathbf {x}}= o(1)$. By the definition of Markov chains we know that

$$\begin{aligned} \alpha _{\mathbf {x}}=P({\mathbf {x}},{\mathbf {x}})\cdot \alpha _{\mathbf {x}}+\sum _{y\in N({\mathbf {x}})} P({\mathbf {x}},{\mathbf y})\cdot p_{\mathbf y}. \end{aligned}$$

We then partition the neighborhood $N({\mathbf {x}})$ of profile ${\mathbf {x}}$ of level i in 5 subsets, $N_1({\mathbf {x}})$, $N_2({\mathbf {x}})$, $N_3({\mathbf {x}})$, $N_4({\mathbf {x}})$, $N_5({\mathbf {x}})$ such that, for two profiles ${\mathbf y}_1,{\mathbf y}_2$ in the same subsets it holds that $P({\mathbf {x}},{\mathbf y}_1)=P({\mathbf {x}},{\mathbf y}_2)$. Then

• $N_1({\mathbf {x}})$ is the set of profiles ${\mathbf y}$ obtained from ${\mathbf {x}}$ by changing the strategy of a player of a plus-block of size 1. Observe that $|N_1({\mathbf {x}})|=s_+({\mathbf {x}})$. Moreover, for every ${\mathbf y}\in N_1({\mathbf {x}})$, ${\mathbf y}$ is at level $i-1$, has $d - 1$ players playing $+1$ and $P({\mathbf {x}},{\mathbf y}) = \frac{1}{n}\cdot (1-\frac{1}{1 + e^{2\Delta \beta }})$.

• $N_2({\mathbf {x}})$ is the set of profiles ${\mathbf y}$ obtained from ${\mathbf {x}}$ by changing the strategy of a player of a minus-block of size 1. Observe that $|N_2({\mathbf {x}})|=s_-({\mathbf {x}})$. Moreover, for every ${\mathbf y}\in N_2({\mathbf {x}})$, ${\mathbf y}$ is at level $i-1$, has $d + 1$ players playing $+1$ and $P({\mathbf {x}},{\mathbf y}) = \frac{1}{n}\cdot (1-\frac{1}{1 + e^{2\Delta \beta }})$.

• $N_3({\mathbf {x}})$ is the set of profiles ${\mathbf y}$ obtained from ${\mathbf {x}}$ by changing the strategy of a border player of a plus-block of size greater than 1. Observe that $|N_3({\mathbf {x}})|=2(i - s_+({\mathbf {x}}))$. Moreover, for every ${\mathbf y}\in N_3({\mathbf {x}})$, ${\mathbf y}$ is at level i, and has $d - 1$ players playing $+1$ and $P({\mathbf {x}},{\mathbf y})=1/2n$.

• $N_4({\mathbf {x}})$ is the set of profiles ${\mathbf y}$ obtained from ${\mathbf {x}}$ by changing the strategy of a border player of a minus-block of size greater than 1. Observe that $|N_4({\mathbf {x}})|=2(i - s_-({\mathbf {x}}))$. Moreover, for every ${\mathbf y}\in N_4({\mathbf {x}})$, ${\mathbf y}$ is at level i, and has $d + 1$ players playing $+1$ and $P({\mathbf {x}},{\mathbf y})=1/{2n}$.

• $N_5({\mathbf {x}})$ is the set of all the profiles ${\mathbf y}\in N({\mathbf {x}})$ that do not belong to any of the previous 4 subsets. Observe that $|N_5({\mathbf {x}})|=n-4i+s({\mathbf {x}})$. Moreover, for every ${\mathbf y}\in N_5({\mathbf {x}})$, ${\mathbf y}$ is at level $i+1$, and $P({\mathbf {x}},{\mathbf y})=\frac{1}{n}\cdot \frac{1}{1+e^{2\Delta \beta }}$.

Moreover, we have that

$$\begin{aligned} P({\mathbf {x}}, {\mathbf {x}}) = \frac{s({\mathbf {x}})}{n} \frac{1}{1 + e^{2\Delta \beta }} + \frac{2i - s({\mathbf {x}})}{n} + \frac{n - 4i + s({\mathbf {x}})}{n} \left( 1 - \frac{1}{1 + e^{2\Delta \beta }}\right) . \end{aligned}$$

Then, we have

$$\begin{aligned} \alpha _{\mathbf {x}}= & {} \frac{1}{n}\left( 1 - \frac{1}{1 + e^{2\Delta \beta }}\right) \left( \sum _{{\mathbf y}\in N_1({\mathbf {x}})} \alpha _{\mathbf y}+ \sum _{{\mathbf y}\in N_2({\mathbf {x}})} \alpha _{\mathbf y}\right) + \frac{1}{2n} \left( \sum _{{\mathbf y}\in N_3({\mathbf {x}})} \alpha _{\mathbf y}+ \sum _{{\mathbf y}\in N_4({\mathbf {x}})} \alpha _{\mathbf y}\right) \nonumber \\&+\, \frac{1}{n}\frac{1}{1 + e^{2\Delta \beta }} \sum _{{\mathbf y}\in N_5({\mathbf {x}})} \alpha _{\mathbf y}\nonumber \\&+ \left( \frac{s({\mathbf {x}})}{n} \frac{1}{1 + e^{2\Delta \beta }} + \frac{2i - s({\mathbf {x}})}{n} + \frac{n - 4i + s({\mathbf {x}})}{n} \left( 1 - \frac{1}{1 + e^{2\Delta \beta }}\right) \right) \alpha _{\mathbf {x}}\nonumber \\= & {} \frac{1}{n} \left( \sum _{{\mathbf y}\in N_1({\mathbf {x}})} \alpha _{\mathbf y}+ \sum _{{\mathbf y}\in N_2({\mathbf {x}})} \alpha _{\mathbf y}\right) + \frac{1}{2n} \left( \sum _{{\mathbf y}\in N_3({\mathbf {x}})} \alpha _{\mathbf y}+ \sum _{{\mathbf y}\in N_4({\mathbf {x}})} \alpha _{\mathbf y}\right) \nonumber \\&+\, \frac{n - 2i}{n} \cdot \alpha _{\mathbf {x}}+ \frac{c}{1 + e^{2\Delta \beta }}, \end{aligned}$$

(27)

where

$$\begin{aligned} c=\frac{1}{n}\left( \sum _{{\mathbf y}\in N_5({\mathbf {x}})} \alpha _{\mathbf y}- \sum _{{\mathbf y}\in N_1({\mathbf {x}}) \cup N_2({\mathbf {x}})} \alpha _{\mathbf y}- (n - 4i) \alpha _{\mathbf {x}}\right) . \end{aligned}$$

We notice that, since $1 \leqslant i \leqslant n/2$ and $|N_1({\mathbf {x}})|+|N_2({\mathbf {x}})|,|N_5({\mathbf {x}})| \leqslant n$, we have $|c|\leqslant 2$ and thus the last term in Eq. 27 is negligible in n (since $\beta =\omega (\log n)$). Hence we have that the following condition holds for every level $i\ge 1$ and every profile ${\mathbf {x}}$ at level i:

$$\begin{aligned} \alpha _{\mathbf {x}}= \frac{1}{2i} \left( \sum _{{\mathbf y}\in N_1({\mathbf {x}})} \alpha _{\mathbf y}+ \sum _{{\mathbf y}\in N_2({\mathbf {x}})} \alpha _{\mathbf y}\right) + \frac{1}{4i} \left( \sum _{{\mathbf y}\in N_3({\mathbf {x}})} \alpha _{\mathbf y}+ \sum _{{\mathbf y}\in N_4({\mathbf {x}})} \alpha _{\mathbf y}\right) + \eta _{\mathbf {x}}, \end{aligned}$$

where $\eta _{\mathbf {x}}$ is negligible in n. This gives us a linear system of equations in which the number of equations is the same that the number of variables.

Let us consider the polished version of this system, in which we omit the negligible part in every equation. We will compute a solution of the polished system. Finally, we argue the solution of original system should be close to the polished one.

Let us denote with $\alpha ^\star _{\mathbf {x}}$ the variable depending on ${\mathbf {x}}$ in the polished system. Note that for each level i and every profile ${\mathbf {x}}$ at level i, $\alpha ^\star _{\mathbf {x}}$ does not depend on profiles at higher level. This enable us to compute a solution for the polished system inductively on level i. Indeed, for every profile ${\mathbf {x}}$ at level 0 (this is only possible for $d=0$ or $d=n$), we have, as discussed above, $\alpha ^\star _{\mathbf {x}}= \frac{d}{n}$. Now, consider ${\mathbf {x}}$ at level i with exactly d players playing $+1$. By assuming that, for every profile ${\mathbf y}$ at level $i-1$, $\alpha ^\star _{\mathbf y}= \frac{d_{\mathbf y}}{n}$, where $d_{\mathbf y}$ is the number of player playing $+1$ in ${\mathbf y}$, we have

$$\begin{aligned} \alpha ^\star _{\mathbf {x}}= \frac{s_+({\mathbf {x}})}{2i} \cdot \frac{d -1}{n} + \frac{s_-({\mathbf {x}})}{2i} \cdot \frac{d +1}{n} + \frac{1}{4i} \left( \sum _{{\mathbf y}\in N_3({\mathbf {x}})} \alpha ^\star _{\mathbf y}+ \sum _{{\mathbf y}\in N_4({\mathbf {x}})} \alpha ^\star _{\mathbf y}\right) . \end{aligned}$$

(28)

Let us now consider Eq. 28 for each ${\mathbf {x}}$ at level i. These gives another linear system of equations. This system has a unique solution: indeed, the number of equations and the number of variables coincides and, moreover, the matrix of coefficients is a diagonally dominant matrix (since $|N_3({\mathbf {x}})| + |N_4({\mathbf {x}})| \leqslant 4i$) and thus it is non-singular. We will show this solution must set $\alpha ^\star _{\mathbf {x}}= \frac{d}{n}$ for every profile ${\mathbf {x}}$ at level i with exactly d players playing $+1$. Indeed, with this assignment the right hand side of the Eq. 28 becomes

$$\begin{aligned} \frac{s_+({\mathbf {x}})}{2i} \frac{d -1}{n} + \frac{s_-({\mathbf {x}})}{2i} \frac{d +1}{n} + \frac{i - s_+({\mathbf {x}})}{2i} \frac{d -1}{n} + \frac{i - s_-({\mathbf {x}})}{2i} \frac{d +1}{n} = \frac{d}{n}. \end{aligned}$$

Thus, we can conclude $\alpha _{\mathbf {x}}^\star = \frac{d}{n}$ for each profile ${\mathbf {x}}$ with exactly d players playing $+1$.

Now, let $\lambda _{\mathbf {x}}= \left| \alpha _{\mathbf {x}}- \alpha _{\mathbf {x}}^\star \right|$. As n grows unbounded, any original equation approaches the polished one, so we need $\alpha _{\mathbf {x}}$ to approach $\alpha _{\mathbf {x}}^\star$. Then $\lambda _{\mathbf {x}}= o(1)$ for every profile ${\mathbf {x}}$. $\square$