Dynamic matching pennies on networks

Abstract

We consider a network game based on matching pennies with two types of agents, conformists and rebels. Conformists prefer to match the action taken by the majority of her neighbors while rebels like to match the minority. We investigate the simultaneous best response dynamic focusing on the lengths of limit cycles (LLC for short). We show that $\hbox {LLC}=1$ or 2 when all agents are of the same type, and $\hbox {LLC}=4$ when there is no conformist-rebel edge and no two even-degreed agents (if any) are neighboring each other. Moreover, $\hbox {LLC}=1$ for almost all type configurations when the network is a line or a ring, which implies that a pure strategy Nash equilibrium is reached from any initial action profile. However, $\hbox {LLC}=4$ for about one half of the type configurations with star networks.

Appendices

Appendix A: Proof of Theorem 1

Proof

Suppose now all agents are conformists. Denote $\alpha (t)=\sum _{i\in N: x_i(t)=1}n_i^1(t-1)$ as the number of edges ij such that $x_i(t)=1$ and $x_j(t-1)=1$, and $\beta (t)=\sum _{i\in N: x_i(t)=1}n_i/2$. Define a Lyapunov function

$$\begin{aligned} \gamma (t)=\alpha (t)-\beta (t)-\beta (t-1). \end{aligned}$$

It is elementary to check that

where

$$\begin{aligned} N_{0\rightarrow 1}(t)= & {} \{i\in N: x_i(t-1)=0, x_i(t+1)=1\}, \\ N_{1\rightarrow 0}(t)= & {} \{i\in N: x_i(t-1)=1, x_i(t+1)=0\}. \end{aligned}$$

By Observation 1, $\gamma (t+1)-\gamma (t)$ is always nonnegative. Consequently, $\gamma (t)$ is a constant in the limit cycle and the following observation holds.

Observation 3

Suppose that a limit cycle is reached at time $t-1$. If $i\in N_{0\rightarrow 1}(t)\cup N_{1\rightarrow 0}(t),$ i.e.,  i takes different actions at time $t-1$ and $t+1,$ then $n_i^1(t)=n_i/2$.

We show in the rest of the proof that $N_{0\rightarrow 1}(t)\cup N_{1\rightarrow 0}(t)=\emptyset$ in the limit cycle, i.e., the action of each agent will be back within two steps, implying that LLC equals 1 or 2.

Suppose on the contrary that $N_{0\rightarrow 1}(t)\cup N_{1\rightarrow 0}(t)\ne \emptyset$ for large enough t such that a limit cycle is reached. If there exists $i\in N_{0\rightarrow 1}(t)$, we must have $x_i(t+1)=1$ due to $x_i^1(t)=n_i/2$. Let $t+k$ be the first time after t such that $x_i(t+k)=0$ with $k\ge 2$. Then $x_i(t+k)=0$ and $x_i(t+k-2)=1$, implying that $i\in N_{1\rightarrow 0}(t+k-1)$ and hence $n_i^1(t+k-1)=n_i/2$ due to Observation 3. Therefore, $x_i(t+k-1)=0$. This contradicts the hypothesis that $t+k$ is the first time after t such that $x_i(t+k)=0$. Using the same logic, the case that $i\in N_{1\rightarrow 0}(t)$ is impossible as well.

When all agents are rebels, the same Lyapunov function $\gamma (t)$, which is nonincreasing due to Observation 2, still applies. We can show that $N_{0\rightarrow 1}(t)\cup N_{1\rightarrow 0}(t)=\emptyset$ holds in the limit cycle in a way similar to the all-conformist case. This finishes the proof. $\square$

Appendix B: Proofs in Sect. 4

We assume throughout this appendix that (i) $n\ge 2$ whenever the underlying network is a line, (ii) $n\ge 3$ whenever it is a ring, and (iii) $n\ge 4$ whenever it is a star.

1.1 Proofs of Theorems 3 and 4

We prove Theorems 3 and 4 simultaneously.

Definition 6

Given an initialized NMP ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0))$, if $x_i(t)$ keeps unchanged when simultaneous BRD enters the limit cycle, we say agent i is immune w.r.t. ${\mathcal {I}}$.

The lemma below characterizes the convergence case.

Lemma 2

Given an initialized NMP ${\mathcal {I}}=(N,\mathrm{g},T, {\mathbf {x}}(0)),$ and suppose the underlying network $(N,\mathrm{g})$ is either a line or a ring. The following statements are equivalent.

• (a) $\ell ({\mathcal {I}})=1;$

• (b) All agents are immune w.r.t. ${\mathcal {I}};$

• (c) At least one agent is immune w.r.t. ${\mathcal {I}}$.

Proof

The equivalence between (a) and (b) is obvious. It is trivial that (b) implies (c), so we are left to show that (c) implies (b). Suppose that i is immune. We claim that $j\in N_i$, an arbitrary neighbor of i, is also immune.

Suppose not, then there would exist some time $t_0\ge r({\mathcal {I}})$ such that $u_j({\mathbf {x}}(t_0))<0$. So i is not liked by agent j at time $t_0$, and j will switch, i.e., $x_j(t_0+1)=1-x_j(t_0)$. Since i is immune, we know that she never switches, i.e., $x_i(t_0+1)=x_i(t_0)$. Therefore, i is liked by agent j at time $t_0+1$, and $u_j({\mathbf {x}}(t_0+1))\ge 0$. Furthermore, the utility of j keeps being nonnegative after time $t_0+1$, because i never switches in the limit cycle and hence she is always liked by j after $t_0$. It follows that agent j will never switch any more, and the action $x_j(t_0)$ will never be taken again by j after $t_0$, contradicting the fact that a limit cycle is entered at time $t_0$. $\square$

Notice that the above lemma relies crucially on the fact that each agent has at most two neighbors in lines and rings. As long as one of her two neighbors is liked by her, she will be satisfied and will not deviate. This is generally not true for other structures. The following lemma characterizes length-two limit cycles.

Lemma 3

Given an initialized NMP ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0)),$ and suppose the underlying network $(N,\mathrm{g})$ is either a line or a ring. The following statements are equivalent.

• (a) $\ell ({\mathcal {I}})=2;$

• (b) $\exists i\in N$ such that $u_i({\mathbf {x}}(t))<0$ holds for all $t\ge r({\mathcal {I}});$

• (c) $\exists t\ge 0$ such that $u_i({\mathbf {x}}(t))<0$ holds for all $i\in N;$

• (d) $u_i({\mathbf {x}}(0))<0$ holds for all $i\in N$.

Proof

(a) implies (b) trivially. Suppose (b) is true, then agent i keeps switching her action in the limit cycle. This can only happen when her neighbors keep switching actions too. Using this argument repeatedly, we know that no agent is satisfied at any step of the limit cycle, thus (c) is valid. Suppose now (c) is true, i.e., at some time t no agent is satisfied. This can only happen when all agents are of the same type, because for any two adjacent agents of different types, it is impossible for both of them to be unsatisfied. In addition, no agent should be initially satisfied, because otherwise the initially satisfied agent and one of her neighbors will both be immune. This indicates that (d) is valid. (d) implies (a) obviously. This finishes the proof. $\square$

Proofs of Theorem 3(b)(i) and Theorem 4(i)

The equivalence between (a) and (d) in Lemma 3 implies that Theorem 3(b)(i) and Theorem 4(i) are both valid. $\square$

Proof of Theorem 3(b)(ii)

Suppose two agents in some weak cluster initially like each other. Then it can be seen that the two agents are immune. It follows from Lemma 2 that $\ell ({\mathcal {I}})=1$. $\square$

Proof of Theorem 3(b)(iii)

Suppose G is strongly clustered and some agent (not necessarily in any strong cluster) is initially satisfied. Then at least one of the following must hold: (i) $T_1=T_2$; (ii) $T_{n-1}=T_n$; and (iii) there are three adjacent agents that are of the same type.

Suppose $T_1=T_2$ and they are both conformists. If at some step they take the same action, then they will both be immune. By Lemma 2, this means that $\ell ({\mathcal {I}})=1$ and we are done. So we suppose now they always take different actions. Since agent 1 has only one neighbor, this means that she will keep switching actions. By the equivalence between (a) and (b) in Lemma 3, we have $\ell ({\mathcal {I}})=2$ and hence no agent is initially satisfied (Lemma 3(d)), contradicting the hypothesis that some agent is initially satisfied. If agents 1 and 2 are both rebels, it can be shown by a similar argument. By symmetry, it is also valid for the case that $T_{n-1}=T_n$.

Suppose now there are three agents, $i,i+1$ and $i+2$, such that $T_i=T_{i+1}=T_{i+2}=C$. If there exists some time t such that either $x_i(t)=x_{i+1}(t)$ or $x_{i+1}(t)=x_{i+2}(t)$, then there will be immune agents and $\ell ({\mathcal {I}})=1$. So we assume that $x_i(t)=x_{i+2}(t)\ne x_{i+1}(t)$ holds for all time t. Therefore, the middle agent $i+1$ will switch her action at all steps. By Lemma 3, it follows that $\ell ({\mathcal {I}})=2$, contradicting the hypothesis that some agent is initially satisfied.

Suppose now there are three agents, $i,i+1$ and $i+2$, such that $T_i=T_{i+1}=T_{i+2}=R$. If there exists some time t such that either $x_i(t)\ne x_{i+1}(t)$ or $x_{i+1}(t)\ne x_{i+2}(t)$, then there will be immune agents and $\ell ({\mathcal {I}})=1$. So we assume that $x_i(t)=x_{i+2}(t)= x_{i+1}(t)$ holds for all time t. Therefore, the middle agent $i+1$ will switch her action at all steps. By Lemma 3, it follows that $\ell ({\mathcal {I}})=2$, contradicting the hypothesis that some agent is initially satisfied. $\square$

Proof of Theorem 4(iii)

The proof is exactly the same as that of Theorem 3(b)(iii), except that the first two cases do not occur. $\square$

To prove part (a) and (b)(iv) of Theorem 3, we need a property about LLC$\ge$3.

Lemma 4

Let ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0))$ be an initialized NMP with the underlying network $(N,\mathrm{g})$ being either a line or a ring. If $\ell ({\mathcal {I}})\ge 3,$ then no agent switches in two consecutive steps,  i.e.,  there is no agent i and time t such that $u_i({\mathbf {x}}(t))=u_i({\mathbf {x}}(t+1)){<}0$.

Proof

We consider first the case that $(N,\mathrm{g})$ is a line. Suppose on the contrary that $u_i({\mathbf {x}}(t))=u_i({\mathbf {x}}(t+1))<0$. It can be shown easily that (i) if $i\in \{1,n\}$, then i is of the same type as her only neighbor, and (ii) if $i\notin \{1,n\}$, then i and her two neighbors must have the same type. Hence ${\mathcal {I}}$ is strongly clustered. Due to Theorem 3 (b)(i)(iii), we know that $\ell ({\mathcal {I}})\le 2$. A contradiction with the hypothesis that $\ell ({\mathcal {I}})\ge 3$. The case that $(N,\mathrm{g})$ is a ring can be proved in exactly the same approach. $\square$

The key method we use to deal with part (b)(iv) of Theorem 3 is the “cutting” technique. To ease the presentation, we need one more term. Given an initialized NMP ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0))$, and suppose that $(N,\mathrm{g})$ is a line. For any agent i, $2\le i\le n-1$, let $\mathcal {I'}(i)=(N',\mathrm{g}',T',x'(0))$ be a new (smaller) initialized NMP constructed by the left i agents of ${\mathcal {I}}$ in the following natural way: $N'=\{1,2,\ldots ,i\}$; $\mathrm{g}'=\{ij: i,j\in N', ij\in \mathrm{g}\}$; $\forall 1\le j\le i$, $T'_j=T_j$; $\forall 1\le j\le i$, $x'_j(0)=x_j(0)$. Similarly, $\mathcal {I''}(i)$ is an initialized NMP constructed by the right $n-i$ agents of ${\mathcal {I}}$.

Definition 7

We say that $\mathcal {I'}(i)$ is a complete sub-instance of ${\mathcal {I}}$ if and only if $x'_j(t)=x_j(t)$ holds for all $t\ge 1$ and all $1\le j\le i$. $\mathcal {I''}(i)$ is called a complete sub-instance in an analogous way.

Lemma 5

Given an initialized NMP ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0)),$ and suppose the underlying network is a line that is not completely heterophilic. Suppose also that $\ell ({\mathcal {I}})\ge 3$ and agents i and $i+1$ are of the same type ($2\le i\le n-2$). Then $\mathcal {I'}(i)$ and $\mathcal {I''}(i)$ are both complete sub-instances of ${\mathcal {I}}$.

Proof

By symmetry, we only consider $\mathcal {I'}(i)$. Suppose w.l.o.g. that i and $i+1$ are both conformists. In the simultaneous BRD of ${\mathcal {I}}$, i and $i+1$ will always take the opposite actions, because otherwise they will both be immune and the hypothesis that $\ell ({\mathcal {I}})\ge 3$ will be violated. Therefore, whether i will switch her action or not at step t is completely determined by the action of agent $i-1$. This is also true for the simultaneous BRD of $\mathcal {I'}(i)$, where $i-1$ is the only neighbor of i. Hence the lemma. $\square$

With the assistance of Lemma 6 and Lemma 7 that are to be presented in Appendix B.2, we are ready to prove the remaining parts of Theorem 3.

Proof of Theorem 3(a)

Suppose now ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0))$ is an initialized NMP, and the underlying network is a completely heterophilic line. Recall first that a PNE does not exist in this case (Lemma 1(d)). Since none of the agents has a utility of zero in the limit cycle (Lemma 7), and 2 will never be adjacent with 2 and neither will -2 be adjacent with -2 in any utility sequence (Lemma 6(a)), the whole sequence of utilities will consist of alternate 2 and -2 (except for the ending agents, whose utilities are 1 or -1). At any step of the limit cycle, either none conformist is satisfied but every rebel is, or none rebel is satisfied but every conformist is. Based on the above discussion, it is easy to check that the length of any limit cycle is exactly 4, just as in the example of the matching pennies game that is illustrated in Fig. 2. $\square$

Proof of Theorem 3(b)(iv)

Suppose now ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0))$ is an initialized NMP, and the underlying network is a line that is a neither strongly clustered nor completely heterophilic (note that this is exactly the case discussed in Theorem 3(b)(iv)). Since ${\mathcal {I}}$ is not strongly clustered, it follows from Theorem 3(b)(i) that $\ell ({\mathcal {I}})\ne 2$. To prove Theorem 3(b)(iv), it suffices to show that $\ell ({\mathcal {I}})\ge 3$ implies $\ell ({\mathcal {I}})=4$.

Suppose now $\ell ({\mathcal {I}})\ge 3$. We cut the line at an arbitrary conformist-conformist link or rebel-rebel link into two pieces. Notice that this cutting will not produce isolated agents because ${\mathcal {I}}$ is not strongly clustered. Due to Lemma 5, they are both complete sub-instances. If the two pieces are both completely heterophilic, then each of them has $\hbox {LLC}=4$, regardless of the initial action profiles (Theorem 3(a)). By the definition of a complete sub-instance, piecing them together gives Theorem 3(b)(iv) immediately. Otherwise, we cut them again into even smaller pieces, and use the same argument. Since the line is finite, this process will terminate and the proof is done. $\square$

Proof of Theorem 4(iv)

To prove Theorem 4(iv), we use a similar cutting technique. Observe that a ring discussed in this case will be cut into completely heterophilic lines instead of completely heterophilic rings. Therefore, although we do not have $\hbox {LLC}=4$ for completely heterophilic rings (see Remark 5), we can still use the same logic as in the proof of Theorem 3(b)(iv) to show that Theorem 4(iv) is correct. $\square$

1.2 Two technical lemmas

The two lemmas provided in this subsection are used in the proofs of Theorem 3(a) and (b)(iv). Since their proofs, mainly the latter one, are rather technical, we present them in this separate subsection.

In this subsection, we use $u_i(t)$ for short of $u_i({\mathbf {x}}(t))$. This is reasonable, because the utility of each agent is a function of the single variable of time t once the initial action profile is fixed.

Since the underlying network $(N,\mathrm{g})$ is a line, $u_i(t)$ takes five possible values, 2, 1, 0, $-1$, and $-2$. We call $[a_1,a_2,\ldots ,a_k]\in \{-2,2,0, -1,1\}^k$ a utility sequence, if there exists a time t and an agent i such that $[u_i(t),u_{i+1}(t),\ldots , u_{i+k-1}(t)]=[a_1,a_2,\ldots ,a_k]$. It is sometimes more convenient to deal with utility sequences than to deal with action sequences.

Lemma 6

Given an initialized NMP ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0)),$ and suppose $(N,\mathrm{g})$ is a completely heterophilic line.

• (a) The following utility sequences never occur :  [2, 2],  $[-2,-2],$ [2, 0, 2],  $[2,0,\ldots ,0,2],$ [2, 0, 1],  $[2,0,\ldots ,0,1],$$[-2,0,-2],$$[-2,0,\ldots ,0,-2],$$[-2,0,-1],$$[-2,0,\ldots ,0,-1],$$[1,0,\ldots ,0,1],$ and $[-1,0,\ldots ,0,-1];$

• (b) If $u_i(t)=-2,$ then $u_i(t+1)=2;$

• (c) If $u_i(t)=0,$ then $u_i(t+1)\in \{-2,0\}$.

Proof

One nice property of the completely heterophilic line is that one agent likes her neighbor at some time if and only if her neighbor dislikes her. Using this property, it is easy to check that the sequences in (a) never occurs.

Since no pair of adjacent agents can be simultaneously unsatisfied, we know that if agent i switches her action at some step, then none of her neighbors switches at the same step. Using this fact, (b) is valid.

If $u_i(t)=0$, then i likes one of her neighbors and dislikes the other at time t. Suppose w.l.o.g. that $i-1\in L_i({\mathbf {x}}(t))$ and $i+1\in D_i({\mathbf {x}}(t))$. Accordingly, $i\in D_{i-1}({\mathbf {x}}(t))$ and $i\in L_{i+1}({\mathbf {x}}(t))$. Therefore, $u_{i-1}(t)\le 0$ and $u_{i+1}(t)\ge 0$. Since $i+1$ is satisfied at time t, she will not switch. In the case that $u_{i-1}(t)<0$, it is evident that $u_{i}(t+1)=-2$. And in the case that $u_{i-1}(t)=0$, we have $u_{i}(t+1)=0$. Hence (c) is correct. This finishes the proof. $\square$

Given an initialized NMP ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0))$, let Z(t) be the set of agents whose utilities are zero at time t, i.e.,

$$\begin{aligned} Z(t)=\{i\in N: u_i(t)=0\}. \end{aligned}$$

Lemma 7

Given an initialized NMP ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0)),$ suppose the underlying network $(N,\mathrm{g})$ is a completely heterophilic line. Then $Z(t)=\emptyset$ for all $t\ge r({\mathcal {I}}),$ i.e.,  no agent has utility zero in the limit cycle.

Proof

First, it follows from Lemma 1(a) that PNE does not exist for ${\mathcal {I}}$ and hence $L({\mathcal {I}})\ne 1$. Due to Lemma 3, we obviously have $L({\mathcal {I}})\ne 2$. Hence

$$\begin{aligned} L({\mathcal {I}})\ge 3, \end{aligned}$$

(1)

and the results in Lemma 4, Lemma 5 and Lemma 6 are all valid. We prove in two steps.

(i) We prove first that the cardinality of Z(t) is non-increasing, i.e.,

$$\begin{aligned} |Z(t+1)|\le |Z(t)|. \end{aligned}$$

(2)

Let

$$\begin{aligned} Z^{+}(t)=Z(t+1){\setminus } Z(t), \end{aligned}$$

and

$$\begin{aligned} Z^{-}(t)=Z(t){\setminus } Z(t+1), \end{aligned}$$

we only need to show that

$$\begin{aligned} |Z^+(t)|\le |Z^{-}(t)|. \end{aligned}$$

If $Z^+(t)$ is empty, we are done. Suppose not. Take an arbitrary agent i from $Z^+(t)$, i.e., $u_i(t+1)=0$ and $u_i(t)\ne 0$. Then by Lemma 6(b), it can only be that $u_i(t)=2$. And it is obvious that $i\notin \{1,n\}$. So agent i does not switch her action at step t, and her utility changes from 2 at step t to 0 at step $t+1$. This can only happen when exactly one neighbor of i is unsatisfied at time t. Due to Lemma 6(a), utility sequence [2, 2] never occurs, so the neighbor of agent i who is satisfied at time t must have a utility of 0. Therefore, $\{u_{i-1}(t), u_{i+1}(t)\}=\{-2,0\}$ (we assume w.l.o.g. that $i-1\ne 1$ and $i+1\ne n$).

Claim 1 (i) If $[u_{i-1}(t),u_i(t), u_{i+1}(t)]=[-2,2,0],$ then $Z^-(t)\cap \{i+1,i+2,\ldots ,n\}\ne \emptyset$. (ii) If $[u_{i-1}(t),u_i(t), u_{i+1}(t)]=[0,2,-2],$ then $Z^-(t)\cap \{1,2,\ldots ,i-1\}\ne \emptyset$.

Suppose now $[u_{i-1}(t),u_i(t), u_{i+1}(t)]=[-2,2,0]$, and suppose on the contrary to Claim 1(i) that $Z^-(t)\cap \{i+1,i+2,\ldots ,n\}=\emptyset$. Then $i+1\notin Z^-(t)$. Since $i+1\in Z(t)$, this implies that $i+1\in Z(t+1)$, i.e., $u_{i+1}(t+1)=0$. As agent $i+1$ does not switch her action at step t, her left neighbor, agent i, does not switch either, and the utility of $i+1$ also keeps unchanged, we know that the right neighbor of agent $i+1$, namely, agent $i+2$, should not switch at step t either. Therefore, $u_{i+2}(t)\ge 0$. By Lemma 6(a), utility sequence [2, 0, 2] and [2, 0, 1] never occurs, implying it is impossible that $u_{i+2}(t)\in \{2,1\}$. It can only be that $u_{i+2}(t)=0$.

In addition, $Z^-(t)\cap \{i+1,i+2,\ldots ,n\}=\emptyset$ implies that $i+2\notin Z^-(t)$. It follows from $u_{i+2}(t)=0$ that $i+2\in Z(t)$. Using exactly the same argument as in the last paragraph, we know that $u_{i+3}(t)=0$. But this cannot continue for ever, because $u_n(t)\in \{1,-1\}$ for all time t. Hence a contradiction. Therefore, Claim 1(i) is valid. By symmetry, Claim 1(ii) is also valid.

Due to Claim 1, the following definition for each $i\in Z^+(t)$ is valid:

$$\begin{aligned} f(i)=\left\{ \begin{array}{llll}\min \big \{j: j\in Z^-(t)\cap \{i+1,i+2,\ldots ,n\}\big \}&{}if~[u_{i-1}(t),u_i(t), u_{i+1}(t)]=[-2,2,0]\\ \max \big \{j:j\in Z^-(t)\cap \{1,2,\ldots ,i-1\}\big \}&{}if~[u_{i-1}(t),u_i(t), u_{i+1}(t)]=[0,2,-2]\end{array}\right. . \end{aligned}$$

The facts below can also be observed from the above discussion. For all $i\in Z^-(t)$,

$$\begin{aligned} f(i)>i~\text { implies}~[u_{i-1}(t),\ldots ,u_{f(i)+1}(t)]=[-2,2,0,\ldots ,0,-2], \end{aligned}$$

(3)

$$\begin{aligned} f(i)<i~\text { implies}~[u_{f(i)-1}(t),\ldots ,u_{i+1}(t)]=[-2,0,\ldots ,0,2,-2], \end{aligned}$$

(4)

where the “$\ldots$”s on the right hand sides both denote a series of 0’s.

Since $f(\cdot )$ is a mapping from $Z^+(t)$ to $Z^-(t)$, if we could show that $i_1\ne i_2$ implies $f(i_1)\ne f(i_2)$, then $|Z^+(t)|\le |Z^-(t)|$ would be valid. Assume w.l.o.g. that $i_1<i_2$. We discuss in four cases.

• $f(i_1)<i_1$ and $f(i_2)>i_2$. Since $f(i_2)>i_2>i_1>f(i_1)$, it is true that $f(i_1)\ne f(i_2)$;

• $f(i_1)>i_1$ and $f(i_2)>i_2$. It can be seen from (3) that $u_j(t)=0$ for all $i_1<j\le f(i_1)$. It follows from $i_2>i_1$ and $u_{i_2}(t)\ne 0$ that $i_2>f(i_1)$. Hence $f(i_2)>i_2>f(i_1)$, and consequently $f(i_1)\ne f(i_2)$;

• $f(i_1)<i_1$ and $f(i_2)<i_2$. It can be seen from (4) that $u_j(t)=0$ for all $f(i_2)\le j<i_2$. It follows from $i_2>i_1$ and $u_{i_1}(t)\ne 0$ that $i_1<f(i_2)$. Hence $f(i_1)<i_1<f(i_2)$, and consequently $f(i_1)\ne f(i_2)$;

• $f(i_1)>i_1$ and $f(i_2)<i_2$. It can be seen from (3) that $u_{f(i_1)+1}(t)=-2$. But we know from (4) that $u_{f(i_2)+1}(t)\in \{0,2\}$, hence $f(i_1)\ne f(i_2)$.

This shows that (2) is indeed correct.

(ii) By the monotonicity property (2) we know immediately that the cardinality of Z(t) keeps unchanged in the limit cycle, i.e.,

$$\begin{aligned} |Z(t+1)|=|Z(t)|, \forall t\ge r({\mathcal {I}}). \end{aligned}$$

(5)

Consequently, we know that

$$\begin{aligned} \forall t\ge r({\mathcal {I}}),\forall j\in Z^-(t), \text{ there } \text{ exists } \text{ exactly } \text{ one } i\in Z^+(t)~ s.t.~ f(i)=j. \end{aligned}$$

(6)

Now, we are ready to prove the final result. We use the minimum counter-example argument. Suppose the lemma is not valid, and ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0))$ is a counter-example with the smallest number of agents. Let $j^*$ be the agent with the largest index among all the agents having a utility of 0 for at least one time in the limit cycle.

First of all, it is verifiable that $j^*=n-1$, because otherwise we could construct a smaller counter-example. In fact, the choice of $j^*$ tells us that if $j^*\ne n-1$, then $x_{j^*+1}(t)\in \{2,-2\}$ holds for all $t\ge r({\mathcal {I}})$. In the limit cycle, if agent $j^*+1$ dislikes $j^*$, then her utility will be $-2$, because 0 is never attained, and she will switch her action. And when agent $j^*+1$ likes $j^*$, evidently she will not switch. Therefore, whether $j^*+1$ switches her action at time $t\ge r({\mathcal {I}})$ or not is completely determined by the action of $j^*$. Using the same argument as in the proof of Lemma 5, we know that $\mathcal {I'}(j^*+1)=(N',\mathrm{g}', T', x'(0))$ is also a qualified counter-example (a limit cycle is entered at the first step), where $N'=\{1,2,\ldots ,j^*+1\}$, $T'$ the corresponding sub-vector of T, $\mathrm{g}'$ the corresponding left part of $\mathrm{g}$, and $x'(0)=(x_1(r({\mathcal {I}})),x_2(r({\mathcal {I}}),\ldots ,x_{j^*+1}(r({\mathcal {I}}))))$. Since ${\mathcal {I}}$ is the smallest counter-example, this is impossible, and thus it can only be that $j^*=n-1$.

Suppose $u_{j^*}(t')=0$ for some $t'\ge r({\mathcal {I}})$, then there must exist $t^*\ge t'$ such that $u_{j^*}(t^*)=0$ and $u_{j^*}(t^*+1)\ne 0$. Because otherwise, the utility of $j^*$ would keep at 0 from time $t'$ on, and thus she would be immune. As a result, $\ell ({\mathcal {I}})=1$ and a PNE would be reached by the simultaneous BRD. This is a contradiction with (1).

Therefore, $j^*\in Z^-(t^*)$. Since $t^*\ge t'\ge r({\mathcal {I}})$, we know by (6) that there exists exactly one $i^*\in Z^+(t^*)$ such that $f(i^*)=j^*$. Since $j^*=n-1$, we know that $i^*<j^*$ (note that agent n never has utility 0, while $i^*\in Z^+(t^*)$ implies that agent $i^*$ has utility 0 at time $t^*+1$). Therefore, it follows from (4) that

$$\begin{aligned} {[}u_{i^*-1}(t^*),\ldots , u_{j^*}(t^*), u_n(t^*)]=[-2,2,0,\ldots ,0,-1]. \end{aligned}$$

By Lemma 6(c), it must hold that $u_{j^*}(t^*+1)=-2$. Therefore,

$$\begin{aligned} {[}u_{i^*-1}(t^*+1),\ldots , u_n(t^*+1)]=[2,0,0,\ldots ,-2,1]. \end{aligned}$$

(7)

It can be observed from (7) that $j^*-1\in Z^-(t^*+1)$. Again, using (6), we know that there is an agent $k^*\in Z^+(t^*+1)$ such that $f(k^*)=j^*-1$. It must hold that $k^*<j^*-1$, because (7) tells us that the sequence $[0,2,-2]$ never occurs on the right side of $j^*-1$ and (4) says that if $k^*>j^*-1$ we would have a sequence $[0,2,-2]$ on the right side of $j^*-1$ (note by definition that $f(i)\ne i$ for any $i\in Z^+(t)$). Furthermore, comparing (3) with (7), we know that $k^*=i^*-1$, which can only happen when $u_{i^*-2}(t^*+1)\in \{-2,-1\}$.

However, $u_{i^*-2}(t+1)=-1$, i.e., $i^*-2=1$, is not true. Because if otherwise, then we would have $[u_{i^*-2}(t^*+2),\ldots , u_n(t^*+2)]=[1,0,0,0,\ldots ,-2, 2,-1]$, which has $n-4$ 0s in total. However, it can be checked that we would further have $[u_{i^*-2}(t^*+3),\ldots , u_n(t^*+3)]=[1,0,0,0,\ldots ,-2,2,-2,1],$ which has $n-5$ 0s in total, a contradiction with (5).

Therefore, we have $u_{i^*-2}(t+1)=-2$ and

$$\begin{aligned} {[}u_{i^*-2}(t^*+2),\ldots , u_n(t^*+2)]=[2,0,0,0,\ldots ,-2, 2,-1]. \end{aligned}$$

(8)

It can be observed from (8) that $j^*-2\in Z^-(t^*+2)$. Again, there is no $[0,2,-2]$ on the right side of $j^*-2$, and so we can only have $f(j^*-2)=i^*-2$. This can only happen when $u_{i^*-3}(t^*+2)\in \{-2,-1\}$. Again, we can deduce that $u_{i^*-3}(t^*+2)=-2$ and

$$\begin{aligned} {[}u_{i^*-3}(t^*+3),\ldots , u_n(t^*+3)]=[2,0,0,0,\ldots ,-2, 2,-2,1]. \end{aligned}$$

(9)

This process will be repeated infinitely. But this is obviously impossible, because the line is finite. This establishes the lemma. $\square$

1.3 Proof of Theorem 5

Before presenting the main proof, we need a lemma that is similar to Lemma 2 and Lemma 3.

Lemma 8

Let ${\mathcal {I}}=(N,\mathrm{g},{\mathbf {T}},{\mathbf {x}}(0))$ be an initialized NMP and suppose the underlying network $(N,\mathrm{g})$ is a star. The following statements are true.

• (a) At every step of the limit cycle,  all the peripheral conformists choose the same action,  and so do all the peripheral rebels; 

• (b) $\ell ({\mathcal {I}})=1$ if and only if all agents are satisfied at some time,  if and only if some agent is always satisfied in the limit cycle.

• (c) $\ell ({\mathcal {I}})=2$ if and only if all agents are unsatisfied at some time,  if and only if some agent is always unsatisfied in the limit cycle.

Proof

• (a) We only prove for conformists, because the rebel half is analogous. Suppose on the contrary that at some step of the limit cycle, part of the peripheral conformists choose action 1 and the other peripheral conformists choose action 0. Since all the peripheral agents have the same single neighbor, namely, the central agent, it must be true that either all the action 1 peripheral conformists are satisfied but the action 0 ones are unsatisfied, or the opposite. Suppose w.l.o.g. the former case occurs. Then the action 1 peripheral conformists switch to action 0, while the action 0 ones keep their actions fixed at 0. Hence, all the peripheral conformists have an action of 0 at the next step. From that time on, at each step, either they simultaneously switch or simultaneously keep unchanged. So, in any of the following steps, they always share the same action, and hence the state we assume that part of the peripheral conformists choose action 1 and the others choose action 0 will never be reached again, contradicting our hypothesis that it is in the limit cycle.

• (b) The first part is straightforward, we only show the second part. The “only if” side is trivial. Suppose now some agent is always satisfied in the limit cycle, then the central agent is also always satisfied in the limit cycle, and consequently so are all the peripheral ones. This shows that the “if” side of the second part is also correct.

• (c) The “if” side of the first part is trivial. $\hbox {LLC}=2$ implies that if some agent is satisfied at some step of the limit cycle, then it is always satisfies in the limit cycle. Therefore, the “only if” side of the first part is true due to (a). The “only if” side of the second part is trivial due to the first part. The “if” side of the second part is also true, because some agent is always unsatisfied in the limit cycle implies that the central agent and furthermore all agents are unsatisfied in the limit cycle.$\square$

Proof of Theorem 5

• (a) Consider the case that the central agent is a rebel. Then peripheral conformists are more than peripheral rebels. Observe first that $\ell ({\mathcal {I}})\ge 2$, because a PNE exists for star networks if and only if at least one half of the peripheral agents are of the same type as the central one. Since all of the conformists choose the same action in the limit cycle (Lemma 8 (a)), it is impossible that $\ell ({\mathcal {I}})=2$, because at any step of the limit cycle, either all of the conformists are satisfied (when they take the same action as the central agent) or the central rebel is satisfied (when the peripheral conformists take a different action with the central rebel).

  Using Lemma 8(c), there must exist some time t in the limit cycle such that the central rebel is satisfied, say she chooses action 0. Since there are more peripheral conformists than peripheral rebels, we know that all peripheral conformists should choose action 1 at time t (hence they are also satisfied). We discuss in two cases about the peripheral rebels.

  • (i) All of the peripheral rebels are unsatisfied at time t. This implies that they all choose action 0 at time t. At time $t+1$, all the peripheral agents switch actions, making themselves happy but the central rebel unhappy. So at time $t+2$, the central rebel switches to action 1, making herself happy and all the peripheral agents unhappy. Similarly, at time $t+3$, all the peripheral agents, but not the central one, switch, and at time $t+4$, only the central rebel switches, leading back to the action profile at time t. Therefore, we have $\ell ({\mathcal {I}})=4$ in this case. This limit cycle is illustrated in the left part of Fig. 6.

  • (ii) All of the peripheral rebels are satisfied at time t. This implies that they all choose action 1 at time t. It can be checked that the state at time $t+1$ in this case is exactly that at time $t+1$ in case (i). However, the time t state in this case is none of the four states in the limit cycle of the previous case, so it will never be reached again, contradicting our hypothesis that a limit cycle has been entered at time t. Therefore, this case never occurs.

  The proof of the situation that the central agent is a conformist is quite similar and thus omitted. The limit cycle in this case is illustrated in the right part of Fig. 6.

• (b) Let’s consider the case that the central agent is a rebel.

  Suppose w.l.o.g. that the central rebel chooses action 1 initially. Since she is initially unsatisfied, at step 2 she will switch to action 0. So do the peripheral rebels who initially take action 1. For the peripheral rebels who initially take action 0, they are initially satisfied and do not switch at the first step. For the peripheral conformists who take action 1 initially, they are satisfied at the first step, and hence still have action 1 at the second step. For the peripheral conformists who take action 0 initially, they are unsatisfied at the first step, and hence switch to action 1 at the second step.

• (c) Naturally, we discuss in four cases.

  • (i) The central agent is a rebel, more than half of the peripheral agents are also rebels, and initially the central rebel is satisfied. Since the central rebel is initially satisfied, her action at the second step will be the same as in the first step. For the peripheral rebels, the ones initially taking the opposite action keep unchanged, and the other peripheral rebels switch to this action too at the second step. So, at the second step, each of the peripheral rebels takes an opposite action to the central rebel. Since more than one half of the central rebel’s neighbors are rebels, we know that all the rebels will be always satisfied from the second step on. Using Lemma 8(b), we have $\ell ({\mathcal {I}})=1$.

  • (ii) The central agent is a conformist, more than one half of the peripheral agents are also conformists, and initially the central conformist is satisfied. The proof is quite similar to that of case (i) and thus omitted.

  • (iii) The central agent is a rebel, and exactly one half of the peripheral agents are rebels and one half of them are conformists. Suppose on the contrary that $\ell ({\mathcal {I}})\ge 2$. It follows from Lemma 8(b) that there is some time t in the limit cycle such that the central rebel is unsatisfied. This must be the case that all the agents take the same action, say action 1, because when the peripheral conformists and the peripheral rebels take different actions, the central rebel is satisfied. So, at step $t+1$, all of the rebels switch to action 0, and the conformists still hold action 1. Therefore, the central rebel is the only satisfied agent at step $t+1$. At time $t+2$, the central rebel still has action 0, the peripheral rebels have action 1, and the conformists have action 0. It can be seen that all of the agents are satisfied at step $t+2$, contradicting the hypothesis that $\ell ({\mathcal {I}})\ge 2$.

  • (iv) The central agent is a conformist, and one half of the peripheral agents are rebels and one half of them are conformists. The proof is quite similar to that of case (iii) and thus omitted. $\square$

1.4 Proof of Theorem 6

Proof

• (a) Suppose now the underlying network is a line or a ring, then the probability that it is strongly clustered goes to 1, as n goes to infinity. This is true because the probability that there are three or more adjacent agents that are of the same type goes to 1, as n tends to infinity. Since $u_j({\mathbf {x}}(0))<0$ for all $j\in N$ can only happen when all agents are of the same type, an event that occurs with a probability $c^n+(1-c)^n\rightarrow 0$, it follows from Theorem 3(b)(iii) and Theorem 4(iii) that Theorem 6(a) is correct.

• (b) Suppose now the underlying network is a star. Since the probability that there are equal numbers of conformists and rebels in the peripheral agents goes to 0, as n goes to infinity, it follows from Theorem 5 that the first formula is valid.

Observe that $\underline{c}+\overline{c}(c\underline{b}+(1-c)\overline{b})+\overline{c}(c\overline{b}+(1-c)\underline{b})=1$ for all $b,c\in (0,1)$. Therefore, it suffices to show that the second and the third formulas are both correct.

Claim 2 The probability that more than one half of the peripheral agents are of the same type as the central agent tends to $\overline{c},$ as n goes to infinity.

Observe that the probability that there are equal numbers of conformists and rebels in the peripheral agents approaches 0, as n goes to infinity. To prove Claim 2, we discuss in three cases. (i) When $c=0.5$, it follows by symmetry that Claim 2 is correct. (ii) When $c>0.5$, using the Law of Large Numbers, the probability that there are more conformist peripheral agents than rebel ones approaches 1, as n goes to infinity. Thus, the probability that more than one half of the peripheral agents are of the same type as the central agent tends to the probability that the central agent is a conformist, which is $c=\overline{c}$. (iii) When $c<0.5$, using the Law of Large Numbers again, it can be seen that the probability that more than one half of the peripheral agents are of the same type as the central agent tends to the probability that the central agent is a rebel, which is $1-c=\overline{c}$. Due to the above discussions, we conclude that Claim 2 is valid.

Claim 3 When the central agent is a conformist,  the probability that she is initially unsatisfied tends to $\underline{b},$ as n goes to infinity.

Observe first that the probability that there are equal numbers of peripheral agents who choose 0 and those who choose 1 goes to 0, as n goes to infinity. We still discuss in three cases. (i) When $b=0.5$, Claim 3 is true by symmetry. (ii) When $b>0.5$, the probability that more than one half of the peripheral agents initially choose action 0 approaches 1. Therefore, the probability that the central agent is initially unsatisfied approaches the probability that she initially chooses action 1, which is $1-b=\underline{b}$, because the central agent is a conformist. (iii) When $b>0.5$, the probability that more than one half of the peripheral agents initially choose action 1 approaches 1. Therefore, the probability that the central agent is initially unsatisfied approaches the probability that she initially chooses action 0, which is $b=\underline{b}$. It follows from the previous discussions that Claim 3 is true.

Claim 4 When the central agent is a rebel,  the probability that she is initially unsatisfied tends to $\overline{b},$ as n goes to infinity.

The proof of Claim 4 is quite similar to that of Claim 3 and thus omitted.

Claim 2 implies that the probability that more than one half of the peripheral agents are of the opposite type to the central agent tends to $1-\overline{c}=\underline{c}$, as n goes to infinity. Combining with Theorem 5(a), it is clear that the second formula in Theorem 6(b) is valid. We are left to show the third formula in Theorem 6(a). By Claim 3 and Claim 4, the probability that the central agent is initially unsatisfied tends to $c\underline{b}+(1-c)\overline{b}$. Combining this with Theorem 5(b) and Claim 2, it can be seen that the third formula in Theorem 6(b) is valid. This finishes the proof. $\square$

Appendix C: Sequential BRD

In this appendix, we discuss properties of the sequential BRD with arbitrary deviation orders. Given an initialized NMP, at each time step, as long as PNE is not reached, we let an arbitrary unsatisfied agent switch her action. We say that the sequential BRD converges if and only if a PNE is reached for any deviation order. It is well known that the sequential BRD converges if and only if the corresponding game possesses an ordinal potential function (Monderer and Shapley 1996). While Zhang et al. (2018) discuss the two benchmark cases, where an NMP is either completely homophilic or completely heterophilic, we consider the line and the ring network structures in this section. We use a slightly weaker convergence concept, to be introduced later, for NMPs with a star network.

1.1 Lines and rings

Observe first that an NMP, in general, is not a potential game for the line or ring networks, because a PNE is not guaranteed in the two cases, while a potential game always possesses a PNE (Monderer and Shapley 1996). However, for a very wide class of situations, it is indeed the case. Lemma 1(a) states that an NMP is an exact potential game if and only if there is no conformist-rebel link, i.e., conformists only interact with conformists and rebels only interact with rebels. The following theorem concerns with ordinal potentials for NMPs with line or ring structures.

Theorem 7

Let $G=(N,\mathrm{g},{\mathbf {T}})$ be an NMP. Suppose the underlying network $(N,\mathrm{g})$ is a line or a ring. The sequential BRD converges for G,  i.e.,  G is an ordinal potential game,  if and only if it is weakly clustered.

Proof

Suppose now the underlying network is a line. Due to Lemma 1(a), the “only if” part of Theorem 7 is straightforward. We show next the “if” part. That the line is not completely heterophilic implies that it is weakly clustered, i.e., there exist two agents, say i and $i+1$, who are adjacent and are of the same type. Suppose on the contrary that G is not an ordinal potential game. Then, there will be an infinite number of deviations from some initial action profile (for some particular deviating order). Since i and $i+1$ are of the same type, at most one of them ever deviates, and deviates for at most once. When neither of i and $i+1$ deviates, $i-1$ and $i+2$, the two neighbors of i and $i+1$, both deviate for at most once. When one of i and $i+1$ deviates, say at time $t_1$, then after $t_1$, $i-1$ and $i+2$ both deviate for at most once. Using the same logic, there exists some time $t_2$ such that after $t_2$ both $i-2$ and $i+3$, the two neighbors of $i-1$ and $i+2$, deviate for at most once. Using this logic repeatedly, we will get that there are a finite number of deviations in total, a contradiction. Thus the “if” part of Theorem 7 in this case is also true.

Suppose now the underlying network is a ring. The “if” part of Theorem 7 in this case can be shown to be correct in precisely the same way as for the line network. To verify the “only if” part, suppose now the underlying ring is completely heterophilic and all agents take the same initial action. Then the conformists are all satisfied and the rebels are all unsatisfied. We let the rebels deviate one by one, during which the payoffs of the conformists keep decreasing. After the last deviation of the rebels, all of them are satisfied, but all of the conformists are unsatisfied. Now it is the conformists’ turn to deviate one by one. When the conformists are satisfied, all of the rebels will be unsatisfied again. Hence under the above particular deviation orders, this process will be infinitely repeated. It follows that the sequential BRD does not converge and the game is not an ordinal potential game. This completes the whole proof.$\square$

When the underlying network $(N,\mathrm{g})$ is a line or a ring, it is very likely to have at least one weak cluster. Similar to Theorem 6(a), we have

$$\begin{aligned} \lim _{n\rightarrow \infty } \text{ Prob }_{c} \left[ (N,\mathrm{g},\mathbb {T}) \text{ is } \text{ an } \text{ ordinal } \text{ potential } \text{ game }\right] =1. \end{aligned}$$

We also have the following nice characterization for the sequential BRD on lines. Let $G=(N,\mathrm{g},{\mathbf {T}})$ be an NMP with the underlying network $(N,\mathrm{g})$ being a line, then the following statements are equivalent: (i) a PNE exists for G; (ii) G is weakly clustered; (iii) the sequential BRD converges for all initial action profiles; and (iv) the sequential BRD converges for some initial action profile. We do not have such a nice characterization when the network is a ring.

1.2 Stars

Observe first that, when the underlying network is a star, an NMP may not be an ordinal potential game, even if a PNE exists. To see this, suppose that the central agent is a conformist, and there are two peripheral conformists and one peripheral rebel. This game has a PNE (e.g., all the conformists choose action 0 and the rebel chooses action 1). Suppose the central conformist initially takes action 1, the rebel takes action 0, one peripheral conformist takes action 0 and the other peripheral conformist takes action 1. We let the rebel have a priority in deviating over the peripheral conformists. Then, it can be seen that the sequential BRD will never converge. Indeed, the central conformist and the peripheral rebel will alternately deviate for ever, like in matching pennies.

Yet, if we slightly weaken the convergence concept, then a result that is similar to Theorem 7 still holds for star networks.

Definition 8

Suppose that we are given an initialized NMP. A (stochastic) sequential BRD is called regular, if in each round, as long as a PNE is not reached, exactly one unsatisfied agent is selected to switch, such that (i) the probability that each unsatisfied agent is selected is positive, and (ii) this stochastic process is Markovian, i.e., the probability that each unsatisfied agent is selected is independent of the history.

Theorem 8

Let $G=(N,\mathrm{g},{\mathbf {T}})$ be an NMP. If the underlying network $(N,\mathrm{g})$ is a star,  then each regular sequential BRD converges with probability one for all initial action profiles if and only if at least one half of the peripheral agents are of the same type as the central agent.

Proof

Since G has a PNE if and only if at least one half of the peripheral agents are of the same type as the central agent (Lemma 1(e)), the “only if” part of Theorem 8 is immediate. The “if” part is a direct application of absorbing Markov chains, as demonstrated below.

Suppose now at least one half of the peripheral agents are of the same type as the central agent. Then at least one PNE exists. In fact, it can be seen that there are exactly two PNEs. Taking the action profiles as states, and letting the transition probabilities be determined by the underlying dynamic, we have a well-defined (not necessarily time-homogeneous) Markov chain. The two PNEs correspond to the only absorbing states of the Markov chain. It remains to show that this Markov chain is absorbing, i.e., every state can reach one of the two absorbing states. This follows from the observation below: if we let the unsatisfied peripheral agents that are of the same type as the central agent deviate first (one by one in an arbitrary order), and the other unsatisfied peripheral agents deviate subsequently (one by one in an arbitrary order), then we will arrive at one of the two PNEs. Since the non-absorbing states are transient in absorbing Markov chains, we know immediately that a PNE will be reached with probability one. This completes the proof. $\square$

Given a star network $(N,\mathrm{g})$, let each agent uniformly choose a type, then the probability that at least one half of the peripheral agents are of the same type as the central agent tends to 0.5, as n goes to infinity. Combining this fact with Theorem 8, it can be seen that when the underlying network is a star and each agent is assigned as a conformist with probability 0.5, the probability that the sequential BRD converges almost surely for all initial action profiles approaches 0.5, as n goes to infinity.

We also have the following nice characterization for the sequential BRD on star networks. Let $G=(N,\mathrm{g},{\mathbf {T}})$ be an NMP with the underlying network $(N,\mathrm{g})$ being a star, then the following statements are equivalent: (i) a PNE exists for G; (ii) at least one half of the peripheral agents are of the same type as the central agent; (iii) the sequential BRD converges almost surely for all initial action profiles; and (iv) the sequential BRD converges almost surely for some initial action profile.

1.3 Simultaneous BRD V.S. sequential BRD

Let’s now make some comparisons between the simultaneous BRD and the sequential BRD. Generally speaking, the sequential BRD has been studied much more extensively than the simultaneous BRD, and it has been widely recognized that the simultaneous BRD is in general more complicated than the sequential BRD. This contrast is also reflected in this paper: while we have provided quite complete characterizations for the sequential BRD of NMP, it is not the case for the simultaneous BRD.

We can also argue that convergence of the simultaneous BRD is stronger than convergence of the sequential BRD in some sense, when the underlying network is a line or a star. (i) Given an NMP with the underlying network being a line, if the simultaneous BRD converges for some initial action profile, then the sequential BRD converges for all initial action profiles. This is true because the former condition implies that a PNE exists, which is equivalent to the latter condition. (ii) Suppose now the underlying network is a star. Similarly, if the simultaneous BRD converges for some initial action profile, then the sequential BRD converges almost surely for all initial action profiles.