Information spillover in multiple zero-sum games

Abstract

This paper considers an infinitely repeated three-player zero-sum game with two-sided incomplete information, in which an informed player plays two zero-sum games simultaneously at each stage against two uninformed players. This is a generalization of the model in Aumann et al. (Repeated games with incomplete information. MIT Press, New York, 1995) of two-player zero-sum one-sided incomplete information games. Under a correlated prior, the informed player faces the problem of how to optimally disclose information among two uninformed players in order to maximize his long-term average payoffs (i.e., undiscounted payoffs). Our objective is to understand the adverse effects of “information spillover” from one game to the other in the equilibrium payoff set of the informed player. We provide conditions under which the informed player can fully overcome such adverse effects and characterize equilibrium payoffs. In a second result, we show how the effects of information spillover on the equilibrium payoff set of the informed player might be severe. Finally, we compare our findings on the equilibrium-payoff set of the informed player with those of Bayesian Persuasion models with multiple receivers.

1 Introduction

In their seminal work, Aumann et al. (1995) analyzed an undiscounted infinitely repeated game with one-sided incomplete information: one player (the informed) knows the stage game being played whereas the other (the uninformed) does not know and cannot observe payoffs, only actions. They showed that this game has a value and constructed optimal strategies for the players. Matters are more complicated if the informed player were to play against more than one uninformed player, as it would be the case of a military power (e.g., USA) negotiating with two different countries (e.g., Russia and Iran).¹ By observing what the informed player plays against some other uninformed player, an uninformed player can make inferences about the game he plays against the informed player. As a consequence, it may not be optimal for the informed player to play his unilaterally optimal strategy against some of the uninformed players. Put differently, the information spillover among the games played between the informed player and the uninformed players adds layers of complexity to the analysis.

We consider a three-player undiscounted infinitely repeated game in which one of the players is informed of the two zero-sum stage games that he plays against each of the other two (uninformed) players. Each uninformed player only knows the prior probability distribution over the finite set of pairs of zero-sum finite-action stage games, and during the play of the game observes the profiles of actions (but not the payoffs). The informed player collects the sum of payoffs from the two component games.

In the absence of information spillover, for instance when each uninformed player cannnot observe the actions played in the other zero-sum game, our three-player game has a single expected payoff, namely, the sum of values of each of the two-player component games. However, when all players are able to observe the actions played across each zero-sum game, the information spillover kicks in and it is in principle unclear whether the informed player can attain the sum of values in equilibrium. This sum of values can actually be shown to be an upper bound on the equilibrium payoffs of the informed player in our three-player game.

Our first main result has two parts which together provide a condition under which the informed player can attain this upper bound in equilibrium, even in the presence of information spillover. More precisely, under such condition, we show the informed player can attain anything as an equilibrium payoff from his individually rational payoff to the above mentioned upper bound, thus characterizing the set of equilibrium payoffs in the three-player game. In particular, this result implies that the three-player model we analyse might have a continuum of equilibrium payoffs, even though it is a zero-sum model.

The method used to obtain this first result is also of interest to the model of two-player games studied by Aumann et al. (1995). Under a sufficient condition on the stage payoffs, we show that different optimal strategies from those constructed by Aumann et al. exist.² The strategy of the informed player, in particular, does not involve any signalling on path of play, even when the standard optimal strategy constructed by Aumann et al. necessarily does.

In a second result we provide a necessary condition for the existence of equilibria yielding the upper bound to the informed player. We explore two consequences of this result. First, we show that a natural class of equilibria which involve signalling on equilibrium path never pays the upper bound to the informed player. Second, we present an example showing that the effects of information spillover might be very severe, in the sense that the informed player is not able to attain the upper bound in equilibrium.

Given the recent heightened interest in Bayesian Persuasion (BP) (since Gentzkow and Kamenica 2011’s seminal contribution) and its overtones with Aumann et al., it is interesting to compare the effects of information spillover in a multiple-receiver setting in BP with the results of this paper. We consider a model with one Sender and two receivers under two alternative specifications: (i) the Sender can send private messages to each Receiver and (ii) the Sender can only send public messages. We show that the difference in the sender’s payoff from (i) to (ii) can be interpreted as a loss due to information spillover, but a similar intuition cannot be extended to our repeated-game model.

1.1 Related literature

To the best of our knowledge, the model analyzed here is new. Although the model we analyse is zero-sum, the results and the techniques presented remain closer to the non-zero-sum literature, especially to Hart (1985) and Sorin (1983). We highlight here a few additional papers on discounted and undiscounted repeated games with incomplete information that have technical and thematic similarities to this project. A significant part of the literature on undiscounted repeated games with incomplete information analyses models under the assumption of “known own payoffs” (see Forges 1992). This is a reasonable assumption in several applications and allows for equilibrium-payoff characterizations which are especially tractable (see Shalev 1988). Under this assumption, Forges and Solomon (2015) provide a simple characterization of Nash equilibrium payoffs in undiscounted repeated games of two players and incomplete information.³ This characterization is used to show that in a class of public good games, uniform equilibria might not exist. More closely related to our paper in terms of the information environment is Forges et al. (2016). In this paper, among other results, cooperative solutions of one-shot games with two players and exactly one informed player are related to noncooperative solutions of two-player repeated games with exactly one informed player. More specifically, under the assumption of existence of uniform punishment strategies for the uninformed player, the joint plan equilibrium payoffs of the repeated game equal the set of cooperative solutions of the one-shot game. This folk theorem is not however an equilibrium payoff characterization, since it is known from Hart (1985) that joint plans cannot account for the whole of equilibrium payoffs in general.

Our work was inspired by Huangfu and Liu (2021) who considered the issue of information spillovers between markets. In their model, a seller holds private information about the quality of goods he sells in two different markets and buyers learn about the seller’s private information from observing past trading outcomes not only in the market in which they directly participate, but also from observing the outcomes of the other market. The authors show that, under certain assumptions on the correlation of qualities between goods in different markets, information spillover mitigates the negative effects of adverse selection.

The literature on Bayesian Persuasion with multiple receivers is extensive. We highlight here a few papers that connect to our discussion in the last section. Wang (2013) presents a Bayesian Persuasion model with multiple Receivers whose payoffs depend not only on the unobserved state drawn by Nature but also on the action choices of the other Receivers. Concretely, the Receivers vote on the outcome after receiving a signal about the underlying state. The paper compares two environments: one in which all Receivers observe a public message drawn from the experiment chosen by the Sender, and another in which each Receiver observes a private message independently drawn from the same experiment set out by the Sender. The main difference with the models we analyse in our last section is the payoff-interdependencies of the receivers: in our models Receivers do not care about each other’s action choices, but only about their own. This implies that there is in effect no game among Receivers in our model, only a decision problem. The payoff interdependencies among Receivers coupled with the assumption of the independence of the messages drawn from the Sender’s experiment drive the result of that paper in which the best equilibrium payoffs for the Sender are higher in the public compared to the private message environment, a result that is the opposite of what we find.

Arieli and Babichenko (2019) are interested in what happens with optimal experiments under different assumptions for the utility of the Sender as well as payoff interdependencies of the Receivers. One result is particularly reminiscent of the one we obtain in our analysis: a specification of the public Bayesian Persuasion model we define in our paper (say, with two perfectly correlated states for each Receiver and a well-chosen utility function for the Sender) yields (Arieli and Babichenko 2019)’s model, where the utility of the Sender is additive over the Receivers, and therefore supermodular in particular. Theorem 3 in that paper then applies and a public signaling experiment is optimal if and only if all Receivers have the same “persuasion level”, i.e., are essentially identical from the point of view of the optimal persuasion policy. Therefore, if Receivers meaningfully differ, the optimal experiment must send private messages. In our words, this result could be read as saying that unless information spillover is not meaningful (i.e., Receivers have identical persuasion levels), public Bayesian Persuasion yields a lower equilibrium payoff  to the Sender compared to private Bayesian Persuasion. Finally, Koessler et al. (2022) generalize the standard Bayesian Persuasion model in a number of directions, most importantly by considering multiple Senders and multiple Receivers.

1.2 Organization

The remainder of the paper is organized as follows. Section 2 presents the model. Our first main result is divided in 2 parts: main result 1 and main result 2. Section 3 presents our main result 1. Section 4 presents our main result 2. Section 5 compares the equilibrium payoff set in our model with that of a Bayesian Persuasion model with multiple receivers and Sect. 6 concludes. The proofs of technical results are left to the Appendix. Additional results can be found in a Supplemental Appendix.⁴

2 Model and equilibrium concept

2.1 Notation

Given a finite set K, $\Delta (K)$ is the set of probability distributions over K; given a topological space X the interior of X will be denoted by $\text {int}(X)$. If X is a subspace, its boundary will be denoted by $\partial (X)$. For a set $Y \subseteq \mathbb {R}^{m}$, its convex closure is denoted by $\text {co}(Y)$. For $p \in \Delta (K_A \times K_B)$, $p_A$ (resp. $p_B$) denotes its marginal on $K_A$ (resp. $K_B$), and $\text {supp}(p)$ its support. We denote a product distribution on $K_A \times K_B$ by $p_A\bigotimes p_B$, and use $\Delta (K_A) \bigotimes \Delta (K_B)$ to denote the set of all such distributions.

2.2 Model

A three-player infinitely repeated zero-sum game with two-sided incomplete information, denoted $\mathcal {G}(p^0)$, is given by the following data:

• Three players, namely player 1 (the informed player), player 2 and player 3 (the uninformed players).

• Finite sets: $I_i$, $J_i$, $K_i$, $i=A,B$ with $I_A\times I_B$ (resp. $J_A$ and $J_B$) being the set of actions of player 1 (resp. players 2 and 3), and $K_A\times K_B$ being the set of states.

• $p^0 \in \Delta (K_A \times K_B)$ is the prior.

• For each $k_A \in K_A$ and $k_B \in K_B$, $A^{k_A}$ and $B^{k_B}$ matrices of dimensions $|I_A| \times |J_A|$ and $|I_B| \times |J_B|$ respectively. $A^{k_A}$ and $B^{k_B}$ are the stage-game payoff matrices.

The play of the infinitely repeated game is as follows:

• At stage 0, state $(k_A, k_B) \in K_A \times K_B$ is drawn according to distribution $p^0$ and only player 1 knows the draw.

• At each stage $t=1,2,...$, the players independently choose an action in their own set of actions: player 1 chooses $(i^t_A, i^t_B) \in I_A \times I_B$ and players 2 and 3 choose $j^t_A \in J_A$ and $j^t_B \in J_B$, respectively. The stage payoff to player 1 is then $A^{k_A}_{i^t_A,j^t_A} + B^{k_B}_{i^t_B, j^t_B}$; to player 2, $-A^{k_A}_{i^t_A, j^t_A}$ and to player 3, $-B^{k_B}_{i^t_B, j^t_B}$. Monitoring is perfect, i.e., the chosen actions are observed by all players before starting stage $t+1$. Realized payoffs are not observed by the players (though player 1 knows them, since he is fully informed).

Players are assumed to have perfect recall and the whole description of the game is common knowledge.

A behavior strategy for player 1 is a tuple $\sigma = (\sigma ^{(k_A,k_B)})_{(k_A,k_B) \in K_A \times K_B}$, where for each $(k_A,k_B) \in K_A \times K_B$, $\sigma ^{(k_A,k_B)} = (\sigma ^{(k_A,k_B)}_t)_{t \ge 1}$ and $\sigma ^{(k_A,k_B)}_t$ is a mapping from the Cartesian product $H_t:= (I_A \times J_A \times I_B \times J_B)^{t-1}$ (with $H_0:= \{\emptyset \}$) to $\Delta (I_A \times I_B)$, giving the lottery on actions played by player 1 at a stage t, when the state is $(k_A, k_B)$. Because players 2 and 3 do not know the state, a behavior strategy for player 2 (resp. player 3) is an element $\tau _A = (\tau _{A,t})_{t \ge 1}$ (resp. $\tau _B = (\tau _{B,t})_{t \ge 1}$), where $\tau _{A,t}$ (resp. $\tau _{B,t}$) is a mapping from $H_t$ to $\Delta (J_A)$ (resp. $\Delta (J_B))$, giving the lottery on actions to be played by player 2 (resp. player 3) on stage t. The set of behavior strategies of player 1 is denoted by $\Sigma$; for player 2, it is denoted by $\mathcal {T}_A$ and for player 3, it is denoted by $\mathcal {T}_B$.

A behavior strategy profile $(\sigma , \tau _A, \tau _B)$ induces, for every state $(k_A, k_B)$ and stage $T>0$, a probability distribution on $H_{T+1}$. Also, $(\sigma , \tau _A, \tau _B)$ and $p^0$ induce a probability distribution over $K_A \times K_B \times H_{T+1}$. We can thus define the expected average payoffs (with $\kappa$ being a random variable taking values on $K_A \times K_B$ distributed according to $p^0$, $\kappa _A$ the random variable obtained from projecting $\kappa$ on $K_A$ and $\kappa _B$ the random variable obtained from projecting $\kappa$ on $K_B$):

$$\begin{aligned}{} & {} \alpha ^{k_A, k_B}_T = \alpha ^{k_A, k_B}_T (\sigma ,\tau _A,\tau _B):= \mathbb {E}_{\sigma ^{(k_A, k_B)},\tau _A, \tau _B}\Big [\frac{1}{T} \sum ^{T}_{t=1}(A^{k_A}_{i^t_A,j^t_A} + B^{k_B}_{i^t_B, j^t_B})\Big ], \\{} & {} \quad \beta ^{A}_T(\sigma , \tau _A, \tau _B):= \mathbb {E}_{\sigma ,\tau _A, \tau _B,p^0}\Big [\frac{1}{T} \sum ^{T}_{t=1}(-A^{\kappa _A}_{i^t_A, j^t_A})\Big ],\\{} & {} \qquad \beta ^{B}_T(\sigma , \tau _A, \tau _B): = \mathbb {E}_{\sigma ,\tau _A, \tau _B,p^0}\Big [\frac{1}{T} \sum ^{T}_{t=1}(-B^{\kappa _B}_{i^t_B, j^t_B})\Big ]. \end{aligned}$$

The number $\alpha ^{k_A, k_B}_T (\sigma ,\tau _A,\tau _B)$ is the expected average payoff (up to time T) of player 1; $\beta ^{A}_T(\sigma , \tau _A, \tau _B)$ is the expected average payoff (up to time T) of player 2 and $\beta ^{B}_T(\sigma , \tau _A, \tau _B)$ is the expected average payoff (up to time T) of player 3.

The model defined is therefore a “combination” of two zero-sum games: at each stage, player 1 plays simultaneously a zero-sum game against player 2 and another zero-sum game against player 3, collecting the sum of the payoffs of each of these games; players 2 and 3 are the minimizers in each of the zero-sum games they play against player 1. One distinctive and important aspect of this model is the fact that each uninformed player can observe not only the actions played in his own zero-sum game, but also the actions played in the other zero-sum game.

2.3 Equilibrium concept

A profile $(\sigma , \tau _{A}, \tau _{B})$ is a uniform equilibrium of $\mathcal {G}(p^0)$ when:

• (1) For each $(k_A, k_B) \in \text {supp}(p^0)$,⁵$(\alpha ^{k_A, k_B}_T (\sigma ,\tau _A,\tau _B))_{T \ge 1}$ converges as T goes to infinity to some $\alpha ^{k_A, k_B}(\sigma ,\tau _A,\tau _B)$, $(\beta ^{A}_T(\sigma , \tau _A, \tau _B))_{T \ge 1}$ converges to some $\beta ^A(\sigma , \tau _A, \tau _B)$ and $(\beta ^{B}_T(\sigma , \tau _A, \tau _B))_{T \ge 1}$ converges to some $\beta ^B(\sigma , \tau _A, \tau _B)$.

• (2) For each $\epsilon >0$, there exists a positive integer $T_0$ such that for all $T \ge T_0$, $(\sigma ,\tau _A, \tau _B)$ is an $\epsilon$-Nash equilibrium in the finitely repeated game with T stages, i.e.,

  • (a) For each $(k_A, k_B) \in \text {supp}(p^0)$ and $\sigma ' \in \Sigma$, $\alpha ^{k_A, k_B}_T(\sigma ', \tau _A, \tau _B) \le \alpha ^{k_A,k_B}_T(\sigma , \tau _A, \tau _B) + \epsilon$;

  • (b) For each $\tau _A' \in \mathcal {T}_A$, $\beta ^{A}_T(\sigma ,\tau _A', \tau _B) \le \beta ^{A}_T(\sigma ,\tau _A, \tau _B) + \epsilon$;

  • (c) For each $\tau _B' \in \mathcal {T}_B$, $\beta ^{B}_T(\sigma ,\tau _A, \tau _B') \le \beta ^{B}_T(\sigma ,\tau _A, \tau _B) + \epsilon$.

Uniform equilibrium is a standard equilibrium concept for the analysis of undiscounted repeated games. It contains a strong requirement, namely (2), which posits that the profile $(\sigma ,\tau _A,\tau _B)$ must generate an $\epsilon$-Nash equilibrium on the finite but sufficiently long-horizon ($T \ge T_0$, where $T < \infty$) version of our model.⁶

Unless explicitly stated otherwise, from now on whenever we refer to uniform equilibrium or equilibria we will use simply equilibrium or equilibria.

If $(\sigma , \tau _A, \tau _B)$ is an equilibrium in $\mathcal {G}(p^0)$, the associated vector

$$\begin{aligned} (\alpha (\sigma , \tau _A, \tau _B), \beta ^A(\sigma , \tau _A, \tau _B), \beta ^B(\sigma , \tau _A, \tau _B)), \end{aligned}$$

where $\alpha (\sigma ,\tau _A, \tau _B):= (\alpha ^{k_A, k_B}(\sigma , \tau _A, \tau _B))_{(k_A,k_B) \in \text {supp}(p^0)}$, is the vector of payoffs of $(\sigma , \tau _A, \tau _B)$. Also $\alpha (\sigma ,\tau _A, \tau _B) \cdot p^0$ (where $\cdot$ is the standard scalar product in Euclidean space) is the ex-ante equilibrium payoff of the informed player.

2.4 Preliminaries on the Aumann et al. model

Our analysis of the equilibrium payoff set of the game $\mathcal {G}(p^0)$ in the next section will rely on certain properties of each of the two-player, infinitely repeated zero-sum games that the informed player plays against each uninformed player. For this reason we now recall some of the main results in Aumann et al. (1995), which is the original reference for this two-player model. Let K be the finite set of states and $M = (M^k)_{k \in K}$ a collection of zero-sum payoff matrices where $M^{k} \in \mathbb {R}^{I \times J}$ for each $k \in K$. Denote by $G_M(p)$ the infinitely repeated, two-player, zero-sum game with one-sided incomplete information with prior $p \in \Delta (K)$ and undiscounted payoffs (see Sorin 2002, Chapter 3, for a detailed description of this model or Aumann et al. 1995). Let $M(p) = \sum _{k \in K} p^k M^k$ and define $v_{M}(p) = \text {min}_{t \in \Delta (J)}\text {max}_{s \in \Delta (I)}s M(p) t = \text {max}_{s \in \Delta (I)}\text {min}_{t \in \Delta (J)}s M(p) t$, where s is a row vector and t a column vector. The function $q \in \Delta (K) \mapsto v_{M}(q) \in \mathbb {R}$ is called the non-revealing value function. Let $\text {Cav}(v_M)$ be the (pointwise) smallest concave function g from $\Delta (K)$ to $\mathbb {R}$ such that $g(q) \ge v_M(q)$ for all $q \in \Delta (K)$. Alternatively, one can define $\text {Cav}(v_M)(q):= \text {sup} \{\sum ^k_{i=1}\alpha _i v_M(q_i) | \exists k \in \mathbb {N}, \forall i \in \{1,...,k\}, \alpha _i \ge 0, \sum ^k_{i=1}\alpha _i q_i = q, \sum ^k_{i=1}\alpha _i =1\}$. Aumann et al. (1995) proved that a (uniform) value of $G_M(p)$ exists and equals $\text {Cav}(v_M)(p)$. They also showed how to construct (uniformily) optimal strategies for both players.

Given the model $\mathcal {G}(p^0)$, the infinitely repeated, two-player, zero-sum game with one-sided incomplete information with prior $p^0_A$ defined by states $K_A$ and payoff matrices $(A^{k_A})_{k_A \in K_A}$ with undiscounted payoffs will be denoted $G_A(p^0_A)$—this game is played by players 1 (informed) and 2 (uninformed). Analogously, we define $G_B(p^0_B)$ as the two-player, infinitely repeated, zero-sum game with one-sided incomplete information played by players 1 and 3. The two-player, infinitely repeated zero-sum game with one-sided incomplete information and prior $p^0 \in \Delta (K_A \times K_B)$ with undiscounted payoffs, where stage payoff matrices are $(C^{k_A, k_B})_{k_A \in K_A, k_B \in K_B}$ given by $C^{k_A, k_B}_{i_A,i_B,j_A,j_B}:= A^{k_A}_{i_A,j_A} + B^{k_B}_{i_B, j_B}$ will be denoted $G_{A+B}(p^0)$. This two-player game will be used as an auxiliary game to construct strategies in the three-player game $\mathcal {G}(p^0)$.

2.5 Example

We would like to illustrate the new strategic difficulties that arise in the model $\mathcal {G}(p^0)$ in comparison to the two-player zero-sum model of Aumann et al. (1995). Specifically, we would like to show that in this example that if the informed player plays the optimal strategies constructed by Aumann et al. in each game $G_A(p^0_A)$ and $G_B(p^0_B)$, he cannot guarantee the ex-ante expected payoff equal to the sum of values Cav$(v_A)(p^0_A)$ + Cav$(v_B)(p^0_B)$.

Two sets $A = \{A^1,A^2\}$ and $B = \{B^1,B^2\}$ of payoff matrices are defined below together with $p^0 \in \Delta (K_A \times K_B)$, where $K_A = \{1,2\}$ and $K_B = \{1,2\}$.

$$\begin{aligned}{} & {} p^0 = \begin{bmatrix} 1/2 &{} 0 \\ 0 &{} 1/2 \end{bmatrix} \\{} & {} A^1 = \begin{bmatrix} 1 &{} 0 \\ 0 &{} 0 \end{bmatrix} \,\, A^2 = \begin{bmatrix} 0 &{} 0 \\ 0 &{} 1 \end{bmatrix} \\{} & {} B^1 = \begin{bmatrix} 4 &{} 0 &{} 2 \\ 4 &{} 0 &{} -2 \end{bmatrix} \,\,B^2 = \begin{bmatrix}0 &{} 4 &{} -2 \\ 0 &{} 4 &{} 2 \end{bmatrix} \end{aligned}$$

In the matrix $p^0$, an entry $p^0_{ij}$ corresponds to the probability with which Nature chooses $A^{i}$ and $B^{j}$. So, the stage-payoffs in the two zero-sum games are given by $A^1$ and $B^1$ with probability 1/2, and they are given by $A^2$ and $B^2$ with probability 1/2. Since the prior assigns perfect correlation between states, there are only two states to consider effectively: states (1, 1) and (2, 2).

In Figs. 1 and 2, q denotes the probability of state (1, 1) and $1-q$ the probability of state (2, 2). Each row of $A^i$ and $B^j, i,j \in \{1,2\}$ corresponds to a stage-game action of the informed player: call the first row “U” and the second row “D”. By computation, we get that the non-revealing values are:

$$\begin{aligned}{} & {} v_A(q)=q(1-q)\text { for all }q \in [0,1]\\{} & {} v_B(q) = {\left\{ \begin{array}{ll} 4q &{}\text {if } q \in [0,1/4)\\ -4q +2 &{}\text {if } q \in [1/4,1/2)\\ 4q -2 &{}\text {if } q \in [1/2,3/4)\\ -4q + 4 &{}\text {if } q \in [3/4,1] \end{array}\right. } \end{aligned}$$

These imply that the concavification of these values are:

$$\begin{aligned}{} & {} \text {Cav}(v_A)(q)= v_A(q)=q(1-q)\text { for all }q \in [0,1]\\{} & {} \text {Cav}(v_B)(q) = {\left\{ \begin{array}{ll} 4q &{} \text {if } q \in [0,1/4)\\ 1 &{} \text {if } q \in [1/4,3/4)\\ -4q + 4 &{} \text {if } q \in [3/4,1] \end{array}\right. } \end{aligned}$$

Fig. 1

Graphs of $\text {Cav}(v_A)$ and $v_A$

Fig. 2

Graphs of $\text {Cav}(v_B)$ and $v_B$

We present the optimal strategy of the informed player in the game $G_B(1/2)$. The optimal strategy of the informed player in the two-player repeated zero-sum game $G_B(1/2)$ is defined as follows: in case the state drawn by Nature is 1, the informed player plays “U” with probability 1/4 and after that plays at each stage, independently, the optimal action of the one-shot zero-sum game whose matrix is B(1/4); in case the state drawn by Nature is 2, the informed player plays “U” with probability 3/4 and, after that, plays the optimal action of the one-shot zero-sum game whose matrix is B(3/4).

After observing the realized action of the informed player in the first stage, the uninformed player updates his beliefs about the states to posteriors about states 1 and 2: in our example, the uninformed player, after observing “U”, assigns probability 1/4 to the state being 1. After observing “D”, the uninformed player assigns probability 3/4 to the state being 1. The strategy just described for the informed player is an example of a signalling strategy: the informed player uses his actions to signal information about the underlying state.

After the first stage, according to the construction described, no more information is signaled and the uninformed player plays the optimal action of the one-shot zero-sum game with matrix B(1/4) or B(3/4) forever, depending on whether U or D was realized, respectively. Playing the signalling strategy guarantees to the informed player an ex-ante payoff of $(1/2) v_B(1/4) + (1/2) v_B (3/4) = (1/2) \text {Cav}(v_B)(1/4) + (1/2) \text {Cav}(v_B)(3/4) = \text {Cav}(v_B)(1/2) = 1$.

Now, in game $G_A(1/2)$ the non-revealing value function of the informed player is strictly concave, which implies that his optimal strategy in this game is non-revealing (at any prior): one optimal strategy for the informed player is to play at each stage the optimal action of the one-shot zero-sum game with matrix A(1/2) independently forever, which generates no uptading of the beliefs on the part of the uninformed players.

If the informed player uses the signalling strategy described in $G_B(1/2)$, because of perfect correlation between $\kappa _A$ and $\kappa _B$, this strategy also induces the same updating on the part of the (uninformed) player 2, inducing, similarly, posteriors 1/4 and 3/4 in the two-player zero-sum repeated game $G_A(1/2)$. By using that strategy in game $G_B(1/2)$, the informed player in game $G_A(1/2)$ can now only guarantee $(1/2)\text {Cav}(v_A)(1/4) + (1/2) \text {Cav}(v_A)(3/4) = 3/16 < 1/4 = \text {Cav}(v_A)(1/2)$.

Therefore, if the informed player plays the strategies described above in games $G_A(1/2)$ and $G_B(1/2)$, he cannnot guarantee in $\mathcal {G}(p^0)$ the sum of the uniform values of each zero-sum game i.e., Cav$(v_A)(p^0_A)$ $+$ Cav$(v_B)(p^0_B)$.

3 Main result 1: equilibrium payoff-set characterization

Our first main result (main result 1) has two parts: The first part of main result 1 is Theorem 3.2 which provides a sufficient condition under which the ex-ante equilibrium payoffs of the informed player in $\mathcal {G}(p^0)$ permits a simple characterization. The second part of our main result 1 is Theorem 3.6, which provides a general class of games under which that sufficient condition holds. The two results provide a condition under which $\mathcal {G}(p^0)$ has a continuum of equilibrium payoffs.

This section is subdivided in three subsections. The first of these, Sect. 3.1, is dedicated to Theorem 3.2 and the main ideas of its proof. Section 3.2 is dedicated to Theorem 3.6. Finally, Sect. 3.3 highlights a by-product for the theory of two-player zero-sum games of the equilibrium constructions used in Theorems 3.2 and 3.6.

3.1 First part of main result 1

We now introduce the necessary concepts and state our main result 1 in full generality. Paralleling the definitions of the previous section, we denote the set of histories at stage $t \ge 1$ for a two-player, zero-sum, infinitely repeated, undiscounted game with one-sided incomplete information $G_A(p^0_A)$ by $H^A_t$ with generic element $h^A_t$. The notation for a behavior strategy of the informed player in $G_A(p^0_A)$ is exactly analogous to the one defined for player 1 in the three-player game $\mathcal {G}(p^0)$: $\sigma _A = (\sigma ^{k_A}_A)_{k_A \in K_A}$ and $\sigma ^{k_A}_A = (\sigma ^{k_A}_{At})_{t \ge 1}$, with $\sigma ^{k_A}_{At}: H^A_t \rightarrow \Delta (I_A)$.

Definition 3.1

An equilibrium $(\sigma _A, \tau _A)$ of $G_A(p^0_A)$ is non-revealing if for each $t \ge 1$, $k_A, k'_A \in \text {supp}(p^0_A)$ and $h^A_t \in H^A_t$ played with positive probability by $(\sigma _A, \tau _A)$, we have $\sigma ^{k_A}_{At}(h^A_t) = \sigma ^{k'_A}_{At}(h^A_t)$. Analogously, an equilibrium $(\sigma _B, \tau _B)$ of $G_B(p^0_B)$ is non-revealing if for each $t \ge 1$, $k_B, k'_B \in \text {supp}(p^0_B)$ and $h^B_t \in H^B_t$ played with positive probability by $(\sigma _B, \tau _B)$, we have $\sigma ^{k_B}_{Bt}(h^B_t) = \sigma ^{k'_B}_{Bt}(h^B_t)$.

In non-revealing equilibria the informed player makes no use of his private information on-path of the equilibrium play. Therefore, the uninformed player can infer nothing from the actions played at each stage, which leaves the “posterior” unchanged and equal to the prior.

In $\mathcal {G}(p^0)$, the number $\text {Cav}(v_A)(p^0_A)+ \text {Cav}(v_B)(p^0_B)$ is an upper bound on the ex-ante equilibrium payoffs of player 1, because each uninformed player can always play the optimal strategy of his repeated zero-sum game, holding the payoffs of the informed player at most at $\text {Cav}(v_A)(p_A)+ \text {Cav}(v_B)(p_B)$. On the other hand, letting $\mathfrak {h}(p):= v_A(p_A)+ v_B(p_B)$, a lower bound on the ex-ante equilibrium payoffs of the informed player is given by the concavification of $\mathfrak{h}$ evaluated at $p^0$.⁷

For $p^0 \in \Delta (K_A \times K_B)$, let

$$\begin{aligned} I(p^0) = [\text {Cav}(\mathfrak {h})(p^0), \text {Cav}(v_A)(p^0_A)+\text {Cav}(v_B)(p^0_B)]. \end{aligned}$$

We call $\text {Cav}(\mathfrak {h})(p^0)$ the lower end of $I(p^0)$ and $\text {Cav}(v_A)(p^0_A)+\text {Cav}(v_B)(p^0_B)$ the upper end of $I(p^0)$. The interval $I(p^0)$ might be degenerate as well as non-degenerate. Whenever $I(p^0)$ is non-degenerate, for any sufficiently small perturbation of the stage-game payoff matrices, the resulting model $\mathcal {G}(p^0)$ also has an associated interval $I(p^0)$ which is non-degenerate. A proof of this robustness property can be found in the Supplemental Appendix (Proposition 9.4). As an example, the interval $I(p^0)$ of the game $\mathcal {G}(p^0)$ of Example 2.4 is non-degenerate: we have $\text {Cav}(v_A)(p_A^0) + \text {Cav}(v_B)(p_B^0) = 1/4 + 1 > 1 + 3/16 = \text {Cav}(\mathfrak {h})(p^0)$. There are cases where $I(p^0)$ is degenerate (for example, when $p^0 \in \Delta (K_A) \bigotimes \Delta (K_B)$). In these cases only one ex-ante equilibrium payoff exists (without any assumptions on the games $G_A(p^0_A)$ and $G_B(p^0_B))$. The first part of our main result can now be stated:

Theorem 3.2

Let $p^0 \in \Delta (K_A \times K_B)$. Suppose there exist non-revealing equilibria in $G_A(p^0_A)$ and $G_B(p^0_B)$. Then the set of ex-ante equilibrium payoffs of the informed player is $I(p^0)$.

3.2 On the proof of theorem 3.2

First, the lower end of $I(p^0)$ is always an ex-ante equilibrium payoff for the informed player in $\mathcal {G}(p^0)$—i.e., no additional assumption is required. A proof of this result can be found in the Supplemental Appendix (Proposition 9.11).⁸ When the upper end of $I(p^0)$ is also an ex-ante equilibrium payoff for the informed player, then we can use a straightforward application of jointly controlled lotteries developed in Aumann et al. Aumann et al. (1995) in order to obtain that the whole interval $I(p^0)$ can be attained as an ex-ante equilibrium payoff to the informed player.⁹ Therefore, the only remaining task to obtain a proof of Theorem 3.2 is to show that the upper end of $I(p^0)$ is an ex-ante equilibrium payoff for the informed player.

We define the set of non-revealing equilibrium payoffs (of the informed player), denoted $\mathcal{N}\mathcal{R}(p^0)$, of game $\mathcal {G}(p^0)$. Let $\mathcal{N}\mathcal{R}(p^0)$ be the set of vectors $(\phi _A, \phi _B) \in \mathbb {R}^{|K_A|} \times \mathbb {R}^{|K_B|}$ that satisfy:

• (1) (Feasibility)

  $$\begin{aligned} (\phi _A, \phi _B) \in F_A \times F_B, \end{aligned}$$

  where $F_A:= \text {co} \{ (A^{k_A}_{i_A,j_A})_{k_A \in K_A}| i_A \in I_A, j_A \in J_A\}$ and $F_B:= \text {co} \{(B^{k_B}_{i_B,j_B})_{k_B \in K_B}| i_B \in I_B, j_B \in J_B \}.$

• (2) (Individual rationality for player 1)

  $$\begin{aligned} \phi \cdot q \ge \mathfrak {h}(q), \, \forall q \in \Delta (\text {supp}(p^0)), \end{aligned}$$

  where

  $$\begin{aligned} \phi = (\phi ^{k_A}_A + \phi ^{k_B}_B)_{(k_A, k_B) \in \text {supp}(p^0)}, \, \mathfrak {h}(q) = v_A(q_A) + v_B(q_B). \end{aligned}$$

• (3) (Individual rationality for players 2 and 3)

  $$\begin{aligned} \phi _A \cdot p^0_A \le \text {Cav}(v_A)(p^0_A), \, \phi _B \cdot p^0_B \le \text {Cav}(v_B)(p^0_B). \end{aligned}$$

The three conditions above defining the set of non-revealing equilibrium payoffs parallel the conditions defined for two-player repeated games with a single informed player (see Hart 1985). We briefly recall the reason why $(\phi _A, \phi _B) \in \mathcal{N}\mathcal{R}(p^0)$ implies we can construct equilibria in $\mathcal {G}(p^0)$ that reveal no information on path of play and have $\phi _A$ (resp. $\phi _B$) as the vectors of payoffs of the informed player in $G_A(p^0_A)$ (resp. $G_B(p^0_B)$). Let $(\phi _A, \phi _B) \in \mathcal{N}\mathcal{R}(p^0)$. Since $\phi _A \in F_A$,

$$\begin{aligned} \phi _A \, = \, \sum _{(i_A, j_A) \in I_A \times J_A}\lambda _{i_A, j_A}(A^{k_A}_{i_A,j_A})_{k_A \in K_A}, \end{aligned}$$

where

$$\begin{aligned} \sum _{(i_A, j_A) \in I_A \times J_A}\lambda _{i_A, j_A} =1, \, \lambda _{i_A, j_A}\ge 0. \end{aligned}$$

Consider then a sequence of (pure) actions $((i^t_A, j^t_A))_{t \ge 1}$ and define a function $\chi ^{(i_A,j_A)}: I_A \times J_A \rightarrow \{0,1\}$ where $\chi ^{(i_A,j_A)}(i'_A, j'_A) = 1$, if $(i'_A, j'_A) = (i_A, j_A)$, and $\chi ^{(i_A,j_A)}(i'_A, j'_A) = 0$, if $(i'_A, j'_A) \ne (i_A, j_A)$. Assume that the sequence $h^A_{\infty } = ((i^t_A, j^t_A))_{t \ge 1}$ satisfies for each $(i_A, j_A) \in I_A \times J_A$, $\lim _{T \rightarrow +\infty }\frac{1}{T}\sum ^{T}_{t=1} \chi ^{(i_A, j_A)}(i^t_A, j^t_A) = \lambda _{i_A, j_A}$.¹⁰ When players 1 and 2 play the sequence of actions $h^A_{\infty }$ in $G_A(p^0_A)$, the payoff achieved is $\phi ^{k_A}_A$, for each state $k_A \in K_A$. Obviously, a similar reasoning applies to $\phi _B \in F_B$, and $\phi ^{k_B}_B$ is the payoff achieved by a deterministic sequence of actions played at each stage. Conditions (2) and (3) now imply that this deterministic path of play can be supported as an equilibrium path of play: condition (2) guarantees that, in case player 1 deviates from the deterministic sequence, players 2 and 3 can punish him. So it guarantees that under no possible state $(k_A, k_B)$ player 1 could obtain more that $\phi ^{(k_A k_B)}$¹¹; condition (3) guarantees, in turn, that any deviation of player 2 (resp. player 3) can be punished by player 1 with an optimal strategy of game $G_A(p^0_A$)(resp. $G_B(p^0_B)$).

We now define the set $NR_A(p^0_A)$ of non-revealing equilibrium payoffs of the two-player repeated game $G_A(p^0_A)$. The set $NR_A(p^0_A)$ is the set of vectors $\alpha _A \in \mathbb {R}^{|K_A|}$ that satisfies (i) $\alpha _A \cdot q \ge v_A(q)$, for all $q \in \Delta (K_A)$; (ii) $\alpha _A \cdot p^0_A = \text {Cav}(v_A)(p^0_A)$ and (iii) $\alpha _A \in F_A$. This is the set of equilibrium payoffs for which no signalling occurs on path. This set is essentially the specification for a two-player, zero-sum infinitely repeated game with one-sided incomplete information of the set of non-revealing equilibrium payoffs (called “G”) defined for nonzero-sum two-player repeated games with lack on information on one side in Hart (1985)(see p. 124). Note that each $k_A$-th entry of a vector $\alpha _A$ in $NR_A(p^0_A)$ is the payoff for the informed player in $G_A(p^0_A)$ when the state is $k_A$. All vectors $\alpha _A$ in $NR_A(p^0_A)$ generate the same ex-ante payoff, i.e., the (uniform) value Cav$(v_A)(p^0_A) = \alpha _A \cdot p^0_A$, but there might be several vectors generating this payoff. It is now easy to see that $NR_A(p^0_A) \times NR_B(p^0_B) \subseteq \mathcal{N}\mathcal{R}(p^0)$. Since we assumed that $NR_A(p^0_A) \times NR_B(p^0_B) \ne \emptyset$, Theorem 3.2 now follows immediately.

Remark 3.3

We remark that property (2) of $\mathcal{N}\mathcal{R}(p^0)$ differs from the usual assumption present in the literature, namely, that supp$(p^0) = K_A \times K_B$. The reason why we do not adopt this assumption is as follows: as shown above, $NR_{A}(p^0_A) \times NR_B(p^0_B) \subseteq \mathcal{N}\mathcal{R}(p^0)$ for any $p^0 \in \Delta (K_A \times K_B)$; if $p^0 \in \text {int}(\Delta (K_A \times K_B))$, then we have in addition that $NR_A(p^0_A) \times NR_B(p^0_B) = \mathcal{N}\mathcal{R}(p^0)$: to see this, take $(\phi _A, \phi _B) \in \mathcal{N}\mathcal{R}(p^0)$. To show $(\phi _A, \phi _B)$ is in $NR_A(p_A) \times NR_B(p_B)$, we just have to check that property (i) defining the sets $NR_A(p_A)$ and $NR_B(p_B)$ is satisfied; the other conditions are immediate. Suppose by contradiction there exists ${\bar{q}}_A \in \Delta (K_A)$ such that $\phi _A \cdot {\bar{q}}_A < v_A({\bar{q}}_A)$. Fix now ${\bar{q}}_B \in \Delta (K_B)$ such that $\phi _B \cdot {\bar{q}}_B \le v_B({\bar{q}}_B)$. It follows that for ${\bar{q}} = {\bar{q}}_A \bigotimes {\bar{q}}_B \in \Delta (K_A \times K_B)$, $\phi _A \cdot {\bar{q}}_A + \phi _B \cdot {\bar{q}}_B < v_A({\bar{q}}_A) + v_B({\bar{q}}_B) = \mathfrak {h}({\bar{q}})$, which is a contradiction, since $(\phi _A, \phi _B)$ satisfies Condition (2) of $\mathcal{N}\mathcal{R}(p^0)$. Hence, when $p^0 \in \text {int}(\Delta (K_A \times K_B))$, $(\phi _A, \phi _B) \in \mathcal{N}\mathcal{R}(p^0)$ implies that $\phi _A$ (respec. $\phi _B$) is a vector of equilibrium payoffs in $G_A(p^0_A)$ (respec. $G_B(p^0_B)$) of the informed player.

One can see immediately that the argument above relies on the product structure of the set of states $\text {supp}(p^0) = K_A \times K_B$; if one assumes a prior $p^0$ for which $\text {supp}(p^0)$ is not a cartesian product, then the argument above cannot be repeated and, in fact, the claim is not true.¹² The example below shows that for a certain $p^0 \notin \text {int}(\Delta (K_A \times K_B))$, $NR_{A}(p^0_A) \times NR_B(p^0_B) \subsetneq \mathcal{N}\mathcal{R}(p^0)$.

Example 3.4

We follow the notation of Example 2.4. For $\varepsilon >0$ and $q^0 \in (0,1)$, consider the game $\mathcal {G}(p^0)$ given by the following data:

$$\begin{aligned}{} & {} p^0 = \begin{bmatrix} q^0 &{} 0 \\ 0 &{} (1-q^0) \end{bmatrix} \\{} & {} A^1 = \begin{bmatrix} 1 &{} 0 \\ 0 &{} 0 \end{bmatrix} \,\, A^2 = \begin{bmatrix} 0 &{} 0 \\ 0 &{} 1 \end{bmatrix} \\{} & {} B^1 = \begin{bmatrix} -\varepsilon &{} -\varepsilon \\ \varepsilon &{} \varepsilon \end{bmatrix} \,\,B^2 = \begin{bmatrix} \varepsilon &{} \varepsilon \\ -\varepsilon &{} -\varepsilon \end{bmatrix} \end{aligned}$$

The first thing to observe is that $NR_B(p^0_B) = \emptyset$: notice that $\phi _B \in NR_B(p^0_B)$ iff $\phi _B = (\varepsilon , \varepsilon )$; but $F_B = \text {co} \{(\varepsilon ,-\varepsilon ), (-\varepsilon , \varepsilon )\}$, which clearly does not contain $(\varepsilon ,\varepsilon )$. In particular, we therefore have that $NR_A(p^0_A) \times NR_B(p^0_B) = \emptyset$. One can now show that $\mathcal{N}\mathcal{R}(p^0) \ne \emptyset$. We sketch the proof for completeness: fix $q^0 = \frac{1}{5}$. The game $G_A(p^0_A)$ trivially satisfies the condition NR at $p^0_A$ (see Definition 3.5), because $v_A$ is strictly concave and smooth in the interval (0, 1). Therefore, by Theorem 3.6, there exists $\phi _A \in F_A$ such that $\phi _A \cdot p^0_A = v_A(p^0_A)$ and $\phi _A \cdot q \ge v_A(q), \forall q \in \Delta (K_A)$. For example, one might take $\phi _A = (\frac{16}{25}, \frac{1}{25})$. In $G_B(p^0_B)$ consider now the vector $\phi _B = (-\varepsilon , \varepsilon )$, which is in $F_B$. Taking $\varepsilon >0$ sufficiently small, the vector $\phi \in \mathbb {R}^{K_A \times K_B}$ whose entries are given by $\phi ^{k_A, k_B}:= \phi ^{k_A}_A + \phi ^{k_B}_B, \forall k_A \in K_A, k_B \in K_B$ is in $\mathcal{N}\mathcal{R}(p^0)$.

3.3 Second part of main result 1

The second part of our main result provides a general sufficient condition for the non-emptyness of $NR_A(p^0_A)$ and $NR_B(p^0_B)$. We introduce a few preliminary definitions in order to state the condition.

Recall that the non-revealing value function $v_A(q)$ is defined by $\text {min}_{\tau }\text {max}_{\sigma }\sigma A(q)\tau '$, where $\sigma$ is a row vector and $\tau '$ a column vector of the one-shot, two-player, zero-sum game with payoff matrix A(q). For q in the affine hull $H_A$ of the simplex $\Delta (K_A)$, one can consider the immediate extension of $v_A(q)$ to $H_A$ given by the same min-max formula. Denote this extension by $v^e_A$. As we will need to make considerations about the derivative of $v^e_A$ at points in $\Delta (K_A)$, we will define once and for all a parametrization for the affine space $H_A$. Let $T: \mathbb {R}^{|K_A|-1} \rightarrow H_A$ be defined as follows: Let $e^{|K_A|-1}_i = (0,..,1,...,0) \in \mathbb {R}^{|K_A| -1}$ with 1 in the i-th position. Analogously, let $e^{|K_A|}_i = (0,..,1,...,0) \in \mathbb {R}^{|K_A|}$. Define $T: \mathbb {R} ^{|K_A| -1} \rightarrow H_A \subseteq \mathbb {R}^{|K_A|}$ as the affine transformation that maps $e^{K_A-1}_i \mapsto e^{K_A}_{i+1}$ and $0 \mapsto e^{|K_A|}_{1}$, for $i \in \{1,2,...,|K_A| - 1\}$. Since T is affine, $Tx = Sx + e^{|K_A|}_1$, where S is an injective linear transformation; we will also denote by S the matrix representation of S according to the canonical basis. The function $(v^e_A \circ T): \mathbb {R}^{|K_A|-1} \rightarrow \mathbb {R}$ is a Lipschitz function and therefore is almost everywhere differentiable in $\mathbb {R}^{|K_A|-1}$. The generalized gradient¹³ of $(v^e_A \circ T)$ at $x^0$ is defined as $\partial (v^e_A \circ T)(x^0) = \text {co}\{\text {lim} (\nabla (v^e_A \circ T))(x^0 +h_k) | h_k \rightarrow 0$ as $k \rightarrow +\infty \}$, where $x^0 + h_k \in \mathbb {R}^{|K_A| -1}$ is a point of differentiability of $(v^e_A \circ T)$, for all $k \in \mathbb {N}$.¹⁴ For notational convenience, we write $\partial v_A(p^0_A) \equiv \partial (v^e_A \circ T)(x^0)$, where $T(x^0) = p^0_A$. Let now $P \subset \mathbb {R}^{|K_A|-1}$ be such that $T(P) = \Delta (K_A)$. Define the restricted superdifferential of $\text {Cav}(v_A \circ T|_{P})$ at p - denoted $\partial ^* \text {Cav}(v_A)(p)$ - as the set of vectors $v \in \mathbb {R}^{|K_A|-1}$ that satisfy $\text {Cav}(v_A)(p) + v \cdot h \ge \text {Cav}(v_A \circ T|_{P})(x+h)$ for all h with $x+h \in P$ and $T(x) = p$. Below, $\phi _AS$ denotes the pre-multiplication of the row vector $\phi _A$ by the matrix S.

Definition 3.5

The two-player infinitely repeated zero-sum game with one-sided incomplete information $G_A(p^0_A)$ satisfies the property NR at $p^0_A$ if there exists $p_A \in \Delta (K_A)$ and $\phi _A \in \mathbb {R}^{|K_A|}$ such that:

• (1) $\text {Cav}(v_A)(p_A) = v_A(p_A) = \phi _A \cdot p_A$ and Cav$(v_A)(p^0_A) = \phi _A \cdot p^0_A$;

• (2) $\phi _A S \in \partial v_A(p_A)$;

• (3) $\phi _A S \in \partial ^* \text {Cav}(v_A)(p_A)$.

The properties (1)–(3) in the Definition of NR at $p^0_A$ (Definition 3.5) are properties of $v_A$: (1) states that $p_A$ is a point of identity between $v_A$, $\text {Cav}(v_A)$ and the affine function $q \in \Delta (K_A) \mapsto \phi _A \cdot q$; (1) also states that $p^0_A$ is a point of identity between the same affine function and $\text {Cav}(v_A)$; (2) states that $\phi _AS$ is a (generalized) gradient at $p_A$ of $v_A$; (3) states that $\phi _A$ is a “supergradient” of $\text {Cav}(v_A)$ at $p_A$. We are now ready to state the sufficient condition.

Theorem 3.6

Let $p^0_A \in \Delta (K_A)$ such that $\text {supp}(p^0_A) = K_A$. Suppose $G_A(p^0_A)$ satisfies NR at $p^0_A$. Then the set of non-revealing equilibrium payoffs $NR_A(p^0_A)$ is nonempty. Evidently, the analogous statement holds for $G_B(p^0_B)$.

A proof of this Theorem can be found in Appendix A. Evidently, for $p^0 \in \Delta (K_A \times K_B)$ with $\text {supp}(p^0_A) = K_A$ and $\text {supp}(p^0_B) = K_B$, if $G_A(p^0_A)$ satisfies NR at $p^0_A$ and $G_B(p^0_B)$ satisfies NR at $p^0_B$, then it follows from Theorem 3.2 that $I(p^0)$ is the ex-ante equilibrium payoff set of the informed player.

Remark 3.7

As both definitions of the property NR at $p^0_A$ and the set $NR_A(p^0_A)$ concern the game $G_A(p^0_A)$, we compare their content in detail. If a vector $\phi _A$ satisfies (3) and the first two equalities of (1) of Definition 3.5, then this implies that $\phi _A$ satisfies (i) in the definition of $NR_A(p^0_A)$. If $\phi _A$ satisfies the last equality of (1) of Definition 3.5, then it is immediate that it satisfies (ii) of $NR_A(p^0_A)$. Property (iii) in the definition of $NR_A(p^0_A)$ bears no immediate relation with the non-revealing value function $v_A$. But, as the proof of Theorem 3.6 shows, it is implied by a geometric property of $v_A$, namely, property (2). Therefore, information about $NR_A(p^0_A)$ can be infered from properties of $v_A$ only. The precise way to do this inference is presented in the proof of Theorem 3.6.

Our aim at introducing the condition NR at $p^0_A$ is to highlight that the attainability of the upper end of $I(p^0)$ as an ex-ante equilibrium payoff of the informed player is not purely an “information problem”, i.e., it does not depend exclusively on the existence of correlation between states in $K_A$ and $K_B$. The property shows how the attainability of the upper end of $I(p^0)$ also fundamentally relies on the payoff structure of the component games, which determines the geometry of the nonrevealing value function. Though the sufficient condition presented might not be easier to check computationally than the direct non-emptyness of $NR_A(p^0_A)$, it isolates the aspects of this geometry which determine the existence of equilibria with such payoffs.

3.4 An interpretation for NR

Even though the property NR at $p^0_A$ is not straightforward to interpret, as it mainly describes certain geometric properties of the non-revealing value function, there is a class of infinitely repeated two-player zero-sum games with one-sided incomplete information in which this property can be interpreted straightforwardly and in which this property is always satisfied.

In this section, whenever a two-player, zero-sum infinitely repeated game $G_A(p^0_A)$ is considered, it is assumed, without loss of generality, that $p^0_A \in \text {int}(\Delta (K_A))$.

Definition 3.8

A two-player, infinitely repeated zero-sum game with lack of information on one-side and undiscounted payoffs $G_A(p)$ is locally non-revealing at p¹⁵ whenever there exist $k \in \mathbb {N}, (\lambda _i)^{k}_{i=1} \in \mathbb {R}^{k}$ and $(p_i)^{k}_{i =1} \in \prod ^{k}_{i=1}\Delta (K_A)$ such that:

• (1) For each $i=1,...,k$, $\lambda _i >0$ and $\sum ^{k}_{i=1}\lambda _i = 1$;

• (2) $\sum ^{k}_{i =1}\lambda _i p_i = p$;

• (3) $\text {Cav}(v_A)(p)= \sum ^{k}_{i=1}\lambda _i v_A(p_i)$;

• (4) For some $i_0 \in \{1,...,k\}$, $p_{i_0} \in$ int($\Delta (K_A)$).

The definition above implicitly describes an optimal strategy for the informed player in the game $G_A(p)$ for which there is signalling on path of play: the informed player “splits” the prior p into finitely many posteriors $(p_i)^k_{i=1}$ such that $\text {Cav}(v_A)(p)= \sum ^{k}_{i=1}\lambda _i v_A(p_i)$. This is the typical optimal strategy constructed by Aumann et al.. The local non-revelation property tells us that whenever the informed player in $G_A(p^0_A)$ has an Aumman et al.’s strategy under which he does not exclude some state (i.e., when there is at least one induced posterior in the interior of the simplex of states), then there exists an equilibrium of the game $G_A(p^0_A)$ where the informed player does not signal on path of play.

Figure 3 illustrates how the conditions of Definition 3.8 are related to the property of NR at $p^0_B$. Let $G_B(p^0_B)$ be the game originating the non-revealing value function $v_B$ (whose graph is depicted in black). The data defining this game is the exact same as in Example 2.4. In this figure, let $p^0_B = 1/2$ denote the prior probability of state 1. One can obviously write $p^0_B$ as a convex combination with equal weights of 1/4 and 3/4 (the “optimal splitting” which determines the Cav$(v_B)$). Note that at the interior posterior $q = 1/4$ in the figure, the vectors $N_2$ and $N_1$ generate the normal cone to the graph of $v_B$ at $q=1/4$. Each vector of this normal cone is uniquely associated with a supergradient of $v_B$ at $q=1/4$, i.e., $N_m \cdot q = (q, \mathfrak {n}_m \cdot q), m \in \{1,2\}$, so $N_m = (1, \mathfrak {n}_m)$ and $\mathfrak {n}_m$ is a super-gradient of $v_A$ at 1/4. Letting $N = (1, \mathfrak {n})$, note that Cav$(v_B)(q) = \mathfrak {n} \cdot q, q \in [1/4, 3/4]$ and N belongs to that normal cone at $q=1/4$, and therefore $\mathfrak {n}$ can be given by a convex combination of $\mathfrak {n}_1$ and $\mathfrak {n}_2$.

Fig. 3

Local non-revelation and NR at $p^0_B =1/2$

Proposition 3.9

Suppose $G_A(p^0_A)$ is locally non-revealing at $p^0_A$. Then $G_A(p^0_A)$ satisfies NR at $p^0_A$. Evidently, the analogous statement holds for $G_B(p^0_B)$.

Proposition 3.9 shows that the local non-revelation property implies the NR property. For a proof of this proposition, see Appendix A. Example 3.10 below shows that the NR property is strictly more general than the local non-revelation property. As an example, the games $G_A(p^0_A)$ and $G_B(p^0_B)$ defined in Example 2.4 are both locally non-revealing at their respective priors. Note that in that example, the game $G_B(p^0_B)$ is such that the optimal strategy of the informed player constructed by Aumann et al. involves signalling on path (this is the signalling strategy we briefly described in the example).

Example 3.10

Let $K_A = \{1,2\}$ be the set of states. Let q denote the probability of state 1 and $p^0_A = 1/2$ (prior of state 1). Consider the following game $G_A(p^0_A)$:

$$\begin{aligned} A^1 = \begin{bmatrix} 0 &{} 0 \\ 0 &{} -1 \end{bmatrix}; A^2 = \begin{bmatrix} -1 &{} 0 \\ 0 &{} 0 \end{bmatrix} \end{aligned}$$

For a row vector $v \in \mathbb {R}^m$, denote by $v'$ the transposed column vector. Figure 4 depicts the graphs of $v_A(q) = -q(1-q)$ and $\text {Cav}(v_A)(q) = 0, \forall q \in [0,1]$. Consider the actions $\sigma _A = (1,0)$ and $\tau _A = (0,1)'$. Then $\alpha _A = (\sigma _A A^1 \tau _A, \sigma _A A^2 \tau _A) = (0,0)$, so $\alpha ^A \cdot q = \text {Cav}(v_A)(q) = 0, \forall q \in [0,1]$. Therefore $\alpha _A \in NR_A(p^0_A)$. First note that the local non-revelation property at $p^0_A$ is not satisfied in $G_A(p^0_A)$. We show that the property NR at $p^0_A$ is satisfied in the example. The linear transformation S in this example is defined by $Sx = (x, -x)$. Notice that the only candidates for p and $\phi _A$ satisfying the conditions of property NR at $p^0_A$ are $p = 0$ or $p=1$ and $\alpha _A$. Notice that at $p=0$, just by looking at the graph depicted above, one can see that conditions (1) and (3) of property NR are satisfied for the vector $\alpha ^A$. Now, $\lim _{q \rightarrow 0^{-}}\nabla (v^e_A \circ T)(q) =0$, since $v^e_A \circ T$ is constant and equal to 0 in $(-\infty , 0)$. Using the notation defined above, we have that $\alpha ^A S$ = 0, so that $\alpha ^A S \in \partial v_A(p)$. Observe that because of the strict convexity of the non-revealing value function $v_A$, the optimal strategy of the informed player as constructed in Aumann et al. necessarily involves signalling on path, namely, inducing posteriors at the boundary of the 1-simplex of states. However, property NR at $p^0_A$ guarantees that an equilibrium exists for which no signalling occurs on path.

Fig. 4

Graphs of $\text {Cav}(v_A)$ and $v_A$

Fig. 5

Graphs of $\text {Cav}(v_A)$(dotted) and $v_A$(continuous)

The next example shows a game $G_A(p^0_A)$ on which NR at $p^0_A$ is not satisfied.

Example 3.11

Let $G_A(p^0_A)$ be defined from the payoff matrices below and q denote the probability of state 1 with prior $p^0_A = 1/2$.

$$\begin{aligned} A^1 = \begin{bmatrix} 1 &{} 1 \\ -1 &{} -1 \end{bmatrix} \,\,; \, A^2 = \begin{bmatrix} -1 &{} -1 \\ 1 &{} 1 \end{bmatrix} \end{aligned}$$

Figure 5 depicts the non-revealing value function and its concavification. The only two candidates for $p_A$ and $\phi _A$ are $p_A=0$ and $p_A=1$ and (1, 1). Notice that $v^e_A \circ T$ has a derivative at $p_A=0$, which is equal to $-2$, so $\partial v_A(p_A) = \{-2\}$, whereas $\phi _A S= (1,1) S =0$. Therefore, $\phi _A S \notin \partial v_A(p_A)$. The same reasoning applies to show that $\phi _A S \notin \partial v_A(p_A)$, when $p_A=1$. Therefore, property (2) of Definition 3.5 is not satisfied.

Remark 3.12

It is possible to prove directly that if $G_A(p^0_A)$ is locally non-revealing at $p^0_A$ then $NR_A(p^0_A) \ne \emptyset$. The proof is illustrative of how the geometry of $v_A$ determines the existence of equilibria in $G_A(p^0_A)$ that involve no signalling on path of play and therefore we include it here.

The proof is divided in two cases: (a) $\text {Cav}(v_A)(p^0_A) > v_A(p^0_A)$ and (b) $\text {Cav}(v_A)(p^0_A) = v_A(p^0_A)$. We first construct the candidate vector of payoffs $\phi _A \in \mathbb {R}^{|K_A|}$ for the informed player. Then we show that $\phi _A \in NR_A(p^0_A)$.

Let $\text {Cav}(v_A)(p^0_A) = \sum _{s \in S}\lambda _s v_A(p_s)$, $\lambda _s \ge 0$, $\sum _{s \in S}\lambda _s =1$ and let $p_{s_0}$ be the posterior which is interior to the simplex of states. Let $\mathcal H= \{(q,\alpha ) \in \Delta (K_A) \times \mathbb {R}| \text {Cav}(v_A)(q) \ge \alpha \}$. Note that $\mathcal H$ is a convex subset of $\mathbb {R}^{|K_A|} \times \mathbb {R}$. Now, note that $(p^0_A, \text {Cav}(v_A)(p^0_A)) = \sum _{s \in S}\lambda _s(p_s, v_A(p_s))$. Since, by (a), $\forall s \in S, p_s \ne p_0$, it follows that $(p^0_A, \text {Cav}(v_A)(p^0_A))$ is not an extremum point of $\mathcal H$ (cf. Rockafellar (1970), Section 18). Therefore, there exists a face F of $\mathcal H$, with dimension $d \ge 1$, such that F contains $(p^0_A, \text {Cav}(v_A)(p^0_A))$ and each point $(p_s, v_A(p_s))$. Take now a supporting hyperplane H to $\mathcal H$ which contains F. The hyperplane H intersected with $\Delta (K_A) \times \mathbb {R}$ is the graph of an affine function $q \in \Delta (K_A) \mapsto \phi _A \cdot q \in \mathbb {R}$, where $\phi _A$ is a vector in $\mathbb {R}^{|K_A|}$. We now claim the following:

Claim 3.13

The vector $\phi _A$ belongs to $NR_A(p^0_A)$.

Proof

As the graph of the affine function $q \in \Delta (K_A) \mapsto \phi _A \cdot q$ is $H \cap (\Delta (K_A) \times \mathbb {R})$, it follows that $\phi _A \cdot q = \text {Cav}(v_A)(q), \forall q \in [p_{s_0}, p^0_A]$, where $[p_{s_0}, p^0_A]$ denotes the segment between $p_{s_0}$ and $p^0_A$. In particular, $\text {Cav}(v_A)(p^0_A) = \phi _A \cdot p^0_A$ is satisfied, and so $\phi _A$ satisfies (ii). From the fact that H supports $\mathcal H$ at F, it follows that $\phi _A \cdot q \ge v_A(q), \forall q \in \Delta (K_A)$. This proves $\phi _A$ satisfies (i). Thus it only remains to show $\phi _A \in F_A$. Let $\sigma _A$ be the Aumann et al. optimal strategy of the informed player in $G_A(p_{s_0})$, which is state-independent since $\text {Cav}(v_A)(p_{s_0}) = v_A(p_{s_0})$. For the uninformed player, there exists a Blackwell strategy¹⁶$\tau _A$ and a constant $L>0$ such that $\mathbb {E}^{k_A}_{\sigma _A,\tau _A, p_{s_0}}[\frac{1}{T} \sum ^{T}_{t=1}(A^{k_A}_{i^t_A, j^t_A})] \le \phi ^{k_A}_A + \frac{L}{\sqrt{T}}, \forall k_A \in K_A$ (see proof of Corollary 3.34 in Sorin 2002), where $\mathbb {E}^{k_A}_{\sigma _A,\tau _A, p_{s_0}}$ is the conditional expectation given $\kappa _A = k_A$. The profile $(\sigma _A, \tau _A)$ is therefore a uniform equilibrium of $G_A(p_{s_0})$, so it follows that the limit $\lim _{T \rightarrow +\infty }\mathbb {E}^{k_A}_{\sigma _A,\tau _A, p_{s_0}}[\frac{1}{T} \sum ^{T}_{t=1}(A^{k_A}_{i^t_A, j^t_A})]$ exists for each $k_A \in K_A$.

For each $k_A \in K_A$, let $v^{k_A}$ be this limit. It follows from the inequality of the Blackwell strategy given above that $\forall k_A \in K_A, v^{k_A} \le \phi ^{k_A}_A$. Since for each $T \ge 1$,

$$\begin{aligned} \mathbb {E}_{\sigma _A,\tau _A, p_{s_0}}\Big [\frac{1}{T} \sum ^{T}_{t=1}(A^{\kappa _A}_{i^t_A, j^t_A})\Big ] = \sum _{k_A \in K_A} p^{k_A}_{s_0} \Big (\mathbb {E}^{k_A}_{\sigma _A,\tau _A, p_{s_0}}\Big [\frac{1}{T} \sum ^{T}_{t=1}(A^{k_A}_{i^t_A, j^t_A})\Big ]\Big ) \end{aligned}$$

it follows that $\sum _{k_A \in K_A} p^{k_A}_{s_0}v^{k_A} = \text {Cav}(v_A)(p_{s_0})$, as any uniform equilibrium pays $\text {Cav}(v_A)(p_{s_0})$. Since $p_{s_0} \in \text {int}(\Delta (K_A))$, it must be that $v^{k_A} = \phi ^{k_A}_A$, for all $k_A \in K_A$. Since $F_A$ is compact and convex and neither $\sigma _A$ nor $\tau _A$ condition on realized states, it follows that $(v^{k_A})_{k_A \in K_A} = v \in F_A$. This shows $\phi _A \in F_A$ and concludes that $\phi _A \in NR_A(p^0_A)$, as claimed.

The proof of case (b) is immediate, since the Aumann et al. optimal strategy of the informed player in $G_A(p^0_A)$ is already state-independent. This concludes the proof of the claim. $\square$

We would like to highlight two things about the above remark. Notice that the first part of the proof does not rely on any assumption on $G_A(p^0_A)$ being locally non-revealing at $p^0_A$, i.e., it does not rely on the assumption that an optimal strategy of the informed player exists inducing an interior posterior. The first paragraph just serves the purpose of pinning down the correct vector $\phi _A$. Notice that the assumption of $p_{s_0} \in \text {int}(\Delta (K_A))$ is used in the proof above only when we show that $\phi _A = v$. Second, we did not show in the proof that the profile $(\sigma _A, \tau _A)$ is an equilibrium of $G_A(p^0_A)$; we used the fact that $(\sigma _A,\tau _A)$ is a uniform equilibrium of $G_A(p_{s_0})$, inducing the vector of payoffs for the informed player equal to $\phi _A$, and obtained that $\phi _A \in NR_A(p^0_A)$.

3.5 Consequences for two-player zero-sum repeated games with incomplete information

Theorem 3.6 tells us that given a two-player zero-sum game with one-sided incomplete information that satisfies the property NR at the prior, there exists an equilibrium of the 2-player game for which no signalling occurs on path of play (i.e., the set of non-revealing equilibrium payoffs of this game is non-empty). Even for games for which Aumann et al. constructed strategies that necessarily involved signalling on path (see Example 3.10 or $G_B(p^0_B)$ in Example 2.4), the NR property implies the existence of an equilibrium for which no signalling occurs on path. We would like to illustrate this message with an example.

Consider the two-player, zero-sum infinitely repeated game with one-sided incomplete information $G_B(1/2)$ between players 1 and 3, defined by the data in Example 2.4. Following Aumann et al.’s technique for constructing optimal strategies, the strategy of the informed player would be the strategy highlighted in Example 2.4, that is, a signalling strategy: the informed player uses his actions to signal information about the underlying state for finitely many stages and after that plays the (mixed) optimal action of the one-shot, zero-sum game given by the posterior at each state independently. Playing this strategy in $G_{B}(p^0_B)$ guarantees to the informed player an ex-ante payoff of $(1/2) v_B(1/4) + (1/2) v_B (3/4) = (1/2) \text {Cav}(v_B)(1/4) + (1/2) \text {Cav}(v_B)(3/4) = \text {Cav}(v_B)(1/2) = 1$. Now, the optimal strategy for the uninformed player in $G_B(1/2)$ is a so-called approachability strategy or Blackwell strategy. For a general game $G_B(p^0_B)$, a Blackwell strategy can be defined as follows: given $\phi _B \in \mathbb {R}^{|K_B|}$ s.t. $\phi _B \cdot q \ge v_B(q), q \in \Delta (K_B), \text {Cav}(v_B)(p^0_B) = \phi _B \cdot p^0_B$, $\tau _B$ is a Blackwell strategy (for $\phi _B$ and $p^0_B$), if for each $\varepsilon >0$, there exists $T_0 \in \mathbb {N}$ such that $\forall T \ge T_0$, $\sigma _B$ a strategy of the informed player and $k_B \in K_B$,

$$\begin{aligned} \mathbb {E}^{k_B}_{\sigma _B,\tau _B,p^0}\Big [\frac{1}{T} \sum ^{T}_{t=1}(B^{\kappa _B}_{i^t_B, j^t_B})\Big ] \le \phi ^{k_B}_B + \varepsilon , \end{aligned}$$

where $\mathbb {E}^{k_B}_{\sigma _B,\tau _B,p^0}$ is the conditional expectation given $\kappa _B = k_B$. So $\tau _B$ precludes the informed player from achieving more than $\phi ^{k_B}_B + \varepsilon$ in a sufficiently long (but finitely repeated) game, for any state $k_B \in K_B$. This implies that the ex-ante expected payoff to the informed player in a sufficiently long game is not larger than $\text {Cav}(v_B)(p^0_B) + \varepsilon$. For our example, take $\phi _B = (1,1)$ and consider $\tau _B$ the Blackwell strategy for $\phi _B$ and $p^0_B$ as defined in the example. It follows the pair $(\sigma _B, \tau _B)$ is a (uniform) equilibrium with associated payoff $\text {Cav}(v_B)(p^0_B)=1$.

We call the strategies just defined standard optimal strategies. In contrast to these standard optimal strategies, for which there is revelation of information on path of play, we now construct equilibrium strategies for both players for which no information is revealed on path. The idea for the construction of these strategies is simple. Both players play a deterministic sequence of actions as long as the other plays it. This deterministic sequence is supported by punishment strategies, in the sense that if any player deviates from his sequence of actions, the other player starts to play the punishment strategy forever. Let us first define the deterministic path of play for this example: the deterministic sequence of actions is defined by $((i^t_B, j^t_B))^{\infty }_{t=1} = ((U,R), (U,M), (U,R), (U,M),...)$; so the uninformed player alternates between R (right column) and M (middle column), whereas the informed player plays U at every stage. For each state $k_B \in \{1,2\}$, the payoff (to the informed player) obtained from this path of play is:

$$\begin{aligned} \lim _{T \rightarrow +\infty }\frac{1}{T}\sum ^{T}_{t=1}B^{k_B}_{i^t_B, j^t_B} = 1. \end{aligned}$$

This implies the ex-ante payoff to the informed player is also $1 = \text {Cav}(v_B)(1/2)$. In case any player deviates from his prescribed sequence of actions, the other player can observe the deviation and play from the next stage onwards his standard optimal strategy in $G_B(1/2)$, which guarantees that a deviation is not profitable. As mentioned, the strategies just defined are also a (uniform) equilibrium in $G_B(1/2)$, but reveal no information on path of play.

4 Main result 2: necessary condition for the upper end of $I(p^0)$ to be attained in equilibrium

Theorem 4.1 is the main result of this section. Intuitively, it shows that equilibria which pay the upper end of $I(p^0)$ to the informed player imply a particular type of signalling procedure. Theorem 4.1 can therefore be viewed as a constraint to the signalling strategies an informed player might play in an equilibrium paying the upper end of $I(p^0)$. Presenting the formal statement of the Theorem requires some preliminary definitions.

Endow $I_A, I_B, J_A, J_B, K_A$ and $K_B$ with the discrete topology. For each $t \in \mathbb {N}$, let $\mathcal {H}_t$ be the discrete field over $H_{t}:= (I_A \times I_B \times J_A \times J_B)^{t-1}$. Endow $H_{\infty }:= (I_A \times I_B \times J_A \times J_B)^{\infty }$ and $\Omega := H_{\infty } \times K_A \times K_B$ with the induced product topology and let $\mathcal {H}_{\infty }$ and $\mathcal {F}(\Omega)$ be the Borel sigma-fields over $H_{\infty }$ and $\Omega$, respectively. For notational convenience, we will also denote by $\mathcal {H}_t$ the field generated by $H_t$ on $\Omega$.

Let $(\sigma , \tau _A, \tau _B)$ be a profile of strategies in $\mathcal {G}(p^0)$. Let $\mathbb {P}_{\sigma , \tau _A, \tau _B, p^0}$ be the probability induced by $(\sigma , \tau _A, \tau _B, p^0)$ on $(\Omega, \mathcal {F}(\Omega))$. We define the martingale of posteriors obtained through Bayesian updating. For $t \in \mathbb {N}$, let $p^{k_A,k_B}_t: =\mathbb {P}_{\sigma ,\tau _A,\tau _B,p}(\kappa = (k_A, k_B)| \mathcal {H}_t)$ and $p_t: = (p^{k_A, k_B}_t)_{(k_A,k_B) \in K_A \times K_B}$. The sequence $(p_t)_{t \in \mathbb {N}}$ is a $\Delta (K_A \times K_B)$-valued martingale with respect to $(\mathcal {H}_t)_{t \in \mathbb {N}}$, satisfying: (i) $p_1 = p^0$; (ii) there exists $p^{\infty }$ such that $p_t \rightarrow p^{\infty }$ a.s. as $t \rightarrow + \infty$. The a.s. limit $p^{\infty }$ of the process $(p_t)_{t \in \mathbb {N}}$ is called the asymptotic posterior.

Theorem 4.1

Let $(\sigma , \tau _A, \tau _B)$ be an equilibrium of $\mathcal {G}(p^0)$ and let $(p_{t})_{t \in \mathbb {N}}$ be the martingale of posteriors induced by the equilibrium. Assume $(\sigma , \tau _A, \tau _B)$ pays ex-ante $\text {Cav}(v_A)(p^0_A) + \text {Cav}(v_B)(p^0_B)$ to the informed player. Then $(\text {Cav}(v_{\ell })(p_{t\ell }))_{t \in \mathbb {N}}$ is a martingale, for each $\ell \in \{A,B\}$.

For an intuition on Theorem 4.1, note that from Jensen’s inequality, it is immediate to see that Cav$(v_A)(p_{t \ell })$ is a supermartingale w.r.t. $(\mathcal H_t)_{t \in \mathbb {N}}$. If Cav$(v_A)(p_{t \ell _0})$ is not a martingale for some $\ell _0 \in \{A,B\}$, then this would imply that the expected payoffs induced by the equilibrium in game $G_{\ell _0}(p^0_{\ell _0})$ are less than Cav$(v_A)(p^0_A)$, contradicting the assumption of the Theorem.

The next corollary of Theorem 4.1 is motivated by the following intuition. Take a game $\mathcal {G}(p^0)$ for which $p^0 \in \text {int}(\Delta (K_A \times K_B))$ and assume that $\mathcal{N}\mathcal{R}(p^0) = \emptyset$. This last assumption implies that if the upper end of $I(p^0)$ can be attained in equilibrium in $\mathcal {G}(p^0)$, then the informed player must use a state-dependent strategy on path, i.e., he must use signalling. An idea of how such signalling procedure could occur is as follows: the informed player could signal so as to induce posteriors that are product distributions; this would imply that, once the posterior realizes, from then onwards no correlation exists between the zero-sum games at the posteriors and therefore the informed player could play each of the zero-sum games without concern for information spillover; is it possible that there are equilibria of $\mathcal {G}(p^0)$ for which the described signalling procedure exists and the upper end of $I(p^0)$ can be attained in equilibrium? Corollary 4.2 provides an answer to this question.

Corollary 4.2

Let $(\sigma ,\tau _A,\tau _B)$ be an equilibrium of $\mathcal {G}(p^0)$. Suppose this equilibrium pays the upper end of $I(p^0)$ and its associated asymptotic posterior $p^{\infty }$ is a product¹⁷ a.s.. Then $I(p^0)$ is degenerate.

Proof

We will denote by $\mathbb {E}$ the expectation operator $\mathbb {E}_{\sigma , \tau _A, \tau _B, p^0}$. Since $p^{\infty }$ is a product a.s., then $\text {Cav}(\mathfrak {h})(p^{\infty }) = \text {Cav}(v_A)(p^{\infty }_A) + \text {Cav}(v_B)(p^{\infty }_B)$ a.s.. Taking expectations on both sides and using Jensen’s inequality, it follows that $\text {Cav}(\mathfrak {h})(p^{0}) \ge \mathbb {E}[\text {Cav}(v_A)(p^{\infty }_A)] + \mathbb {E}[\text {Cav}(v_B)(p^{\infty }_B)]$. By Theorem 4.1, $\mathbb {E}[\text {Cav}(v_A)(p^{\infty }_A)] = \text {Cav}(v_A)(p^0_A)$ and $\mathbb {E}[\text {Cav}(v_B)(p^{\infty }_B)] = \text {Cav}(v_B)(p^0_B)$. Therefore, $\text {Cav}(\mathfrak {h})(p^{0}) \ge \text {Cav}(v_A)(p^0_A) + \text {Cav}(v_B)(p^0_B)$. Since, $\text {Cav}(\mathfrak {h})(p^{0}) \le \text {Cav}(v_A)(p^0_A) + \text {Cav}(v_B)(p^0_B)$, we have that $I(p^0)$ is degenerate. $\square$

Therefore, if $I(p^0)$ is non-degenerate, no equilibrium paying the upper end of $I(p^0)$ to the informed player induces a product asymptotic posterior. In other words, such an equilibrium must maintain the correlation (even at infinity) between the zero-sum games with positive probability.

4.1 On the proof of theorem 4.1

The proof of Theorem 4.1 requires some preliminary work. In particular, it requires an auxiliary Lemma (Lemma 4.3) which provides a necessary condition for equilibria of $\mathcal {G}(p^0)$.

Lemma 4.3

Let $(\sigma , \tau _A, \tau _B)$ be an equilibrium in $\mathcal {G}(p^0)$. Then there exists a sequence of random variables $(p_t, \beta _{A,t}, \beta _{B,t})_{t \in \mathbb {N}}$ on the probability space $(\Omega , \mathcal {F}(\Omega), \mathbb {P}_{\sigma ,\tau _A,\tau _B,p^0})$ taking values in $\Delta (K_A \times K_B) \times \mathbb {R}\times \mathbb {R}$ such that:

• (1) $(p_t, \beta _{A,t}, \beta _{B,t})_{t \in \mathbb {N}}$ is a martingale adapted to $(\mathcal {H}_t)_{t \in \mathbb {N}}$.

• (2) $\beta _{A,1} + \beta _{B,1}$ is the expected payoff of the equilibrium to player 1.

• (3) $\beta _{A,t} \le \text {Cav}(v_A)(p_{tA})$ a.s., $\forall t \in \mathbb {N}$.

• (4) $\beta _{B,t} \le \text {Cav}(v_B)(p_{tB})$ a.s., $\forall t \in \mathbb {N}$.

Proof of theorem 4.1

First, notice that for each $\ell \in \{A,B\}$ and $k,s \in \mathbb {N}$ with $k \le s$ we have that $\text {Cav}(v_{\ell })(p_{k\ell }) = \text {Cav}(v_{\ell })(\mathbb {E}[p_{s \ell }| \mathcal {H}_k]) \ge \mathbb {E}[\text {Cav}(v_{\ell })(p_{s \ell }) | \mathcal {H}_k]$ a.s. – where the equality follows from the fact that $(p_s)_{s \in \mathbb {N}}$ is a martingale, and the inequality follows from Jensen’s inequality. Assume by contradiction that there exist $k,s \in \mathbb {N}$ with $k < s$, $\ell _0 \in \{A,B\}$ and an atom $h_k \in H_{k}$ such that Cav$(v_{\ell _0})(p_{k\ell _0})(h_k) > \mathbb {E}[\text {Cav}(v_{\ell _0})(p_{s \ell _0})| \mathcal {H}_{k}](h_k)$. It follows that $\text {Cav}(v_{\ell _0})(p^0_{\ell _0}) \ge \mathbb {E}[\text {Cav}(v_{\ell _0})(p_{k\ell _0})] > \mathbb {E}[\text {Cav}(v_{\ell _0})(p_{s\ell _0})] \ge \mathbb {E}[\beta _{\ell _0,s}] = \beta _{\ell _0,1}$, where the first inequality is given by Jensen’s inequality, the second by assumption, the third by (3) and (4) of Lemma 4.3 and the last equality by the martingale property. This then implies that $\text {Cav}(v_A)(p^0_A) + \text {Cav}(v_B)(p^0_B) > \beta _{A,1} + \beta _{B, 1}$. Contradiction, since by (2) of Lemma 4.3 we have that $\beta _{A,1} + \beta _{B,1} = \text {Cav}(v_A)(p^0_A) + \text {Cav}(v_B)(p^0_B)$. $\square$

Example 4.4

In the next example, $I(p^0)$ is non-degenerate and the upper end of $I(p^0)$ is not an ex-ante equilibrium payoff. More precisely, only the lower end of $I(p^0)$ is an ex-ante equilibrium payoff for the informed player. We will provide a proof of this claim through an application of Theorem 4.1. Later in a remark, we provide a more elementary proof of this claim, which will not make any reference to the stochastic process of payoffs and posteriors of Lemma 4.3. Consider $\mathcal {G}(p^0)$ defined by the following data:

$$\begin{aligned}{} & {} p^0 = \begin{bmatrix} 1/2 &{} 0 \\ 0 &{} 1/2 \end{bmatrix} \\{} & {} A^1 = \begin{bmatrix} 1 &{} 1 \\ -1 &{} -1 \end{bmatrix}; \,\, A^2 = \begin{bmatrix} -1 &{} -1 \\ 1 &{} 1 \end{bmatrix}\\{} & {} B^1 = \begin{bmatrix} 1 &{} 0 \\ 0 &{} 0 \end{bmatrix}; \,\, B^2 = \begin{bmatrix} 0 &{} 0 \\ 0 &{} 1 \end{bmatrix} \end{aligned}$$

Claim 4.5

For the game $\mathcal {G}(p^0)$ defined by the data above, only the lower end of $I(p^0)$ is an ex-ante equilibrium payoff for the informed player.

Proof

Assume by way of contradiction that $(\sigma , \tau _A, \tau _B)$ is an equilibrium that pays ex-ante $\text {Cav}(v_A)(p_A^0)+\text {Cav}(v_B)(p_B^0)$ for the informed player in $\mathcal {G}(p^0)$. Let $V_A: \triangle (K_A)\rightarrow \mathbb {R}$ be given by $V_A(p):= \text {max}_{\sigma ,\tau }\{ \sigma A(p) \tau | \sigma A(p) \tau \le \text {Cav}(v_A)(p) \}$, i.e., the maximum payoff player 1 attains in the one-shot zero-sum game with payoff matrix A(p) which is less than Cav$(v_A)(p)$. For this example we have that $V_A(p) = v_A(p), \forall p \in \triangle (K_A)$, which can be checked by computation. Let $(\beta _{A,s})_{s \in \mathbb {N}}, (\beta _{B,s})_{s \in \mathbb {N}}$ be the martingales from Lemma 4.3. We need the following auxiliary claim, whose proof is left to Appendix B.

Claim 4.6

For each $t \in \mathbb {N}$, $\beta _{A,t} \le V_A(p_{tA}) + Z_t$ a.s., where $(Z_t)_{t \in \mathbb {N}}$ is a (a.s.) nonnegative, bounded sequence that converges (a.s.) to 0.

By the claim, we have $\beta _{A,t} \le v_A(p_{tA}) + Z_t$ a.s.. The Martingale Convergence Theorem now implies that $\beta _{A,\infty } \rightarrow \beta _{A, \infty }, p_{tA} \rightarrow p^{\infty }_{A}$, as $t \rightarrow +\infty$. By the claim $Z_t \rightarrow 0$ a.s.. Therefore, we obtain $\mathbb {E}[\beta _{A,\infty }] \le \mathbb {E}[v_A(p^{\infty }_A)]$. From (2) and (3) in Lemma 4.3, we have that $\mathbb {E}[\beta _{A,\infty }] = \text {Cav}(v_A)(p^0_A) \ge \mathbb {E}[v_A(p^{\infty }_A)]$. So it follows that $\mathbb {E}[\beta _{A,\infty }] = \mathbb {E}[v_A(p^{\infty }_{A})]$, which implies that the distribution of $p^{\infty }_{A}$ is concentrated at the boundary of $\Delta (K_A)$. Since $v_B$ is strictly concave¹⁸ and $(v_B(p_{tB}))_{t \in \mathbb {N}}$ is a martingale (by Theorem 4.1), it follows that $p_{tB} = p^0_B$ a.s., $\forall t \in \mathbb {N}$. Hence, we have that for any history $h_{\infty }$ outside a set of $\mathbb {P}_{\sigma ,\tau _A, \tau _B,p}$-measure zero, the matrix representation of $p^{\infty }(h_{\infty })$ has either the first or the second row filled with zeros (recall that an entry $p^0_{ij}$ represents the probability of states i and j in game $G_A(p^0_A)$ and $G_B(p^0_B)$, respectively), i.e., $p^{\infty }(h_{\infty })$ is either:

$$\begin{aligned} \begin{bmatrix} 1/2 &{} 1/2 \\ 0 &{} 0 \end{bmatrix} \text { or }\begin{bmatrix} 0 &{} 0 \\ 1/2 &{} 1/2 \end{bmatrix} \end{aligned}$$

Now the process of posteriors is a martingale, which implies that the expectation of $p^{\infty }$ is $p^0$. This implies that the following equation has a solution in $\lambda \in [0,1]$:

$$\begin{aligned} \begin{bmatrix} 1/2 &{} 0 \\ 0 &{} 1/2 \end{bmatrix} = \lambda \begin{bmatrix} 0 &{} 0 \\ 1/2 &{} 1/2 \end{bmatrix} + (1 - \lambda ) \begin{bmatrix} 1/2 &{} 1/2 \\ 0 &{} 0 \end{bmatrix}. \end{aligned}$$

But this equation has no solution for $\lambda \in [0,1]$, which finally implies a contradiction. Hence, there is no equilibrium paying ex-ante to the informed player $\text {Cav}(v_A)(p^0_A)+\text {Cav}(v_B)(p^0_B)$. The arguments above give us more: recall that we had $\beta _{A, \infty } \le V_A(p^{\infty }_A) = v_A(p^{\infty }_A)$ a.s. and since $\beta _{B, \infty } \le \text {Cav}(v_B)(p^{\infty }_B) = v_B(p^{\infty }_B)$ a.s., these imply that $\beta _{A, \infty } + \beta _{B,\infty } \le v_A(p^{\infty }_A) + v_B(p^{\infty }_B)$ a.s. and therefore $\mathbb {E}[\beta _{A, \infty } + \beta _{B, \infty }] \le \mathbb {E}[v_A(p^{\infty }_A) + v_B(p^{\infty }_ B)] \le \mathbb {E}[\text {Cav}(\mathfrak {h})(p^{\infty })] \le \text {Cav}(\mathfrak {h})(p^0)$, where the second inequality follows by definition of $\text {Cav}(\mathfrak {h})$ and the last inequality is given by Jensen’s inequality. The number $\text {Cav}(\mathfrak {h})(p^0)$ is the lowest possible ex-ante equilibrium payoff to the informed player. This implies that every uniform equilibrium of the example pays $\text {Cav}(\mathfrak {h})(p^0)$ to the informed player. $\square$

Remark 4.7

We would like to provide an alternative proof of the claim that in the game of Example 4.4 only the lower end of $I(p^0)$ is an equilibrium payoff.¹⁹ It is obvious that the matrix $A^1$ (respec. $A^2$) can be substituted by the following equivalent matrix $A^1_r$ (respec. $A^2_r$), by simply eliminating the redundant column action of player 2. So,

$$\begin{aligned}{} & {} p^0 = \begin{bmatrix} 1/2 &{} 0 \\ 0 &{} 1/2 \end{bmatrix} \\{} & {} A^1_r = \begin{bmatrix} 1 \\ -1 \end{bmatrix}; \,\, A^2_r = \begin{bmatrix} -1 \\ 1 \end{bmatrix} \\{} & {} B^1 = \begin{bmatrix} 1 &{} 0 \\ 0 &{} 0 \end{bmatrix}; \,\, B^2 = \begin{bmatrix} 0 &{} 0 \\ 0 &{} 1 \end{bmatrix} \end{aligned}$$

The model $\mathcal {G}(p^0)$ defined by the data above is evidently equivalent to the model $\mathcal {G}(p^0)$ defined by the data of the previous example, as only redundant actions have been eliminated, which leaves the best-reply correspondences of all players unaltered. Player 2 now, evidently, is a dummy player. We can define therefore a two-player non-zero-sum game between players 1 and 3, whose equilibria immediately induce the equilibria of the three-player game $\mathcal {G}(p^0)$. To be precise, we define a two-player, non-zero-sum infinitely repeated game with one-sided incomplete information and undiscounted payoffs $\mathbb {G}(q^0)$, where the set of states will be $K=\{1,2\}$ with prior $q^0=1/2$ for state 1: the payoffs are given by the following bimatrix $C^k$ ($k \in K$), where the informed player plays row and the uninformed player 3 plays column:

The rows and column labels in matrix $C^k$ should be read as follows: L and R stand for the stage-game actions of the uninformed player (i.e., player 3). For the row player (i.e., player 1), (U, D) corresponds to choosing the top row in game in $A^{k}$ and the bottom row in game $B^{k}$. A generic entry is therefore $C^k_{(i_A, i_B), j_B} \equiv ((A^{k}_r)_{i_A} + B^{k}_{i_B, j_B}, -B^k_{i_A, j_B})$. The other entries are analogously constructed.

We can now modify the stage-game payoffs $C^k, k=1,2$, so that the best-reply correspondence of both players remains unaltered and, after the modification, we obtain a zero-sum game between players 1 and 3. Define new stage-game payoff matrices $D^k, k=1,2$ by: $D^k_{(i_A, i_B), j_B} \equiv C^k_{(i_A,i_B), j_B} - (0, (A^k_r)_{i_A}) = ((A^{k}_r)_{i_A} + B^{k}_{i_B, j_B}, -(A^{k}_r)_{i_A} - B^k_{i_A, j_B})$. Consider now the two-player, zero-sum infinitely repeated game with one-sided incomplete information and undiscounted payoffs where the stage-game payoff matrices are given by $(D^{1}, D^2)$ and the prior of state 1 is $q^0 = 1/2$. This modification leaves the payoffs of player 1 unaltered, and therefore does not change his best-reply correspondence when compared to $\mathbb {G}(q^0)$. Though the payoffs of player 3 are modified, his best-reply correspondence is not, which finally implies that the equilibria under this modification are the same as in $\mathbb {G}(q^0)$. Since the modified game is now zero-sum, it follows from Aumann et al. that it has a (uniform) value, which is the unique ex-ante uniform equilibrium payoff to player 1, and is then evidently the lower end of $I(p^0)$ (cf. footnote 7).

5 Information spillover in Bayesian persuasion: a comparison with our results

The problem of information spillover can also be studied in the Bayesian Persuasion (BP) setting. This possibility is briefly discussed in the paper by Gentzkow and Kamenica (2011) in the section “Multiple Receivers”. We would like to draw a comparison between the effects of information spillover over equilibrium payoffs in our model and over equilibrium payoffs in BP. We first describe the game form of the BP model we have in mind in detail. We refer to this model as public BP.

Let $K_A \times K_B$ be the set of states, with $K_A$ and $K_B$ finite sets. The set $M_A \times M_B$ is the set of messages, with $M_A$ and $M_B$ being finite sets and $|M_i| \ge |K_i|, i \in \{A,B\}$. The actions of player 2 (respec. player 3) are denoted $j_A \in J_A$, ($j_B \in J_B$), with both $J_A$ and $J_B$ being finite sets. At an ex-ante stage, player 1 chooses a state-dependent lottery or experiment $x \in \Delta (M_A \times M_B)^{K_A \times K_B}$ (the set of pure actions of Player 1). Then Nature draws a state $(k_A, k_B)$ according to some prior probability $p^0 \in \Delta (K_A \times K_B)$ and a message $(m_A, m_B)$ according to $x^{(k_A,k_B)} \in \Delta (M_A \times M_B)$. Players 2 and 3 observe the message $(m_A, m_B)$ but not the states; player 2 takes an action $j_A$ and player 3 takes an action $j_B$ and the game ends. We now define payoffs for the players. Given state $(k_A, k_B) \in K_A \times K_B$ and actions $j_A$ and $j_B$ of players 2 and 3, player 1 obtains payoff $u_A(k_A, j_A) + u_B(k_B, j_B)$; player 2 obtains $\nu _A(k_A, j_A)$ and player 3 obtains $\nu _B(k_B, j_B)$.

In the model just described, players 2 and 3 observe messages publicly (which motivates the terminology public BP). We will also be interested in the model where messages are observed privately by each player (i.e., player 2 observes only $m_A$ and player 3 observes only $m_B$), and will call this model private BP.

Using the equilibrium concept in Gentzkow and Kamenica (2011), it is not hard to show that any equilibrium will pay to player 1 the same payoff. We compute this equilibrium payoff: for any $q_A \in \Delta (K_A)$, let $\tau _A(q_A) = \text {argmax}_{\tau _A \in \Delta (J_A)}\sum _{k_A}\sum _{j_A}q^{k_A}_A \tau _A(j_A)\nu _A(k_A, j_A)$; for $q_B \in \Delta (K_B)$ we define $\tau _B(q_B) \in \Delta (J_B)$ analogously for player 3. Each $x \in \Delta (M_A \times M_B)^{K_A \times K_B}$ uniquely corresponds to a distribution over $\Delta (K_A \times K_B)$ with finite support and with mean $p^0$, so we can without loss assume that player 1 chooses a distribution over $\Delta (K_A \times K_B)$ with finite support and mean $p^0$. Concretely, this amounts to choosing a vector $\lambda = (\lambda _m)_{m \in M_A \times M_B}, \lambda _m \ge 0$ and $\sum _{m}\lambda _m = 1$ and $(p_m)_{m \in M_A \times M_B}$, $p_m \in \Delta (K_A \times K_B)$ such that $\sum _{m \in M_A \times M_B} \lambda _m p_m = p^0$. For each realized message $m \in M_A \times M_B$, the induced posterior is denoted $p_m \in \Delta (K_A \times K_B)$ and $\lambda _m$ corresponds to the probability with which $p_m$ realizes. In equilibrium, player 1 chooses a distribution over posteriors so as to maximize $\sum _{m \in M_A \times M_B}\lambda _m(U_A(p_{mA}) + U(p_{mB}))$, where $U_A(q_A) = \sum _{k_A \in K_A}\sum _{j_A \in J_A}q^{k_A}_A \tau _A(q_A)(j_A) u_A(k_A, j_A)$ and $U_{B}(q_B) = \sum _{k_B \in K_B}\sum _{j_B \in J_B}q^{k_B}_B \tau _B(q_B)(j_B) u_B(k_B, j_B)$. It is now clear that the maximum value of this program corresponds precisely to the definition of Cav$(U_A + U_B)(p^0)$, which is the equilibrium payoff of player 1.

It is easy to construct examples where $\text {Cav}(U_A)(p^0_A) + \text {Cav}(U_B)(p^0_B) > \text {Cav}(U_B + U_A)(p^0)$.²⁰ If in addition we assume we are in the public BP model, we have shown in the previous paragraph $\text {Cav}(U_A)(p^0_A) + \text {Cav}(U_B)(p^0_B)$ cannot be an equilibrium payoff, because of the information spillover phenomenon.

When messages are privately sent to players, however, this is simply the standard BP model of a sender simultaneously playing two receivers, which implies that $\text {Cav}(U_A)(p^0_A) + \text {Cav}(U_B)(p^0_B)$ is an equilibrium payoff. The difference between $\text {Cav}(U_A)(p^0_A) + \text {Cav}(U_B)(p^0_B) - \text {Cav}(U_B + U_A)(p^0)$ can be interepreted, therefore, as the loss to the Sender generated by information spillover in the public BP model.

If we specify payoffs to be zero sum, i.e., $\nu _A = - u_A$ and $\nu _B = - u_B$, the public BP model yields that $U_A$ and $U_B$ are concave functions, therefore implying that $\text {Cav}(U_A)(p^0_A) + \text {Cav}(U_B)(p^0_B) = \text {Cav}(U_A + U_B)(p^0) = U_B(p^0_A) + U_A(p^0_A)$. The next claims settles this result.

Claim 5.1

In the public BP zero-sum model, $U_A$ as well as $U_B$ are concave. Therefore, for $p^0 \in \Delta (K_A \times K_B), \text {Cav}(U_A)(p^0_A) + \text {Cav}(U_B)(p^0_B) = \text {Cav}(U_A + U_B)(p^0) = U_B(p^0_B) + U_A(p^0_A)$.

Proof

We prove that $U_A$ is concave. The proof of concavity of $U_B$ is similar. For each $p_A \in \Delta (K_A)$ and $j_A \in J_A$, let $f_{j_A}(p_A) = \sum _{k_A \in K_A} p^{k_A}_A u_A(k_A, j_A)$. Note that $f_{j_A}$ is an affine function of $p_A$. Since player 2 is a minimizer, $U_A(p_A) = \text {min}_{j_A \in J_A}\{f_{j_A}(p_A)\}_{j_A \in J_A}$. The map $U_A$ is therefore piecewise affine and concave in $p_A$. The remainder of the claim is immediate from the definition of Cav. $\square$

Hence, the effect of information spillover in the zero-sum public BP model is inexistent from the perspective of equilibrium payoffs, but is relevant in the non-zero sum public BP model, since there might be loss to player 1 generated by information spillover. As we showed with our main result 1 in this paper, for the model $\mathcal {G}(p^0)$ the difference between $\text {Cav}(v_A)(p^0_A) + \text {Cav}(v_B)(p^0_B) - \text {Cav}(\mathfrak {h})(p^0)$ cannot be interpreted similarly as the loss generated by information spillover, because $\text {Cav}(v_A)(p^0_A) + \text {Cav}(v_B)(p^0_B)$ might be attained in equilibrium.

6 Conclusion

We studied a three-player generalization of the Aumann et al. model and analysed the effects of information spillover on the equilibrium payoff set of the informed player. Our first two results provided a sufficient condition under which a continuum of equilibrium payoffs exist in the model and which implies, in particular, the existence of equilibria where the informed player circumvents the adverse effects of information spillover. These equilibria involve no signalling on equilibrium path. This sufficient condition is implied by the more interpretable local non-revelation condition. Our second main result presented a necessary condition for equilibria to attain the upper end of $I(p^0)$, which provides a restriction on the signalling processes that can be generated by such an equilibrium. A corollary of this result is that equilibria which “uncorrelate” the two two-player zero-sum games $G_A(p^0_A)$ and $G_B(p^0_B)$(whenever $I(p^0)$ is, of course, non-degenerate) do not achieve the upper end of $I(p^0)$.

Several questions remain unanswered with regards to the model $\mathcal {G}(p^0)$. What are the ex-ante equilibrium payoffs of the informed player that can be achieved through signalling on path of play? Is it possible, when $\mathcal{N}\mathcal{R}(p^0) = \emptyset$ and $I(p^0)$ is non-degenerate, that the upper end of $I(p^0)$ is achieved as an equilibrium payoff of the informed player? As our last example (Example 4.4) showed, it might be the case that only the lower end of $I(p^0)$ is achievable as an equilibrium payoff. Is it possible that an example of $\mathcal {G}(p^0)$ exists for which the upper end of $I(p^0)$ is not achievable, but something in the interior of $I(p^0)$ is an equilibrium payoff? These questions remain to be answered in future work.

Appendices

Appendix A

1.1 Proof of Theorem 3.6

Proof

For the proof we maintain the notation for the parametrization T, which is established before the statement of Theorem 3.6. Let $\phi _A$ and $p_A$ be the vector and probability distribution over $\Delta (K_A)$ respectively, given by the property NR at $p^0_A$. We will show that $\phi _A \in NR_A(p^0_A)$. We first show that $\phi _A$ satisfies (i) and (ii) of $NR_A(p^0_A)$. The vector $\phi _A$ satisfies (1) of Definition 3.5, which implies it satisfies (ii) of $NR_A(p^0_A)$ immediately. For (i), let $x \in \mathbb {R}^{|K_A|-1}, T(x) = p_A$ and $h \in \mathbb {R}^{|K_A|-1}$ such that $x+h \in P$. Because $\phi _AS$ satisfies (3) of Definition 3.5, we have that $\text {Cav}(v_A \circ T|_{P})(x)+ \phi _A S \cdot h \ge \text {Cav}(v_A \circ T|_{P})(x+h)$. From (1) in Definition 3.5 and the definition of $\text {Cav}(v_A)$, $(v_A \circ T|_{P})(x) + \phi _AS \cdot h \ge (v_A \circ T|_{P})(x+h)$. Again, from (1), the left hand side of the last inequality can be written as $\phi _A \cdot T(x) + \phi _AS \cdot h = \phi _A \cdot S(x+h) + \phi _A \cdot e^{K_A}_{1} = \phi _A \cdot T(x+h)$. Therefore, we have that $\phi _A \cdot T(x+h) \ge (v_A \circ T|_{P})(x+h)$, for any $h \in \mathbb {R}^{|K_A|-1}$ such that $x+h \in P$. Hence, $\phi _A \cdot q \ge v_A(q), \forall q \in \Delta (K_A)$. This proves $\phi _A$ satisfies (i).

It now remains to prove $\phi _A \in F_A$. We will show that $\partial v_A(p_A) \subseteq F_A$.²¹ Let $f = v_A \circ T$, $x \in P$ and $h \in \mathbb {R}^{|K_A|-1}$. Let $\Sigma (x)$ denote the set of optimal strategies of the maximizer in the one-shot zero-sum game with matrix A(T(x)) and analogously denote $\mathcal {T}(T(x))$ for the optimal strategies of the minimizer in the same game. Applying Proposition 3.4.2 in Laraki et al. (2019):

$$\begin{aligned} \lim _{\varepsilon \rightarrow 0^+} \frac{1}{\varepsilon }(f(x + \varepsilon h) - f(x)) = \max _{\sigma \in \Sigma (T(x))}\min _{\tau \in \mathcal {T}(T(x))} \sigma A(S h) \tau ' = \max _{\sigma \in \Sigma (T(x))}\min _{\tau \in \mathcal {T}(T(x))} (\sigma A^{k_A} \tau ')_{k_A \in K_A} \cdot Sh. \end{aligned}$$

Let B be the closed unit ball in $\mathbb {R}^{|K_A|-1}$. If f is differentiable at x, the above result implies that

$$\begin{aligned} \max _{h \in B} \min _{\phi _A \in F_A} (\nabla f(x) - \phi _A S) \cdot h \le 0. \end{aligned}$$

Applying the minmax theorem gives now that $\nabla f(x) \in F_A$. Let $T({\bar{x}}) = p_A$. By definition of the generalized gradient of f at ${\bar{x}}$, we have that $\partial v_A(p_A) \subseteq F_A$, as we wanted to show. $\square$

1.2 Proof of Proposition 3.9

Proof

The proof is divided in two-cases: (a) $\text {Cav}(v_A)(p^0_A) > v_A(p^0_A)$ and (b) $\text {Cav}(v_A)(p^0_A) = v_A(p^0_A)$. We start with (a). We first construct the candidate vector of payoffs $\phi _A \in \mathbb {R}^{|K_A|}$ and a probability distribution $p_A$ and show the pair $(\phi _A, p_A)$ satisfies the definition of NR at $p^0_A$.

Let $\text {Cav}(v_A)(p^0_A) = \sum _{s \in S}\lambda _s v_A(p_s)$, $\lambda _s \ge 0$, $\sum _{s \in S}\lambda _s =1$ and let $p_{s_0}$ be the posterior which is interior to the simplex of states. Let $\mathcal H= \{(q,\alpha ) \in \Delta (K_A) \times \mathbb {R}| \text {Cav}(v_A)(q) \ge \alpha \}$. Note that $\mathcal H$ is a convex subset of $\mathbb {R}^{|K_A|} \times \mathbb {R}$. Now, note that $(p^0_A, \text {Cav}(v_A)(p^0_A)) = \sum _{s \in S}\lambda _s(p_s, v_A(p_s))$. Since, by (a), $\forall s \in S, p_s \ne p_0$, it follows that $(p^0_A, \text {Cav}(v_A)(p^0_A))$ is not an extremum point of $\mathcal H$ (cf. Rockafellar (1970), Section 18). Therefore, there exists a face F of $\mathcal H$, with dimension $d \ge 1$, such that F contains $(p^0_A, \text {Cav}(v_A)(p^0_A))$ and each point $(p_s, v_A(p_s))$. Take now a supporting hyperplane H to $\mathcal H$ which contains F. The hyperplane H intersected with $\Delta (K_A) \times \mathbb {R}$ is the graph of an affine function $q \in \Delta (K_A) \mapsto \phi _A \cdot q \in \mathbb {R}$, where $\phi _A$ is a vector in $\mathbb {R}^{|K_A|}$. The candidate pair satisfying NR at $p^0_A$ is $(\phi _A, p_{s_0})$. We now check that it satisfies the conditions of Definition 3.5. It is clear from the construction that $\phi _A \cdot q \ge \text {Cav}(v_A)(q), \forall q \in \Delta (K_A)$. Using the definition of T, we can rewrite this inequality as $\text {Cav}(v_A \circ T|_{P})(x) + \phi _AS \cdot h \ge \text {Cav}(v_A \circ T|_{P})(x+h), T(x) = p_{s_0}, \forall h \in \mathbb {R}^{|K_A|-1}$ such that $x+h \in P$. We obtain $\phi _A S \in \partial ^* \text {Cav}(v_A)(p_{s_0})$. Since

$$\begin{aligned} \phi _A \cdot p_{s_0} = \text {Cav}(v_A)(p_{s_0}) = v_A(p_{s_0}) \qquad \qquad (*) \\ \text {Cav}(v_A)(p^0_A) = \phi _A \cdot p^0_A, \qquad \qquad (**) \end{aligned}$$

(1) in Definition 3.5 is satisfied. It remains to prove that $\phi _A S \in \partial v_A(p_{s_0})$. In order to show this, we are going to apply a result in Clark (1975).²² Let $v_A \circ T = f$. Note that f has directional derivatives in all directions (see Proposition 3.4.2 in Laraki et al. (2019)). We denote the directional derivative of f at point x in the direction h by $f'(x; h)$.

Notice now that from (*) and (**) we have that for each $h \in \mathbb {R}^{|K_A|-1}$, $(-\phi _A) S \cdot h \le (-f)'(x; h)$. Corollary 1.10 in Clark (1975) now implies that $-\phi _A S \in \partial (-f)$. Since we have that $\partial (f)(x)=-\partial (-f)(x)$, it follows immediately that $\phi _A S \in \partial v_A (p_{s_0})$. This concludes the proof of case (a).

Now we prove case (b). Assume $\text {Cav}(v_A)(p^0_A) = v_A(p^0_A)$. From Aumann et al. (1995), an optimal strategy $\sigma _A$ of the informed player in $G_A(p^0_A)$ is to play the mixed action which is optimal in the (one-shot) zero-sum game with matrix $A(p^0_A)$, independently at each stage, whereas the strategy of the uninformed player 2 is an approachability strategy $\tau _A$ at $p^0_A$. By definition of the approachability strategy, there exists a vector $\phi _A \in \mathbb {R}^{|K_A|}$ satisfying $\phi _A \cdot q \ge v_A(q), \forall q \in \Delta (K_A)$ such that player 2 approaches $\phi _A - \mathbb {R}^{|K_A|}_+$ and the profile $(\sigma _A, \tau _A)$ is a uniform equilibrium, with associated vector of payoffs equal to $\phi _A$. We now observe that the pair $(\phi _A, p^0_A)$ satisfies the conditions of NR at $p^0_A$. First, $\phi _A \cdot p^0_A = \text {Cav}(v_A)(p^0_A) = v_A(p^0_A)$ (which is condition (1)) is clear. By the same reasoning as in the first paragraph of this proof $\phi _A \cdot q \ge v_A(q), \forall q \in \Delta (K_A)$ is then equivalent to $\phi _AS \in \partial ^* \text {Cav}(v_A)(p^0_A)$, which is condition (3). The same reasoning as in the paragraph above (changing now $x = T(p_{s_0})$ to $x^0 = T(p^0_A)$) gives that $\phi _A \in \partial v_A(p^0_A)$, which is condition (2). This concludes the proof. $\square$

Appendix B

1.1 Proof of Lemma 4.3

Let $(\sigma , \tau _A, \tau _B)$ be an equilibrium profile of $\mathcal {G}(p^0)$. We start with the construction of the sequence of random variables $(p_t, \beta _{A,t}, \beta _{B,t})_{t \in \mathbb {N}}$. The sequence $(p_t)_{t \in \mathbb {N}}$ is the martingale of posteriors obtained from $(\sigma , \tau _A, \tau _B)$. It is therefore, immediately adapted to the sequence of increasing fields $(\mathcal H_t)_{t \in \mathbb {N}}$. Fix now a Banach limit $L: \ell ^{\infty } \rightarrow \mathbb {R}$.²³ A triple of strategies $(\sigma , \tau _{A}, \tau _{B})$ is an L-equilibrium if:

• (1) $L((\alpha ^{k_A, k_B}_T(\sigma , \tau _A, \tau _B))_{T \in \mathbb {N}}) \ge L((\alpha ^{k_A, k_B}_T(\sigma ',\tau _A, \tau _B))_{T \in \mathbb {N}}), \forall \sigma \in \Sigma , (k_A, k_B) \in K_A \times K_B$.

• (2) $L((\beta ^A_T(\sigma , \tau _A, \tau _B))_{T \in \mathbb {N}}) \ge L((\beta ^A_T(\sigma ,\tau '_A, \tau _B))_{T \in \mathbb {N}}), \forall \tau '_A \in \mathcal {T}_A$.

• (3) $L((\beta ^B_T(\sigma , \tau _A, \tau _B))_{T \in \mathbb {N}}) \ge L((\beta ^B_T(\sigma ,\tau _A, \tau '_B))_{T \in \mathbb {N}}), \forall \tau '_B \in \mathcal {T}_B$.

For notational convenience, we shall denote $L((u_t)_{t \in \mathbb {N}})$ by $L(u_t)$. A uniform equilibrium of the game $\mathcal {G}(p^0)$ automatically satisfies (1), (2) and (3) above, so it is an L-equilibrium. For each $t \in \mathbb {N}$, let $\beta _{A,t} = L(\mathbb {E}[\alpha _T|\mathcal {H}_t])$, where $\alpha _T = \frac{1}{T}\sum \nolimits _{t=1}^T A^{\kappa _A}_{i^t_A, j^t_A}$ and $\beta _{B,t} = L(\mathbb {E}[\beta _T|\mathcal {H}_t])$, where $\beta _T = \frac{1}{T}\sum \nolimits _{t=1}^T B^{\kappa _B}_{i^t_B,j^t_B}$. We now show (1) of Lemma 4.3: we have already argued that $(p_t)_{t \in \mathbb {N}}$ is a martingale and it is immediate it is bounded. Fix now $s <t$ and $h_s \in H_s$ with $\mathbb {P}(h_s)>0$: $\mathbb {E}[\beta _{A,t}|\mathcal {H}_s](h_s)$ = $\mathbb {E}[L(\mathbb {E}[\alpha _T| \mathcal {H}_t])| \mathcal {H}_s](h_s) = \frac{1}{\mathbb {P}(h_s)}\mathbb {E}[L(\mathbb {E}[\alpha _T| \mathcal {H}_t])\mathbbm {1}_{h_s}]$. Letting now $X_T:= \mathbb {E}[\alpha _T| \mathcal {H}_t]\mathbbm {1}_{h_s}$, we have that $X:= (X_T)_{T \in \mathbb {N}}: \Omega \rightarrow \ell ^{\infty }$ is a random variable that takes finitely many values in $\ell ^{\infty }$. Therefore, by Lemma 4.6 in Hart (1985), L commutes with $\mathbb {E}$ and we obtain: $\frac{1}{\mathbb {P}(h_s)}\mathbb {E}[L(X_T)] = \frac{1}{\mathbb {P}(h_s)}L(\mathbb {E}[X_T])$ = $\frac{1}{\mathbb {P}(h_s)}L(\mathbb {E}[\mathbb {E}[\alpha _T|\mathcal {H}_t]\mathbbm {1}_{h_s}]) = L(\frac{1}{\mathbb {P}(h_s)}\mathbb {E}[\mathbb {E}[\alpha _T|\mathcal {H}_t]\mathbbm {1}_{h_s}])$, where the last equality follows from L being linear. Now, $L(\frac{1}{\mathbb {P}(h_s)}\mathbb {E}[\mathbb {E}[\alpha _T|\mathcal {H}_t]\mathbbm {1}_{h_s}]) = L(\mathbb {E}[\mathbb {E}[\alpha _T| \mathcal {H}_t]| \mathcal {H}_s](h_s))$, from the definition of the conditional expectation on a finite field. As $h_s$ was arbitrarily chosen, we have $\mathbb {E}[\beta _{A,t}|\mathcal {H}_s] = L(\mathbb {E}[\alpha _T| \mathcal {H}_s]) = \beta _{A,s}$ a.s., proving $(\beta _{A,t})_{t \in \mathbb {N}}$ is a martingale adapted to $(\mathcal {H}_t)_{t \in \mathbb {N}}$ (boundedness of this martingale is immediate from the fact that payoffs are bounded). The proof that $(\beta _{B,t})_{t \in \mathbb {N}}$ is a bounded martingale adapted to $(\mathcal {H}_t)_{t \in \mathbb {N}}$ is the same. We now show condition (2) of Lemma 4.3. Notice that $\beta _{A,1} = L(\mathbb {E}[\alpha _T| \mathcal {H}_1]) = L(\mathbb {E}[\alpha _T]) = \lim _{T \rightarrow +\infty }\mathbb {E}[\alpha _T]$, where the last equality follows from the fact that payoffs of an equilibrium profile converge, by definition; the limit $\lim _{T \rightarrow +\infty }\mathbb {E}[\alpha _T]$ is precisely the expected payoff to player 1 in the game $G_A(p^0_A)$ from the equilibrium. By the same reasoning, $\beta _{B,1}$ is precisely the expected payoff of player 1 in game $G_B(p^0_B)$.

We now prove (3) from Lemma 4.3. Aiming for a contradiction, suppose there exists $h^0_t \in H_t$ with $\mathbb {P}(h^0_t) >0$, such that $\beta _{A,t}(h^0_t) > \text {Cav}(v_A)(p_{tA})(h^0_t)$. We will construct a profitable deviation for player 2. Let $\tau '_A$ be the following strategy. After $h^0_t$ has occurred, $\tau '_A$ will be equal to a Blackwell strategy $\hat{\tau }_A(p_t)$ for the uninformed player in $G_A(p_{t A})$, where $p_t:= p_t(h^0_t)$ is the posterior given $h^0_t$; otherwise, $\tau '_A$ is equal to $\tau _A$. We have,

$$\begin{aligned} \mathbb {E}[\alpha _T] - \mathbb {E}'[\alpha _T] = \mathbb {P}(h^0_t)(\mathbb {E}[\alpha _T|h^0_t] - \mathbb {E}'[\alpha _T|h^0_t]), \end{aligned}$$

where $\mathbb {E}'$ is the expectation taken with respect to the $p^0, \sigma , \tau '_A, \tau _B$. We claim that $L(\mathbb {E}'[\alpha _T|h^0_t]) \le \text {Cav}(v_A)(p_{tA}(h^0_t)), \forall t \in \mathbb {N}$. We will assume the claim for now and conclude the proof of (3). We then prove the claim. Taking L on the right-hand-side of the highlighted equation and using linearity of L, one gets: $\beta _{A,t}(h^0_t) - L(\mathbb {E}'[\alpha _T|h^0_t]) \ge \beta _{A,t}(h^0_t) - \text {Cav}(v_A)(p_{tA})(h^0_t)>0$. This implies a contradiction: since taking L on the left hand side we obtain the difference between the expected payoffs from $(\sigma ,\tau _A,\tau _B)$ to player 1 and the expected payoffs from $(\sigma ,\tau '_A,\tau _B)$ to player 1. As the expected payoffs to player 1 decreased strictly, player 2 is strictly better, which is a contradiction with the fact that $(\sigma ,\tau _A,\tau _B)$ is an equilibrium.

We now prove the claim: note that, $\mathbb {E}'[\alpha _T |h^0_t] = \frac{\mathbb {E}'[\alpha _T.\mathbbm {1}_{h^0_t}]}{\mathbb {P'}(h^0_t)} = \frac{1}{\mathbb {P}'(h^0_t)}\frac{1}{T}[\int _{h^0_t}\sum ^{t-1}_{s=1}A^{\kappa _A}_{i^s_A, j^s_A} dP'] + \frac{1}{\mathbb {P}'(h^0_t)}\frac{T-t}{T}[\int _{h^0_t} \frac{1}{T-t}\sum ^T_{s=t}A^{\kappa _A}_{i^s_A, j^s_A} dP']$. Let $C_T:= \frac{(T-t)}{T}$ and $K_T:= \frac{1}{\mathbb {P}'(h^0_t)}\frac{1}{T}[\int _{h^0_t}\sum ^{t-1}_{s=1}A^{\kappa _A}_{i^s_A, j^s_A} dP']$. Then, $\mathbb {E}'[\alpha _T |h^0_t] = \frac{1}{\mathbb {P}'(h^0_t)}C_T[\int _{h^0_t} \frac{1}{T-t}\sum ^T_{s=t}A^{\kappa _A}_{i^s_A, j^s_A} dP'] + K_T$. Now letting $P^{\circ }(\cdot ):= P'(\cdot | h^0_t)$ and $\mathbb {E}^{\circ }[\cdot ]$ the expectation operator with respect to $P^{\circ }(\cdot )$, we can re-write $\mathbb {E}'[\alpha _T |h^0_t]$ as $C_T \mathbb {E}^{\circ }[\frac{1}{T-t}\sum ^{T}_{s=t}A^{\kappa _A}_{i^s_A,j^s_A}] + K_T$. Let now $\varepsilon >0$. Note that because ${\hat{\tau }}_A(h^0_t)$ is a Blackwell strategy, there exits $T_0 \in \mathbb {N}$ s.t. $\forall T \ge T_0$, $\mathbb {E}^{\circ }[\frac{1}{T-t}\sum ^{T}_{s=t}A^{\kappa _A}_{i^s_A,j^s_A}] \le \text {Cav}(v_A)(p_{tA}) + \varepsilon$. Therefore, $C_T \mathbb {E}^{\circ }[\frac{1}{T-t}\sum ^{T}_{s=t}A^{\kappa _A}_{i^s_A,j^s_A}] + K_T \le C_T \text {Cav}(v_A)(p_{tA}(h^0_t)) + \varepsilon C_T + K_T$. Since $K_T \rightarrow 0$ and $C_T \rightarrow 1$ (as $T \rightarrow +\infty$), taking L on both sides of the inequality gives $L(\mathbb {E}'[\alpha _T| h^0_t ]) \le \text {Cav}(v_A)(p_{tA}(h^0_t)) + \varepsilon$. As $\varepsilon >0$ is arbitrary, $L(\mathbb {E}'[\alpha _T| h^0_t ]) \le \text {Cav}(v_A)(p_{tA}(h^0_t))$, as required. This concludes the proof of the claim and the proof of (3). The proof of (4) is the exact same.

Proof of the Claim 4.6

Fix $t,s \in \mathbb {N}$ with $s \ge t$. Conditioning over $\mathcal {H}_{s+1}$ and $\kappa _A$, we have

$$\begin{aligned} \mathbb {E}[A^{\kappa _A}_{i^s_A,j^s_A} | \mathcal {H}_t] = \mathbb {E}[ \sum _{k_A \in K_A}p^{k_A}_{s+1 A}A^{k_A}_{i^s_A,j^s_A} | \mathcal {H}_t] = \sum _{k_A \in k_A}p^{k_A}_{tA}\mathbb {E}[A^{k_A}_{i^s_A, j^s_A} | \mathcal {H}_t] + \sum _{k \in K}\mathbb {E}[(p^{k_A}_{s+1A} - p^{k_A}_{tA}) A^{K_A}_{i^s_A, j^s_A} | \mathcal {H}_t] = \end{aligned}$$

$$\begin{aligned} =\mathbb {E}[A(p_{tA})_{i^s_A,j^s_A} | \mathcal {H}_t] + \sum _{k_A \in K_A}\mathbb {E}[(p^{k_A}_{s+1A} - p^{k_A}_{tA}) A^{k_A}_{i^s_A, j^s_A} | \mathcal {H}_t] \le V_A(p_{tA}) + \sum _{k_A \in K_A}\mathbb {E}[(p^{k_A}_{s+1A} - p^{k_A}_{tA}) A^{k_A}_{i^s_A, j^s_A} | \mathcal {H}_t]. \end{aligned}$$

Summing over $s \in \mathbb {N}$ from $t \le s \le T$, and dividing by T, we have

$$\begin{aligned} \mathbb {E}[\alpha _T | \mathcal {H}_t] \le \frac{t}{T} + V_A(p_{tA}) + \frac{1}{T}\sum _{t \le s \le T}\sum _{k_A \in K_A} \mathbb {E}[|p^{k_A}_{s+1A} - p^{k_A}_{tA}||\mathcal {H}_t], \end{aligned}$$

where we used the fact that stage payoffs at any state are bounded 1. Denote $Z_t:= \frac{t}{T} + \frac{1}{T}\sum _{t \le s \le T}\mathbb {E}[ \pi _t |\mathcal {H}_t]$, where $\pi _t:= \sum _{k_A \in K_A}\sup _{s \ge t}|p^{k_A}_{s+1A} - p^{k_A}_{tA}|$. Taking Banach-limits (on T) on both sides, we have $\beta _{A, t} \le V_A(p_{tA}) + \mathbb {E}[\pi _t | \mathcal {H}_t]$ a.s.. Since $p_t \rightarrow p^{\infty }$ a.s., as $t \rightarrow \infty$, it follows by Lemma 4.24 in Hart (1985), that $\mathbb {E}[\pi _t | \mathcal {H}_t] \rightarrow 0$ a.s.. $\square$