Correlated equilibria of two person repeated games with random signals

Abstract

In this work we extend a result of Lehrer (Math Oper Res 17(1):175–199, 1992a) characterising the correlated equilibrium payoffs in undiscounted two player repeated games with partial monitoring to the case in which the signals are permitted to be stochastic. In particular, we develop appropriate versions of Lehrer’s concepts of “indistinguishable” and “more informative.” We also show that any individually rational payoff associated with a (correlated) distribution on pure action profiles in the stage game such that neither player can profitably deviate from one of his actions to another that is indistinguishable and more informative is the payoff of a correlated equilibrium of the infinitely repeated game.

Appendix: The approachability theorem

In this appendix we give some of the basic results concerning matrix games with vector payoffs. The results are due to Blackwell (1956) and our treatment follows (in a less general setting) that of Mertens et al. (2015).

As before we consider a finite stage game with pure action sets $S^1$ and $S^2$ and mixed action sets $X^1$ and $X^2$. Rather than having a payoff associated with each pair of actions we assume that there is a function $\varphi$ from $S=S^1 \times S^2$ to the set of probability distributions over some finite subset of ${\mathbb R}^k$. The game is played more or less as before. At each stage n player i chooses an action in $S^i$ and then a point $g_n$ is chosen at random according to $\varphi (s^1_n,s^2_n)$. Both players then obtain some signal that reveals for player 1, at least, $g_n$. We let $\bar{g}_n = \frac{1}{n}\sum _{t=1}^n g_t$.

Definition 11

(Approachable) A set C in ${\mathbb R}^k$ is said to be approachable by player 1 if there is a strategy for player 1 in the infinitely repeated game for which $d(\bar{g}_n,C)$ converges to zero almost surely.

Let $f(s^1,s^2)$ be the expected value of $\varphi (s^1,s^2)$ and for any $x^1$ in $X^1$ let $Z(x^1)$ be the convex hull of the points in $\{\sum _{s^1\in S^1} x^1(s^1)f(s^1,s^2) \mid s^2 \in S^2\}$.

Theorem 3

(The approachability theorem) Let C be any closed set in ${\mathbb R}^k$. Suppose that for any g not in C there is $x^1$ $({=}x^1(g))$ in $X^1$ such that the hyperplane through h(g) a closest point in C to g perpendicular to the line segment between g and h(g) separates g from $Z(x^1(g))$. Then C is approachable by player 1 using strategy $\sigma ^1(\cdot ),$ a strategy depending on the history only through $\bar{g}_n$ where

$$\begin{aligned} \sigma ^1(\bar{g}_n) ={\left\{ \begin{array}{ll} x^1(\bar{g}_n) &{}\quad \text {if } n>0 \text { and } \bar{g}_n \not \in C, \\ \text {arbitrary} &{}\quad \text {otherwise}. \end{array}\right. } \end{aligned}$$

With that strategy, 

$$\begin{aligned} E(d(\bar{g}_n,C)^2) \le 4K/n \end{aligned}$$

(1)

and

$$\begin{aligned} P\left( \sup _{n \ge N} d(\bar{g}_n,C)\ge \varepsilon \right) \le 8K/(\varepsilon ^2 N). \end{aligned}$$

(2)

where K is a bound on the second order moments of $\varphi (s^1,s^2)$ for all $s^1$ and $s^2$.

We need, in fact, only one relatively simple implication of the approachability theorem.

Corollary 1

(Mertens et al. 2015, Corollary II.4.4) For any $x^1$ in $X^1$ the set $Z(x^1)$ is approachable by player 1 using the constant strategy $\sigma ^1(\cdot )=x^1.$ And,  again,  with this strategy inequalities (1) and (2) hold.