Uniformly Strict Equilibrium for Repeated Games with Private Monitoring and Communication

Cooperation through repetition is an important theme in game theory. In this regard, various celebrated ``folk theorems'' have been proposed for repeated games in increasingly more complex environments. There has, however, been insufficient attention paid to the robustness of a large set of equilibria that is needed for such folk theorems. Starting with perfect public equilibrium as our starting point, we study uniformly strict equilibria in repeated games with private monitoring and direct communication (cheap talk). We characterize the limit equilibrium payoff set and identify the conditions for the folk theorem to hold with uniformly strict equilibrium.


Introduction
Cooperation through repetition is an important theme in game theory.In this regard, various celebrated "folk theorems" have been proposed for repeated games in increasingly more complex environments.There has, however, been insufficient attention paid to the robustness of a large set of equilibria that is needed for such folk theorems.
In this paper, we study uniformly strict equilibria in repeated games with private monitoring and direct communication (cheap talk).Our starting point is perfect public equilibrium (PPE) ( [9]).In each period, players take actions simultaneously, observe private signals, and send public messages simultaneously.A perfect public equilibrium is a profile of public strategies that specifies a Nash equilibrium as their continuation play after every public history (a sequence of past message profiles).We impose strict incentives at every public history by requiring that, in each period, a player would incur a positive payoff loss (in terms of the value at the period) when deviating in either action or message from the equilibrium strategy.We also require such payoff losses from a unilateral deviation to be uniformly bounded away from 0 across all public histories.
It is well known that strict equilibrium has desirable robustness properties.For example, strict equilibria survive most equilibrium refinements in strategic form games.In our setting of infinitely repeated games, our uniform strictness requirement is a natural strengthening of strict equilibrium.
We present two main results.Our first result is a characterization of the limit set of uniformly strict perfect public equilibrium payoffs via a collection of static programming problems.We follow the approach of Fudenberg and Levine [7] (henceforth FL) to characterize the limit equilibrium payoff set.It also builds on other classic results from Abreu, Pearce and Stacchetti [1] and Fudenberg, Levine and Maskin [9].We adapt their ideas to our model and generalize them by introducing uniformly strict incentives.In our second result, we establish a folk theorem by identifying conditions ensuring that this limit set coincides with the set of feasible and individually rational payoffs generated by the data of the underlying stage game.
There is a large literature dealing with folk theorems for repeated games with varying assumptions regarding public or private monitoring with or without communication.Most relevant to our paper are the various folk theorems for repeated games with private monitoring and communication ( [2], [3], [5], [8], [12], [13], [14], [18], [23]). 1   1 There is an extensive literature on folk theorems for repeated games with private mon-Our detectability and identifiability conditions for the folk theorem are similar to and weaker than the conditions (A2) and (A3) in Kandori and Matsushima [12].(A2) and (A3) imply that, for any pair of players, their deviations are detectable and identifiable (i.e. one player's deviation can be statistically distinguished by the other player's deviation) based on the private signals of the other n−2 players.Our detectability and identifiability condition instead impose a similar restriction on the joint distributions of the messages of all players.Their conditions allow each player's future payoff independent of her message.This indifference makes truth-telling incentive compatible for each player.On the other hand, we require uniform strictness of incentive for sending any (nontrivial) message. 2  Our conditions are also similar to the sufficient conditions in Tomala [23], but Tomala studies a type of perfect equilibrium with mediated communication, which is more flexible than cheap talk, and does not impose strict incentive constraints.As a consequence, the conditions for the folk theorem in Tomala [23] are weaker than ours.

Repeated Games with Private Monitoring and Communication Stage Game
We present the model of repeated games with private monitoring and communication.The set of players is N = {1, ..., n}.The game proceeds in stages and in each stage t, player i chooses an action from a finite set A i .An action profile is denoted by a = (a 1 , ..., a n ) ∈ Π i A i := A. Stage game payoffs are given by g : A → R n .We denote the resulting stage game by G = (N, A, g) .Actions are not publicly observable.Instead, each player i observes a private signal s i from a finite set S i .A private signal profile is denoted is the distribution on S given action profile a.We assume that the marginal distributions have itoring and without communication, including [4], [6] , [11], [15], [16], [17], [19], [21], [22].They usually rely on non-strict equilibrium (such as belief-free equilibrium) to establish the folk theorem.
2 [12] also discusses a way to provide strict incentive for truth-telling via a scoring rule, but the strict incentive vanishes in the limit for the minmax points for their folk theorem (Theorem 2).We instead fix the level of the required strict incentive first, then prove the folk theorem by letting δ → 0.
full support, that is, s −i p (s i , s −i |a) > 0 for all s i ∈ S i , a ∈ A and i ∈ N.
Players communicate publicly each period.Player i sends a public message m i from a finite set M i after observing a private signal s i in each period. 3layer i's message strategy ρ i : S i → M i in the stage game is a mapping from private signals to public messages.Let R i be the set of player i's message strategies.An action profile a ∈ A and a profile of message strategies We normalize payoffs so that each player's pure strategy minmax payoff is 0 in the stage game.The pure strategy minmax payoff is the relevant payoff lower bound for our folk theorem because we study equilibrium with strict incentives and without any mediator.Note that the pure strategy minmax payoff may be strictly larger than the mixed minmax payoff.The set of feasible payoff profiles is is the set of feasible, individually rational payoff profiles.

Repeated Game with Public Communication
In the repeated game, play proceeds in the following way.At the beginning of period t ≥ 1, player i chooses an action contingent on (h t i , h t ), where is player i's private history that consists of her private actions and private signals and h t ∈ H t = M t−1 is the public history of message profiles. 4Player i also chooses a message strategy ρ i ∈ R i contingent on (h t i , h t , a i ).Then player i's pure strategy For the sake of simple exposition, we drop h t i from any public strategies.We denote player i's action and on-path message strategy at h t for public strategy σ i by σ where the expectation is taken with respect to this measure.

Uniformly Strict Perfect Public Equilibrium
A profile of public strategies σ is a perfect public equilibrium if its continuation strategies constitute a Nash equilibrium after every public history( [9]).In this paper, we impose an additional robustness requirement by requiring uniformly strict incentive compatibility at every public history.Let w σ (h t ) be a profile of discounted average continuation payoffs at public history h t ∈ H given a public strategy profile σ.Given σ and h t , let Σ σ,h t i be the set of deviations changes the distribution of continuation payoff profiles w σ (h t , •) from period t + 1.We call such one-shot deviations nontrivial deviations at h t with respect to w σ .Any other one-shot deviation is called a trivial deviation, as it does not change any outcome in the current period and in the future at all.
Definition 1 (η-USPPE) A profile of public strategies σ is an η -uniformly strict perfect public equilibrium for η ≥ 0 if the following conditions are satisfied for any h t ∈ H for any This condition means that player i would lose at least η at any public history if she makes any nontrivial deviation. 5, 6 This definition just checks the one-shot deviation constraints at each public history, but all the incentive constraints for the continuation game after each public history are satisfied, because the one-shot deviation principle holds.
Note that η-USPPE σ may assign a suboptimal message off-path, i.e. σ m i (h t , a ′ i ) may not be an optimal message strategy when a ′ i = σ a i (h t ), since it is just a Nash equilibrium.But we can replace them with an optimal message to obtain a sequential equilibrium that is realization equivalent to σ, because other players never learn about player i's deviation to a ′ i due to the full support assumption. 7s an example of η-USPPE, consider any stage game with an η-strict Nash equilibrium.Then repeating this η-strict Nash equilibrium and sending some message independent of histories is an η-uniformly strict PPE. 8 In the following, let E η (δ) ⊂ R n denote the set of all η-USPPE payoff profiles given δ.In general, η-USPPE may not exist, hence E η (δ) may be an empty set.The equilibrium payoff set for the standard PPE is compact, but the compactness of E η (δ) may not be entirely obvious because the set of nontrivial deviations at each public history depends on the continuation payoff profile.However, we can show that E η (δ) is compact.
Proof.See the Appendix.
3 Characterization of Limit Equilibrium Payoff Set

Constructing The Bounding Set for Equilibrium Payoffs
We characterize the limit η-USPPE payoff set in two steps.In this subsection, we construct a compact set Q η with the property that E η (δ) ⊆ Q η for all δ ∈ (0, 1).In the next subsection, we show that, if intQ η = ∅, then for any ǫ > 0, there exists a nonempty, compact, convex set W ⊆ Q η and δ ∈ (0, 1) such that W ⊆ E η (δ) for any δ ∈ (δ, 1) and the Hausdorff distance between W and Q η is less than ǫ.
does not change the distribution of x(•) given (a −i , ρ −i ).Naturally, we call such a deviation nontrivial deviation with respect to x for (P λ,η ).
Since the value of the problem is bounded above by max a∈A λ • g(a), it is either some finite value or −∞ when there is no feasible solution.
This programming problem is different from FL's problem in [7] in two aspects because of our uniform strictness requirement.First, an η -wedge is added to the incentive constraints for nontrivial deviations (with respect to x(•)).Secondly, we restrict attention to pure actions because uniformly strict equilibrium must be in pure strategies by definition.Note that this problem is independent of δ like FL's problem.
Let k η (λ) denote the vale of the supremum for (P λ,η ).Let H η (λ) = {x ∈ R n |λ • x ≤ k η (λ)} be the half space below the hyperplane λ The next theorem shows that Q η is a bound of the equilibrium payoff set given any η and δ.
Theorem 3 For any η ≥ 0 and any δ ∈ (0, 1), trivially true.So suppose that E η (δ) = ∅ and recall that E η (δ) is a nonempty compact set by Lemma 2. Fix any η ≥ 0 and pick any λ ∈ Λ.Let v * be the η-uniformly strict PPE payoff profile that solves max v∈E η (δ) λ • v. Let σ * be the equilibrium strategy profile to achieve v * and (a * , ρ * ) ∈ A × R be the equilibrium action profile and the message strategy profile in the first period.Since σ * is an η-USPPE, it must satisfy the following conditions: x * ) satisfies all the η-strict incentive compatibility constraints with respect to the set of nontrivial deviations Σσ * (h 1 ),x * i for player i in the programming problem (P λ,η ).Finally, Since this is true for all λ ∈ Λ, we have E η (δ) ⊂ λ∈Λ H η (λ) = Q η for any δ ∈ (0, 1).
The following lemma, which corresponds to Lemma 3.2. in FL, is useful to assess the possibility of a uniformly strict folk theorem.
Lemma 4 k η (−e i ) is bounded above by −v η i , where Proof.Suppose that (v, a, ρ, x) is feasible for problem (P −e i ,η ).The last constraint of the problem becomes x i (m) ≥ 0 ∀m ∈ M. Then player i's payoff is bounded from below by g i (a).By the η-strict incentive constraint, v i is also bounded from below by max This v η i coincides with the minmax payoff 0 when η = 0, but can be strictly positive when η > 0. As the next lemma shows, it coincides with the minmax payoff if and only if there exists a minmax action profile for player i where player i plays an η-strictly optimal action.
Proof.Fix i and choose any a ∈ A. If a i is a best response to a −i , then If not, then max Suppose that a i ∈ A satisfies the conditions of the Lemma.Then g i (a i ) = 0 and max a ′ Hence v η i = 0 is achieved at a = a i .Suppose that there is no such a i ∈ A. Then η must be strictly positive since the minmax action profile would satisfy the conditions when η = 0. Take any When a i is not a best response to a −i , then max

If this bound v η
i is strictly positive, then k(−e i ) = −v η i < 0, hence the minmax payoff can never be approximated by any η-USPPE by Theorem 3. So, it is necessary for a folk theorem that an η-strict incentive is provided by the current payoffs at the minmax point.Also note that this bound may be achieved by some non-minmax action profile when it is strictly positive.If no minmax action profile for player i is η-strictly optimal for i, then some non-minmax action profile â ∈ A may achieve v η i > 0 if g i (â) is close to 0 and max a ′ i =â i g i (a ′ i , â−i ) + η is small as well (any deviation from â is very costly for player i).
Similarly, we observe that k η (e i ) may be strictly below max a g i (a) unless it is η-strictly optimal for player i to play the action that achieves this value.Otherwise, an additional incentive needs to be provided for player i through some punishment (as λ = e i ).This necessarily leads to some inefficiency because punishment occurs with positive probability (Green and Porter [10]).Thus it is necessary for a folk theorem that g i (a) − g i (a ′ i , a −i ) ≥ η holds for every a ′ i = a i for some action profile a that solves max a g i (a).If this is not the case, then k η (e i ) must be less than max a g i (a) and may be achieved by some action profile that does not solve max a g i (a).

Decomposability and Local Decomposability
Our main theorem claims that Q η provides the limit η-USPPE payoff set when Q η has an interior point.We prove it by establishing η-uniformly strict versions of many well-known results in [7] and in [1].
We first observe that a set of payoffs can be supported by η-USPPE if it is self-decomposable with respect to η-strict incentive constraints with respect to nontrivial deviations.Given δ ∈ (0, 1) and w : M → R n , we consider the static game Γ δ (G, p, w), where player i's strategy set is where w assigns payoffs for each message profile and the expectation is computed with respect to p(•|a, ρ).Definition 6 A pair consisting of an action profile a ∈ A and a profile of message strategies ρ ∈ R is η-enforceable for η > 0 with respect to nonempty set W ⊂ R n and δ ∈ (0, 1) if there exists a function w : M → W such that, for all i ∈ I, is the set of nontrivial deviations from (a, ρ) for player i with respect to w.
for some η-enforceable pair (a, ρ) and w : M → W , then we say that v is η-decomposable and that ((a, ρ), w) η-decomposes v with respect to W and δ.Define the set of ηdecomposable payoffs with respect to W and δ as follows: We say that W is η-self decomposable with respect to δ if W ⊂ B (δ, W, η).
It is easy to see that a "uniformly strict" version of Theorem 1 in Abreu, Pearce, and Stacchetti [1] holds here: if W is η-self decomposable with respect to δ, then every v ∈ B (δ, W, η) can be supported by some η-USPPE.Since the following lemma follows easily from the result in Abreu, Pearce and Stacchetti, its proof is omitted.
For the rest of this subsection, we prove that local η-self decomposability of W implies η-self decomposability of W .In the framework of repeated games with imperfect public monitoring, Fudenberg, Levine, and Maskin ( [9]) introduced a notion of local self decomposability that is sufficient for self decomposability.Here we prove the corresponding lemma in our setting.We begin with a lemma that establishes a certain monotonicity property of B. It implies that, if W is η-self decomposable with respect to δ ∈ (0, 1) , then W is η-self decomposable for every δ ′ ∈ (δ, 1).
Next we introduce local η-self decomposability and show that local η-self decomposability implies η-self decomposability for sufficiently large discount factors.
Definition 9 A nonempty set W ⊆ R n is locally η-self decomposable if, for any v ∈ W, there exists δ ∈ (0, 1) and an open set U containing v such that U ∩ W ⊂ B (δ, W, η) .

Local Decomposability of a Smooth Set in The Bounding Set
We call a nonempty compact and convex set in R n smooth if there exists a unique supporting hyperplane at every boundary point of the set.
The following lemma shows that, if Q η has an interior point in R n , then there exists a smooth, compact and convex set in intQ η that is arbitrarily close to Q η .
Lemma 11 Suppose that Q η ⊆ R n has an interior point.Then, for every ε > 0, there exists a smooth compact and convex set W ′ ⊂ intQ η such that the Hausdorff distance between W ′ and intQ η is at most ε.
Proof.Choose any ε > 0. Since bounded sets in Euclidean space are totally bounded, there exists a finite set Z ⊆ intQ η such that, for each v ∈ intQ η , there exists z ∈ Z such that z − x < ε.Let W = coZ.Then W is nonempty, compact and convex.Since Hence the Hausdorff distance between W and intQ η is at most ε.
Next we construct W ′ from W . Since W ⊆ intQ η is a polyhedron and has only a finite number of vertices, we can find a small enough ǫ ′ > 0 such that, at every v ∈ W , the closed ball , the Hausdorff distance between W ′ and intQ η is at most ε.
Next we show that W ′ is a smooth compact convex set.To show that W ′ is compact, it suffices to show that W ′ is closed.Suppose that w k ∈ W ′ for each k and {w k } is convergent with limit w * .For each k, there exists To show that W ′ is convex, choose any x, y ∈ W ′ .Then there exists Finally, to see that W ′ has a unique supporting hyperplane at every boundary point, first note that every boundary point of W ′ must be a boundary point of B ǫ ′ v for some v ∈ W . Since a supporting hyperplane of a boundary point of W ′ must be a supporting hyperplane of B ǫ ′ v at the same point and B ǫ ′ v cannot have multiple supporting hyperplanes at any boundary point, the supporting hyperplane must be unique at every boundary point of W ′ .
We now show that any such set W ′ that approximates Q η from the inside is η-locally decomposable, which leads to our main result by Lemma 10.
We need two technical lemmas for local decomposability.
Lemma 12 A smooth, compact and convex set C ⊆ R n has non-empty interior in R n .
Proof.Suppose otherwise.Then the affine hull of C has dimension less than n.Let S denote the affine hull and, translating if necessary, we may assume that 0 ∈ C and S is a vector subspace of R n .Since C is smooth, C is not a singleton set.So we can find are distinct hyperplanes because the former does not include 0 (since p • x * ≥ p • p > 0), while the latter does.
Lemma 13 Let W ⊂ R n be a smooth, compact and convex set and let v be a boundary point of W .Let λ v = 0 ∈ R n be a normal to the unique supporting hyperplane of W at v, i.e., x for all x ∈ W and, for any y ∈ R n such that λ v •v > λ v •y, there exists α * ∈ (0, 1) such that (1 − α) v+αy ∈ intW for any α ∈ (0, α * ), then there is the unique supporting hyperplane of W at v and λ v is its normal vector.
Proof.Translating if necessary, we may assume that v = 0. We argue by contradiction.Suppose that there exists y ∈ R n such that λ v • y < 0 but for each α * ∈ (0, 1) there exists α ∈ (0, α * ) such that αy / ∈ intW.Then there exists a sequence {α k } such that 0 < α k < 1, α k → 0 and α k y / ∈ intW for each k.Since intW is non-empty by the previous lemma and convex, there exists for each k a q k = 0 such that for all x ∈ intW by the separating hyperplane theorem.Let k → ∞ and q k ||q k || → q for some q = 0, extracting a subsequence if necessary.Then q • x ≤ 0 for all x ∈ intW.Then q • x ≤ 0 for all x ∈ W , hence q is a normal vector for a supporting hyperplane of W at v = 0. To derive a contradiction, we show that q = βλ v for all β > 0. To see this, take any z ∈ intW and note that α 2 k z ∈ intW (since 0 ∈ W ). Therefore which satisfies λ ′ • x ≤ 0 for any x ∈ W . Then there must exist y ′ ∈ R n such that λ ′ • y ′ > 0 and λ v • y ′ < 0. Then αy ′ is in W for small enough α ∈ (0, 1) by assumption, but λ ′ • (αy ′ ) > 0, which is a contradiction.Hence there is the unique supporting hyperplane of W at v = 0 and λ v is its normal vector.
Theorem 14 Suppose that Q η has an interior point in R n .For any ǫ > 0, there exists a smooth, compact and convex set W ⊆ intQ η and δ ∈ (0, 1) such that W ⊂ E η (δ) for any δ ∈ (δ, 1) and the Hausdorff distance between W and Q η is at most ǫ.
Proof.By Lemma 11, there exists a smooth, compact and convex set W in intQ η such that the Hausdorff distance between W and intQ η is at most ǫ.Since cl(intQ η ) = Q η , it follows that the Hausdorff distance between W and Q η is at most ǫ.By Lemma 10, it suffices to show that W is locally η-self decomposable.Take any boundary point w * of W . Since W is smooth, there is unique λ * ∈ Λ such that λ * • w * = max w∈W λ * • w.Since W ⊆ intQ η , there exists a feasible solution (v, (a, ρ), x) for the programming problem Then (w * , (a, ρ), x ′ ) is feasible in the programming problem because the η-strict incentive constraints for nontrivial deviations are not affected, For each δ and m, define w δ (m) by w δ (m) := w * + 1−δ δ x ′ (m).If (a, ρ) is played and w δ is used as the continuation payoff profile, then, for all (a ′ i , ρ Next we show that w δ (m) is in intW for every m if δ is large enough.Since W is smooth, λ * is a normal vector of the unique supporting hyperplane of W at w * .Choose any δ ′ ∈ (0, 1).Since for any δ, δ ′ ∈ (0, 1) by definition, we have for δ ∈ (δ ′ , 1).Then, for each m, there exists δ m ∈ (0, 1) such that w δ (m) ∈ int(W ) for any δ ∈ (δ m , 1) by Lemma 13.Let δ = max m δ m .Then ((a, ρ), w δ ) η-decomposes w * with respect to int(W ) and δ for any δ > δ.

Uniformly Strict Folk Theorem
In this section, we prove a folk theorem with η-uniformly strict PPE by showing that Q η coincides with V * (G) under certain conditions.In the following, we use φ i to denote a mixed strategy over and let p−i (•|φ j , (a −j , ρ −j )) be the marginal distribution of p(•|φ j , (a −j , ρ −j )) over M −i .
We need the following four conditions on the private monitoring structure and the payoff functions for our folk theorem.Recall that A(G) ⊆ A is the set of action profiles that generate an extreme point in V (G).
Definition 15 (η-detectability) For each a ∈ A(G), there exists ρ ∈ R that satisfies the following condition: This condition means that if player i's unilateral deviation to a mixed strategy (with 0 probability on (a i , ρ i )) cannot be detected, then she must lose at least η in terms of the stage-game expected payoff When we approximate the minmax point, we need a slightly stronger detectability condition.
This condition means that the above η-detectability condition holds for j = i without using player i's message.
The next condition means that, if player i's deviation is not linearly independent from some other player's deviation, then she must lose at least η in terms of the stage-game expected payoff.
Definition 17 (η-identifiability) For each a ∈ A(G), there exists ρ ∈ R that satisfies the following condition: for each pair i The last conditions require that, for every player i, there exists the best action profile and the minmax action profile, where player i would lose at least η by deviating to any other pure action.Remember Lemma 4 and our discussion following the lemma; we know that they are necessary for the folk theorem.
Definition 18 (η-best response property) G satisfies η-best response property for a i , a i , i ∈ N ⊂ A if the following conditions are satisfied for any i ∈ N: It may be useful to compare these conditions to the similar conditions (A1)-(A3) for Theorem 1 in [12].(A1) requires 0 * -detectability condition at the minmax action profile.We instead assume η * -detectability condition at the minmax action profile and the best action profile for each player.Kandori and Matsushima [12] assumes (A2) and (A3) for every action profile in A(G), which is a restriction on the distribution of the private signals of any subset of n − 2 players, whereas we assume η-detectability and η-identifiability for every a ∈ A(G), which is a restriction on the joint distribution of all messages.η-detectability and η-identifiability are weaker than (A2) and (A3) when the message space is rich enough in the following sense.For η-identifiability, if (A2) and (A3) are satisfied at a and M i = S i for every i ∈ N, then η-identifiability is automatically satisfied with truthful message strategies, since the type of linear dependency that appears in the definition of η-identifiability would never occur given (A2) and (A3).For the same reason, (A2) implies η-detectability for every a ∈ A(G).
As an example of monitoring structure that satisfies our conditions, consider p that satisfies the individual full rank condition for each player with respect to the other players' signals and the pairwise full rank condition for every pair of players with respect to the private signals of the other n − 2 players.Then (A2) and (A3) are satisfied.In addition, η * -detectability is trivially satisfied.Hence all our conditions on the monitoring structure (η *detectability and η-detectability & η-identifiability) are satisfied.
Using these conditions, we can state our folk theorem with η-USPPE using as follows.
Theorem 19 Fix any private monitoring game (G, p).Suppose that intV * (G) = ∅ and G satisfies η-best response property for a i , a i , i ∈ N ⊂ A. If (G, p) satisfies both η-detectability and η-identifiability with the same ρ a ∈ R for each a ∈ A(G) and satisfies η * -detectability at a i and a i with respect to i for every i ∈ N, then, for any ǫ > 0, there exists a smooth, compact and convex set W ⊆ intV * (G) and δ ∈ (0, 1) such that W ⊂ E η (δ) for any δ ∈ (δ, 1) and the Hausdorff distance between W and V * (G) is at most ǫ.
We prove this theorem through a series of lemma.We first observe that η-detectability is equivalent to the existence of a transfer x that guarantees η-strict incentive compatibility.
Lemma 20 (G, p) satisfies η-detectability if and only if for any a ∈ A(G), there exists ρ such that there exists x i : M → R for each i ∈ N that satisfies Since the proof for this result is standard, it is omitted. 9By the same argument, we can show that η * -detectability with respect to i is equivalent to the existence of a transfer that does not depend on player i's message and guarantees η-strict incentive for every player other than i.
Lemma 21 (G, p) satisfies η * -detectability for i with respect to a ∈ A if and only if there exists a ρ and, for each j = i, a function x j : The next lemma shows that k η (λ) is equal to max a∈A λ • g(a) for any regular λ (with at least two nonzero elements) when η-detectability and ηidentifiability are satisfied.
Proof.Pick any a λ ∈ A(G) that solves max a∈A i λ i g i (a).By assumption, there is the same ρ λ ∈ R for which the conditions for η-detectability and ηidentifiability are satisfied at a λ .We show that there exists x : M → R n satisfying the following conditions: This implies k η (λ) = i λ i g i (a λ ) because ((a λ , ρ λ ), x) is feasible for the problem (P λ,η ) and achieves the upper bound i λ i g i (a λ ), hence is clearly a maximum point for (P λ,η ).Note that every on-path deviation is a nontrivial deviation with respect to transfer x we find.The existence of such (x i ) i∈N is equivalent to the feasibility of the following linear programming problem (with value 0): The dual problem of this problem is: where q i ((a ′ i , ρ ′ i )) ≥ 0 is the multiplier for the strict incentive constraint for (a ′ i , ρ ′ i ) = (a λ i , ρ λ i ) and d(m) ∈ R is the multiplier for the λ-"budget balancing" condition for m ∈ M.
By the strong duality theorem, the value of the primal problem is 0 if and only if the value of the dual problem is 0. Take any (q, d) that is feasible for the dual problem.For each i, we consider two cases.First suppose λ i = 0. Then the following holds for all m ∈ M: . Note that the ith term of the objective function can be written as which is bounded above by 0 by η-detectability.
Next suppose that λ i = 0. Then there exists j such that λ j = 0 since λ / ∈ {±e i , i ∈ N}.Consequently, for all m ∈ M we have p(m|(a λ , ρ λ )) − p(m|(a ′ j , ρ ′ j ), (a λ −j , ρ λ −j )) q j ((a ′ j , ρ ′ j )) = λ j d(m) If d(m) = 0 for all m, then we can apply the same argument as before to show that the ith and jth terms of the objective function are at most 0. If d(m) = 0, then q i is not identically 0 nor is q j identically 0. So we can "cross multiply" the two equalities, cancel d(m) and conclude that, for all m ∈ M, where φ ′ i and φ ′ j are defined by φ , a λ −j ) − g j (a λ ) + η are bounded above by 0. This implies that the ith term (and the jth term) of the objective function are bounded above by 0.
Hence the ith term of the objective function is bounded above by 0 in either case for any feasible (q, d), implying that the value of the dual problem is bounded above by 0 for any feasible (q, d).Since 0 can be achieved by q(•) = 0 and d(•) = 0, the value of the dual problem is exactly 0 as we wanted to show.
The next lemma shows that η-best response property and η * -detectability with respect to i is sufficient to guarantee k η (e i ) = max a g i (a) and k η (−e i ) = 0.
Lemma 23 Suppose that G satisfies η-best response property for a i , a i , i ∈ N ⊂ A. Then the following holds for each i ∈ N.
• If (G, p) satisfies η * -detectability with respect to i at a i , then k η (e i ) = max a g i (a).
• If (G, p) satisfies η * -detectability with respect to i at a i , then k η (−e i ) = − min a −i max a i g i (a) = 0.
Proof.For λ = e i , we can find a i ∈ A such that g i (a i ) = max a g i (a) and g i (a i ) − g i (a ′ i , a i −i ) ≥ η for any a ′ i = a i i by assumption.Let ρ i ∈ R be the profile of message strategies for which the conditions for η * -detectability with respect to i are satisfied at a i for any j = i.By Lemma 21, for each j = i, there exists x j : M −i → R such that all the η-strict incentive compatibility conditions are satisfied for any (a ′ j , ρ ′ j ) = (a i j , ρ i j ).For player i, set x i (m) = 0 for all m ∈ M. Then the η-strict incentive compatibility conditions for player i are satisfied for every nontrivial deviation (a ′ i , ρ ′ i ) ∈ Σ(a i ,ρ i ),x i , since i's message does not affect the transfer x for any player (so deviating in message after the equilibrium action is a trivial deviation).Since i λ i x i (m) = 0 for each m by construction, (a i , ρ i , x) generates an objective function value of max a g i (a) for the problem (P e i ,η ).Clearly this is the largest possible value for (P e i ,η ), hence k η (e i ) = max a g i (a).
For λ = −e i , we can find a i ∈ A such that g i (a i ) = min a −i max a i g i (a) = 0 and g i (a i ) − g i (a ′ i , a i −i ) ≥ η for any a ′ i = a i i .Let ρ i ∈ R be any profile of message strategies for which the conditions for η * -detectability with respect to i are satisfied at a i for any j = i.As in the previous case, we can find x j : M −i → R for each j = i such that all the η-strict incentive compatibility conditions are satisfied for j.Set x i (m) = 0 for all m for player i.Since i λ i x i (m) = 0, (a i , ρ i , x) generates an objective function value of −g i (a i ) = − min a −i max a i g i (a) = 0 for (P −e i ,η ).
Since k(−e i ) is bounded from above by 0 by Lemma 4 and Lemma 5, a i solves (P −e i ,η ).Hence k η (−e i ) = −g(a i ) = − min a −i max a i g i (a) = 0. Now we complete the proof of Theorem 19.The last two lemmas prove k η (λ) = max a i λ i g i (a) for any λ / ∈ {−e i , i ∈ N} and k η (−e i ) = 0 for every Then the theorem follows from Theorem 14 when intV * (G) = ∅.

More Strict Incentive Constraints
For our uniformly strict folk theorem, we require a fixed level of strict incentive compatibility at every public history.In terms of average payoff, the strict incentive (1−δ)η converges to 0 as δ → 1.This means that the loss from a single deviation becomes negligible relative to the size of the total payoff in the limit.We could instead require η-strict incentive compatibility in terms of average payoff.This means that the loss from a deviation is comparable to a permanent payoff shock, say, losing $1 in all the future periods.To do this, we would replace η in the definition of η-USPPE (Definition 1) with η 1−δ .However, it turns out that the set of η-USPPE in this sense becomes empty for any η > 0 as δ → 1.
More generally, we can impose f (δ)-strict incentive constraint in terms of average payoff, where f (δ) may not converge to 0 or converge to 0 more slowly than (1 − δ) as δ → 1.We can show that, for any such f , the set of "funiformly strict" PPE becomes empty for large enough δ.In this sense, our folk theorem cannot be improved in terms of the order of the strict incentive in the limit.
The reason for this is as follows.The effect of the current stage game payoff vanishes at the rate of (1 − δ) as δ → 1 in terms of average payoff.So, if we like to provide f (δ)-strict incentive with f (δ) such that lim δ→1 f (δ) 1−δ → ∞, it must come from the variation in continuation payoffs. 10However, the maximum variation of continuation payoffs for player i must vanish at the same rate of (1 − δ) if her continuation payoff w i (m) is always at least as large as the equilibrium payoff v from the present period.This is because the distance between the expected continuation payoff and the equilibrium payoff is E[w(•)|(a, ρ)] − v = 1−δ δ (v − g(a)), which shrinks to 0 at the rate of 1 − δ.Hence, to provide f (δ)-strict incentive, continuation payoff must be strictly less than the equilibrium payoff after some message profile, i.e., there exists ǫ > 0 and m ∈ M such that w(m) < v − ǫ for any large δ.However this cannot happen at every public history, hence there is no f (δ)-USPPE with such f (δ) for any large enough discount factor.

Folk Theorem with Double Limits
We prove our folk theorem by fixing a level of strict incentive η > 0 and letting δ → 1.If we instead allow η go to 0 and δ go to 1, then we can prove a folk theorem with weaker conditions.When η is small, we can construct Q η using the minmax action profiles if (G, p) just satisfies η-detectability instead of η * -detectability at the minmax action profiles.Since Q η converges to V * (G) as η → 0 and is the limit η-USPPE payoff set (with full dimensionality), we can prove a version of folk theorem only with η-detectability (for minmax action profiles in addition to A(G)) and η-identifiability, where η goes to 0 and δ goes to 1 at the same time.

Appendix Proof of Lemma 2
Proof.This is trivial if E η (δ) is an empty set, so suppose that it is not.First, note that E η (δ) is bounded, so we must show that E η (δ) is closed.Take any v * ∈ cl(E η (δ)).Choose a sequence v k ∈ E η (δ) in R n that converges to v * .For each k, let (a k , ρ k ) ∈ A × R be the strategy profile in the first period and w k : M → R n be the continuation payoff profile from the second period of the equilibrium strategy that supports v k .Note that w k (m) ∈ E η (δ) for all m ∈ M. Then for each i ∈ N, Since A × R is compact and E η (δ) is bounded, we may, extracting a subsequence if necessary, assume that (a k , ρ k ) and w k are convergent with respective limits (a * , ρ * ) and w * .Furthermore, we may assume that (a k , ρ k ) = (a * , ρ * ) for all sufficiently large k. .Since w * (m) ∈ cl(E η (δ)) for all m ∈ M, it follows that v * ∈ B(δ, cl(E η (δ)), η), therefore cl(E η (δ)) ⊆ B(δ, cl(E η (δ)), η).Since cl(E η (δ)) is bounded (in fact compact), by η-self decomposability (Lemma7), we can conclude that cl(E η (δ)) ⊆ E η (δ), i.e., E η (δ) is closed.