Learning to cooperate in the shadow of the law

Abstract

Formal enforcement punishing defectors can sustain cooperation by changing incentives. In this paper, we introduce a second effect of enforcement: it can also affect the capacity to learn about the group’s cooperativeness. Indeed, in contexts with strong enforcement, it is difficult to tell apart those who cooperate because of the threat of fines from those who are intrinsically cooperative types. Whenever a group is intrinsically cooperative, enforcement will thus have a negative dynamic effect on cooperation because it slows down learning about prevalent values in the group that would occur under a weaker enforcement. We provide theoretical and experimental evidence in support of this mechanism. Using a lab experiment with independent interactions and random rematching, we observe that, in early interactions, having faced an environment with fines in the past decreases current cooperation. We further show that this results from the interaction between enforcement and learning: the effect of having met cooperative partners has a stronger effect on current cooperation when this happened in an environment with no enforcement. Replacing one signal of deviation without fine by a signal of cooperation without fine in a player’s history increases current cooperation by 10%; while replacing it by a signal of cooperation with fine increases current cooperation by only 5%.

Appendices

Appendix 1: Data description

Our data delivers 934 game observations, 48% of which are played with no fine. Figure 4a displays the empirical distribution of game lengths in the sample split according to the draw of a fine. With the exception of two-rounds matches, the distributions are very similar between the two environments. This difference in the share of two-rounds matches mainly induces a slightly higher share of matches longer than 10 rounds played with a fine. In both environments, one third of the matches we observe lasts one round, and one half of the repeated matches last between 2 and 5 rounds. A very small fraction of matches (less than 5% with a fine, less than 2% with no fine) feature lengths of 10 rounds or more.

Fig. 4

Sample characteristics: distribution of game lengths and repeated-game strategies. Note: Left-hand side: empirical distribution of game lengths in the experiment, split according to the draw of the fine. Right-hand side: distribution of repeated-games strategies observed in the experiment. One-round matches are excluded. AD: Always Defect; AC: Always Cooperate; TFT: Tit-For-Tat; GT: Grimm Trigger

As explained in the text, Sect. 2.2, for matches that last more than one round (2/3 of the sample), we thus reduce the observed outcomes to the first round decision in each match, consistently with the theory. The first round decision is a sufficient statistic for the future sequence of play if subjects choose among the following repeated-game strategies: Always Defect (AD), Tit-For-tat (TFT) or Grim Trigger (GT). While AD dictates defection at the first round, both TFT and GT induce cooperation and are observationally equivalent if the partner chooses within the set restricted to these three strategies and give rise to the same expected payoff. Figure 4b displays the distribution of strategies we observe in the experiment (excluding games that last one round only). Decisions are classified in each repeated game and for each player based on the observed sequence of play. For instance, a player who starts with C and switches forever to D when the partner starts playing D will be classified as playing GT. In many instances, TFT and GT cannot be distinguished (so that the classes displayed in Fig. 4b overlap): it happens for instance for subjects who always cooperate against a partner who does the same (in which case, TFT and GT also include Always Cooperate, AC), or if defection is played forever by both players once it occurs. Last, the Figure also reports the share of Always Cooperate that can be distinguished from other match strategies—when AC is played against partners who do defect at least once.

All sequences of decisions that do not fall in any of these strategies cannot be classified—this accounts for 14% of the games played without a fine, and 24% of those played with fine. The three strategies on which we focus are thus enough to summarize a vast majority of match decisions: AD accounts for 70% of the repeated-game observations with no fine, and 41% with a fine, while TFT and GT account for 14% and 34% of them.

Appendix 2: Alternative identification strategy

The empirical evidence presented in the paper relies on the insights from the model to provide a reduced-form statistical analysis of the interaction between learning and enforcement institutions. As a complement to this evidence, this section provides a structural analysis which takes into account both learning and behavioral spillovers. To that end, we first generalize the model presented in Sect. 3.2 to the case in which individual values evolve over time as a result of past experience. We then estimate the parameters of the model. This provides an alternative strategy to separately estimate learning and spillovers.

1.1 The dynamics of learning with behavioral spillovers

Consistent with Galbiati et al. (2018), we allow both for past fines and past behaviors of the partners to affect values:

$$\begin{aligned} \beta _{it} { } = { } \beta _{i} { } + { } \phi _{F} { } \mathbbm {1}_{\{F_{it-1}=1\}} { } + { } \phi _{C} { } \mathbbm {1}_{\{a_{j_{t-1},t-1}=C\}} { }. \end{aligned}$$

(3)

According to this simple specification, personal values evolve through two channels. First, direct spillovers increase the value attached to cooperation in the current match if the previous one was played with a fine, as measured by parameter $\phi _{F}$. Second, indirect spillovers, measured by $\phi _{C}$, increase the value attached to cooperation if in the previous match the partner cooperated.¹⁶

1.1.1 Introducing behavioral spillovers in the benchmark

We start by introducing spillovers in the benchmark model. Under the assumption that values follow the process in (3) and $\phi _{F}>0, \phi _{C}>0$, the indifference condition (1) remains unchanged,¹⁷ but now $\beta _{it}$ is no longer constant and equal to $\beta _{i}$ since past shocks affect values. In this context, individual i cooperates at t if and only if:

$$\begin{aligned} \beta _{it} { }\ge & {} { } \varPi _{1} - F \times \mathbbm {1}_{\{F_{it}=1\}} { } + { } p_{it} \varPi _{2} { } - { } \frac{\delta }{1-\delta } p_{it} \left( F \times \mathbbm {1}_{\{F_{it}=1\}} + \varPi _{3}\right) { }. \end{aligned}$$

The cutoff value is defined in the same way as before:

$$\begin{aligned} \beta _{t}^{*}(F_{it})= & {} { } \varPi _{1} - F \times \mathbbm {1}_{\{F_{it}=1\}} { } + { } p^{*}_{t}(F_{it}) \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \left( F \times \mathbbm {1}_{\{F_{it}=1\}} + \varPi _{3}\right) \right] { }. \end{aligned}$$

(4)

The main difference with the benchmark model is in the value of $p^{*}_{t}(F_{it})$. There is now a linkage between the values of the cutoffs at match t, $\beta _{t}^{*}$, and the values of the cutoffs $\beta _{t'}^{*}$ in all the preceding matches $t'<t$ through $p^{*}_{t}(F_{it})$. Indeed, when an individual evaluates the probability that her current partner in t, player $j_{t}$, will cooperate, she needs to determine how likely it is that she received a direct and/or an indirect spillover from the previous period. The probability of having a direct spillover is given by $P[F_{jt-1}=1]=1/2$ and is independent of any equilibrium decision. By contrast, the probability of having an indirect spillover is linked to whether the partner of $j_{t}$ in her previous match cooperated or not. This probability in turn depends on the cutoffs in $t-1$, $\beta _{t-1}^{*}$, which also depends on whether that individual himself received indirect spillovers, i.e on the cutoff in $t-2$. Overall, these cutoffs in t depend on the entire sequence of cutoffs.

In the remaining, we focus on stationary equilibria, such that $\beta ^{*}$ is independent of t. We show in Proposition C that such equilibria do exist.

Proposition C

(Spillovers) In an environment with spillovers ($\phi _{F}>0$ and $\phi _{C}>0$) and no uncertainty on values, there exists a stationary equilibrium. Furthermore all equilibria are of the cutoff form, i.e individuals cooperate if and only if $\beta _{it} \ge \beta ^{*} (F_{it})$.

Proof

See “Appendix 5: Proofs”. $\square$

Proposition C proves the existence of an equilibrium and presents the shape of the cutoffs. The Proposition also allows to express the probability that a random individual cooperates as:

$$\begin{aligned} 1-\varPhi _{H}\left[ \varLambda _{1} - \phi _{F} { } \mathbbm {1}_{\{F_{it-1}=1\}} { } - { } \phi _{C} { } \mathbbm {1}_{\{a_{j_{t-1},t-1}=C\}} { } - \varLambda _{2} \mathbbm {1}_{\{F_{it}=1\}} \right] { }, \end{aligned}$$

(5)

where

$$\begin{aligned} \varLambda _{1} { }\equiv & {} { } \beta ^{*}(0) { } = { } \varPi _{1} { } + { } p^{*}(0) \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \varPi _{3} \right] { },\\ \varLambda _{2} { }\equiv & {} { } \beta ^{*}(1) - \beta ^{*}(0) { } = { } F { } + { } [ p^{*}(0) - p^{*}(1)]\left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \varPi _{3} \right] + \frac{\delta }{1-\delta } p^{*}(1) F { }. \end{aligned}$$

1.1.2 The dynamics of cooperation with learning and spillovers

We now solve the full model with uncertainty about the group’s values and with spillovers. As in the main text, we denote $q_{it}$ the belief held by player i at match t that the state is H.

In this expanded model, the beliefs on how likely it is that the partner cooperates in match t, $p^{*}_{t}(F_{it},q_{it})$, depends on the probability that the partner experienced spillovers. In addition, the probability that the partner j had an indirect spillover itself depends on whether his own partner k in the previous match did cooperate, and thus depends on the beliefs $q_{kt-1}$ of that partner in the previous match. The general problem requires to keep track of the higher order beliefs. The proof of the following Proposition shows the existence of such a stationary equilibrium, under a natural restriction on higher order beliefs, i.e if we assume that a player who had belief $q_{it}$ in match t believes that players in the preceding match had the same beliefs $q_{j',t-1} = q_{it}$.

Proposition D

In an environment with spillovers and learning, if an equilibrium exists, all equilibria are of the cutoff form, i.e individuals cooperate if and only if $\beta _{i} \ge \beta ^{*} (F_{it},q_{it})$. Furthermore, if in equilibrium $\beta ^{*} (0,q) > \beta ^{*} (1,q)$ for all beliefs q, then the beliefs are updated in the following way given the history in the previous interaction:

$$\begin{aligned} q_{it} (0,C,q_{it-1})> q_{it} (1,C,q_{it-1}) > q_{it-1} { }, \\ q_{it} (1,D,q_{it-1})< q_{it} (0,D,q_{it-1}) < q_{it-1} { }. \end{aligned}$$

Proof

See “Appendix 5: Proofs”. $\square$

Proposition D derives a general property of equilibria. The Proposition also allows to express the probability of cooperation for given belief $q_{it-1}$ as:

$$\begin{aligned}{} & {} 1-\varPhi _{H}\left[ \varLambda _{3} - \phi _{F} { } \mathbbm {1}_{\{F_{it-1}=1\}} { } - { } \phi _{C} { } \mathbbm {1}_{\{a_{j_{t-1},t-1}=C\}} { } - \varLambda _{4} \mathbbm {1}_{\{F_{it}=1\}} \right. \nonumber \\{} & {} \quad \left. - \sum _{j,k \in \{0,1 \}, l \in \{C,D \} } \varLambda ^{j}_{k,l} { } \mathbbm {1}_{ \{ F_{it}=j, F_{it-1}=k, a_{j_{t-1},t-1}=l \}} \right] { }. \end{aligned}$$

(6)

where

$$\begin{aligned} \varLambda _{3} { } &\equiv \varPi _{1} { } + { } p^{*}(0,0,D,q) \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \varPi _{3} \right] { }, \\ \varLambda _{4} { } &\equiv F { } - { } \left[ p^{*}(1,0,D,q) - p^{*}(0,0,D,q)\right] \\& \quad \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \varPi _{3} \right] + \frac{\delta }{1-\delta } p^{*}(1,0,D,q) F >0 { }, \\ \varLambda ^{0}_{k,l}{ } &\equiv [p^{*}(0,k,l,q) - p^{*}(0,0,D,q)] \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \varPi _{3} \right] { }, \\ \varLambda ^{1}_{k,l} &\equiv - [p^{*}(0,k,l,q) - p^{*}(0,0,D,q)] \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } (F +\varPi _{3}) \right] { }. \end{aligned}$$

Note that the parameters $\varLambda _{3}$, $\varLambda _{4}$ and $\varLambda ^{j}_{k,l}$ in Eq. (6) depend on $q_{it-1}$. Compared to the case without learning, there are 6 additional parameters, reflecting the updating of beliefs. According to the result in Proposition D, these parameters, both in the case where the current match is played with a fine and when it is not, are such that:

$$\begin{aligned}{} & {} \varLambda ^{1}_{0,C}> \varLambda ^{1}_{1,C}> 0> \varLambda ^{1}_{1,D} { }, \nonumber \\{} & {} \varLambda ^{0}_{0,C}> \varLambda ^{0}_{1,C}> 0 > \varLambda ^{0}_{1,D} { }. \end{aligned}$$

(7)

Overall, having fines in the previous match can potentially decrease average cooperation in the current match. There are two countervailing effects. On the one hand, a fine in the previous match increases the direct and indirect spillovers and thus increases cooperation. On the other hand, if the state is low, a fine can accelerate learning if, on average, sufficiently many people deviate in the presence of a fine. This then decreases cooperation in the current match.

1.1.3 Statistical implementation of the model

The main behavioral insights from the model are summarized by Eq. (6), which involves both learning and spillover parameters. As the equation clearly shows, exogenous variations in legal enforcement are not enough to achieve separate identification of learning and spillover parameters—an exogenous change in any of the enforcement variables, or past behavior of the partner, involves both learning and a change in the values $\beta _{it}$. In the main text, identification relies on the assumption that spillovers are short-living in the sense that their effect on behavior is smaller the earlier they happen in one’s own history—while learning should not depend on the order in which a given sequence of actions happens. In this section, we report the results from an alternative identification strategy that relies on the assumption that learning has converged once a large enough number of matches has been played. Under this assumption, in late games, behavior is described by Eq. (5), which involves only spillover parameters. Exogenous variations in enforcement thus provide identification of spillover parameters in late games, which in turn allows to identify learning parameters in early ones.

To that end, as explained in the text, we split the matches in three groups, in such a way that one third of the observed decisions are classified as “early”, and one third as “late”. We use matches, rather than periods, as a measure of time since we focus on games for which the first stage decision summarizes all future actions within the current repeated game—hence ruling out learning within a match. Observed matches are accordingly defined as “early” up to the 7th, as late after the 13th—we disregard data coming from intermediary stages.¹⁸ Denote $\mathbbm {1}_{\{\text {Early}\}}$ the dummy variable equal to 1 in early games and to 0 in late games. Under the identifying assumption that learning has converged in late games, the model predicts that behavior in the experiment is described by:

$$\begin{aligned} P(C_{it} =1)= & {} 1 - \varPhi _{H}\left[ \varLambda _{1} + ( \varLambda _{3} -\varLambda _{1} ) { } \mathbbm {1}_{\{\text {Early}\}} - \phi _{F} { } \mathbbm {1}_{\{F_{it-1}=1\}} { } \right. \\{} & {} \left. - { } \phi _{C} { } \mathbbm {1}_{\{a_{j_{t-1},t-1}=C\}} { } - \varLambda _{2} \mathbbm {1}_{\{F_{it}=1\}} \right. \\{} & {} - \left. (\varLambda _{4} - \varLambda _{2}) { } \mathbbm {1}_{\{F_{it}=1\}} \times \mathbbm {1}_{\{\text {Early}\}} \right. \\{} & {} \left. - \sum _{j,k \in \{0,1 \}, l \in \{C,D \} } \varLambda ^{j}_{k,l} { } \mathbbm {1}_{ \{F_{it}=j, F_{it-1}=k, a_{j_{t-1},t-1}=l\}}\times \mathbbm {1}_{\{\text {Early}\}}\right] \end{aligned}$$

which is the structural form of a Probit model on the individual decision to cooperate. This probability results from equilibria of the cutoff form involving the primitives of the model. Denoting $\varepsilon _{it}$ observation specific unobserved heterogeneity, ${\varvec{\theta }}$ the vector of unknown parameters embedded in the above equation, ${\varvec{x}}_{it}$ the associated set of observables describing participant i experience up to t and $C^*_{it}$ the latent function generating player i willingness to cooperate at match t, observed decisions inform about the model parameters according to:

$$C_{it} = \mathbbm {1}[C^*_{it} = {\varvec{x}}_{it} {\varvec{\theta }} + \varepsilon _{it} > 0]$$

The structural parameters govern the latent equation of the model. Our empirical test of the model is thus based on estimated coefficients, ${\varvec{\theta }}=\frac{\partial C^*}{\partial {\varvec{x}}_{it}}$, rather than marginal effects, $\frac{\partial C}{\partial {\varvec{x}}_{it}}={\varvec{\theta }} \frac{\partial \varPhi ({\varvec{x}}_{it}{\varvec{\theta }})}{\partial {\varvec{x}}_{it}}$.

In the set of covariates, both current ($\mathbbm {1}_{\{F_{it}=1\}}$) and past enforcement ($\mathbbm {1}_{\{F_{it-1}=1\}}$) are exogenous by design. The partner’s past decision to cooperate, $C_{jt-1}$, is exogenous to $C_{it}$ as long as player i and j have no other player in common in their history. Moreover, due to the rematching of players from one match to the other, between subjects correlation arises over time within an experimental session. We address these concerns in three ways. We include the decision to cooperate at the first stage of the first match in the set of control variables, as a measure of individual unobserved ex ante willingness to cooperate. To further account for the correlation structure in the error of the model, we specify a panel data model with random effects at the individual level, control for the effect of time thanks to the inclusion of the match number, and cluster the errors at the session level to account in a flexible way for within sessions correlation.

Table 3 reports the estimation results from several specifications, in which each piece of the model is introduced sequentially. The parameters of interest are the learning parameters $\varLambda _{k,l}, k \in \{0,1 \}, l \in \{C,D \}$.¹⁹ Columns (1) and (2) focus on the effect of past and current enforcement. While we do not find any significant change due to moving from early to late games per se (the Early variable is not significant), the effect of current enforcement on the current willingness to cooperate is much weaker in early games. This is consistent with participants becoming less confident that the group is cooperative, thus less likely to cooperate, as time passes—i.e., prior belief over-estimate the average cooperativeness of the group. The disciplining effect of current fines is thus stronger in late games.

Table 3 Learning and spillovers arising from past enforcement

Column (3) introduces learning parameters. As stressed above, the learning parameters play a role before beliefs have converged. They are thus estimated in interaction with the Early dummy variable. Once learning is taken into account, enforcement spillovers turn-out significant. More importantly, the model predicts that learning is stronger when observed decisions are more informative about societal values, which in turn depends on the enforcement regime under which behavior has been observe: cooperation is more informative about cooperativeness under weak enforcement, while defection is stronger signal of non-cooperative values under strong enforcement. This results in a clear ranking between learning parameters—see Eq. (7). We use defection under weak enforcement as a reference for the estimated learning parameters. The results show that cooperation under weak enforcement ($\text {Early}\times {C_0}$) leads to the strongest increase in the current willingness to cooperate. Observing this same decision but under strong enforcement institutions rather than weak ones ($\text {Early}\times {C_1}$) has almost the same impact as observing defection under strong institutions (the reference): in both cases, behavior is aligned with the incentives implemented by the rules and barely provides any additional insights about the distribution of values in the group. Last, defection under strong institutions ($\text {Early}\times D_0$) is informative about a low willingness to cooperate in the group, and results in a strongly significant drop in current cooperation.

Column (4) adds indirect spillovers, induced by the cooperation of the partner in the previous game. The identification of learning parameters in this specification is quite demanding since both past enforcement and past cooperation are included as dummy variables. We nevertheless observe a statistically significant effect of learning in early games, with the expected ordering according to how informative the signal delivered by a cooperative decision is, with the exception of $C_1$—i.e., when cooperation has been observed under fines. Finally, column (5) provides a robustness check for the reliability of the assumption that learning has converged in late games. To that end, we further add the interaction between observed behavior from partner in the previous game and the enforcement regime. Once learning has converged, past behavior is assumed to affect the current willingness to cooperate through indirect spillovers only. Absent learning, this effect should not interact with the enforcement rule that elicited this behavior. As expected, this interaction term is not significant: in late games, it is cooperation per se, rather than the enforcement regime giving rise to this decision, that matters for current cooperation.

Appendix 3: Replication of the statistical analysis on the full sample

In this section, we replicate the results on the full sample of observations: instead of restricting the analysis to the working sample made of decisions consistent with the subset of repeated-game strategies described in Sect. 2.2, we include all available observations. As already mentioned in the text, the matches we exclude from the working sample all appear in late games so that Fig. 2a is not affected by the choice of the working sample. Figure 5 below replicates Fig. 3 in the paper; and Table 4 replicates Table 2. In both instances, the data is more noisy but the qualitative conclusions all remain the same.

Fig. 5

Current cooperation as a function of history in the previous 5 games, computed on the full sample

Table 4 Replication of Table 2 on the full sample

Appendix 4: Replication of the main results with bootstrapped standard errors

As explained in the main text, the statistical analysis presented in Table 2 clusters the standard errors at the session level, so as to take into account in a flexible way the possible correlations between subjects due to random rematching of subjects within pairs. While this approach is conservative (since it does not impose any structure on the correlation between subjects over time), the number of clusters is small and there is a risk of small sample downward bias in the estimated standard errors. Table 5 provides the results from a robustness exercise replicating Table 2 in the text with bootstrapped standard error based a delete-one jackknife procedure (see Bell & McCaffrey, 2002; Cameron et al. 2008).

Table 5 Replication of Table 2 using bootstrapped errors

Appendix 5: Proofs

1.1 Proof of Proposition 1

As derived in the main text, if an equilibrium exists, it is necessarily such that players use cutoff strategies. Reexpressing characteristic Eq. (2), we can show that the cutoffs are determined by the equation $g(\beta ^{*} [F_{it}])=0$, where g is given by

$$\begin{aligned} g(x) = - x { } + { } \varPi _{1} - F { } \mathbbm {1}_{\{F_{it}=1\}} { } + { } \left( 1 - \varPhi _{H} \left( x \right) \right) \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \left( F { } \mathbbm {1}_{\{F_{it}=1\}} + \varPi _{3}\right) \right] . \end{aligned}$$

The function g has the following properties: $g(x)>0$ when x converges to $-\infty$ and $g(x)<0$ when x converges to $+\infty$. Thus, since g is continuous, there is at least one solution to the equation $g(\beta ^{*} [F_{it})]=0$. At least one equilibrium exists.

If g is non monotonic, there could exist multiple equilibria. However, in all stable equilibria, $\beta ^{*}$ is such that g is decreasing at $\beta ^{*}$, i.e.,

$$\begin{aligned} -1 - \phi _{H} \left[ \beta ^{*} \right] \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \left( F { } \mathbbm {1}_{\{F_{it}=1\}} + \varPi _{3}\right) \right] <0 { } . \end{aligned}$$

(8)

Using the implicit theorem we have:

$$\begin{aligned} \frac{\partial \beta ^{*}}{\partial F} = - \frac{\partial g}{\partial F} / \frac{\partial g}{\partial \beta ^{*}} = - \frac{-1 - \left( 1 - \varPhi _{H} \left[ \beta ^{*} \right] \right) \frac{\delta }{1-\delta } }{ -1 - \phi _{H} \left[ \beta ^{*} \right] \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \left( F { } \mathbbm {1}_{\{F_{it}=1\}} + \varPi _{3}\right) \right] } { }, \end{aligned}$$

where $\phi _{H}$ is the density corresponding to distribution $\varPhi _{H}$. For stable equilibria, the denominator is negative as shown in (8), so that overall

$$\begin{aligned} \frac{\partial \beta ^{*}}{\partial F} < 0 { }. \end{aligned}$$

Similarly,

$$\begin{aligned} \frac{\partial \beta ^{*}}{\partial \mu _{H}}= & {} - \frac{ - \frac{\partial \varPhi _{H} \left[ \beta ^{*} \right] }{\partial \mu _{H}} \left[ \varGamma _{2} { } - { } \frac{\delta }{1-\delta } \left( F { } \mathbbm {1}_{\{F_{it}=1\}} + \varPi _{3}\right) \right] }{ -1 - \phi _{H} \left[ \beta ^{*} \right] \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \left( F { } \mathbbm {1}_{\{F_{it}=1\}} + \varPi _{3}\right) \right] } { }. \end{aligned}$$

Again, in stable equilibria the denominator is negative by (8). Furthermore we have $\frac{\partial \varPhi _{H} \left[ \beta ^{*} \right] }{\partial \mu _{H}}<0$ since an increase in the mean of the normal distribution decreases $\varPhi _{H} \left[ x \right]$ for any x. Overall we get

$$\begin{aligned} \frac{\partial \beta ^{*}}{\partial \mu _{H}} <0 { }. \end{aligned}$$

1.2 Proof of Lemma 1

We first show the result: $q_{i2} (1,D)< q_{i2} (0,D) < q_{0}$. According to Baye’s rule, the belief that the state is H following a deviation by the partner in the first match (which has been played with a fine) is:

$$\begin{aligned} q_{i2} (1,D)= & {} \frac{ q_{0} { } P [ D| F_{i1}=1, s=H] }{ q_{0} { } P [ D| F_{i1}=1, s=H]+ (1-q_{0}) { } P [ D| F_{i1}=1, s=L]} \nonumber \\= & {} \frac{ q_{0} { } \varPhi _{H} \left[ \beta ^{*}(1) \right] }{ q_{0} { } \varPhi _{H} \left[ \beta ^{*}(1) \right] + (1-q_{0}) { } \varPhi _{L} \left[ \beta ^{*}(1) \right] } \nonumber \\= & {} \frac{ 1 }{1 + \frac{1-q_{0}}{q_{0}} { } \frac{\varPhi _{L} \left[ \beta ^{*}(1) \right] }{\varPhi _{H} \left[ \beta ^{*}(1) \right] }} { }. \end{aligned}$$

(9)

Furthermore, since $\varPhi _{L} \left[ \beta ^{*}(1) \right] > \varPhi _{{H}} \left[ \beta ^{*}(1) \right]$, we have $q_{i2} (1,D) {<} q_{0}$. Similarly we have:

$$\begin{aligned} q_{i2} (0,D)= & {} \frac{ 1 }{1 + \frac{1-q_{0}}{q_{0}} { } \frac{\varPhi _{L} \left[ \beta ^{*}(0) \right] }{\varPhi _{H} \left[ \beta ^{*}(0) \right] }} { } >q_{0} { }. \end{aligned}$$

(10)

Thus,

$$\begin{aligned} q_{i2} (1,D)< q_{i2} (0,D) \Leftrightarrow \frac{\varPhi _{L} \left[ \beta ^{*}(0) \right] }{\varPhi _{H} \left[ \beta ^{*}(0) \right] } < \frac{\varPhi _{L} \left[ \beta ^{*}(1) \right] }{\varPhi _{H} \left[ \beta ^{*}(1) \right] } { }. \end{aligned}$$

Using the fact that $\frac{\varPhi _{L} \left[ x \right] }{\varPhi _{H} \left[ x \right] }$ is decreasing in x as shown in Property 1 below, and the fact that in stable equilibria, we have $\beta ^{*}(1) \le \beta ^{*}(0)$, as shown in Proposition 1, implies directly that $q_{i2} (1,D) < q_{i2} (0,D)$. The proof that $q_{i2} (0,C)> q_{i2} (1,C) > q_{0}$ follows similar lines.

Property 1

$\frac{\varPhi _{H} \left[ x \right] }{\varPhi _{L} \left[ x \right] }$ is increasing in x.

Proof

Denote $\phi _{H}$ (resp. $\phi _{L}$) the density of $\varPhi _{H}$ (resp. $\varPhi _{L}$). Given that $\phi _{H}$ (resp. $\phi _{L}$) is the density of a normal distribution with standard deviation $\sigma$ and mean $\mu _{H}$ (resp. $\mu _{L}$), it is the case that:

$$\begin{aligned} \frac{\phi _{H} \left[ x \right] }{\phi _{L} \left[ x \right] }&= \frac{\frac{1}{\sigma \sqrt{2\pi }} e^{-(x-\mu _{H})^{2} / \sigma ^{2} } }{ \frac{1}{\sigma \sqrt{2\pi }} e^{-(x-\mu _{L})^{2} / \sigma ^{2} }} \\ &= e^{-(x-\mu _{H})^{2} / \sigma ^{2} + (x-\mu _{L})^{2} / \sigma ^{2} } \\&= e^{ \frac{1}{\sigma ^{2}} (\mu _{H} - \mu _{L}) (2x - \mu _{L}- \mu _{H})} { }. \end{aligned}$$

Thus $\frac{\phi _{H} }{\phi _{L}}$ is increasing in x. In particular for $y<x$, we have: $\phi _{H} \left[ y \right] \phi _{L} \left[ x \right] < \phi _{L} \left[ y \right] \phi _{H} \left[ x \right]$. By definition, $\varPhi _{s} (x) = \int _{-\infty }^{x} \phi _{s} (y) dy$. Integrating with respect to y between $-\infty$ and x thus yields:

$$\begin{aligned} \varPhi _{H} \left[ x \right] \phi _{L} \left[ x \right] < \varPhi _{L} \left[ x \right] \phi _{H} \left[ x \right] { }. \end{aligned}$$

(11)

Consider now the function $\frac{\varPhi _{H} }{\varPhi _{L}}$. The derivative of this function is given by $\frac{\phi _{H}\varPhi _{L} - \phi _{L}\varPhi _{H} }{\varPhi _{L}^{2}}$, which is positive by Eq. (11). This establishes Property 1 that $\frac{\varPhi _{H} \left[ x \right] }{\varPhi _{L} \left[ x \right] }$ is increasing in x. $\square$

1.3 Proof of Proposition D (that generalizes Proposition 2)

In the first part of the proof we assume a stationary equilibrium exists and is such that the equilibrium cutoffs are always higher without a fine in the current match $\beta ^{*} (0,q) > \beta ^{*} (1,q)$ for any given belief q. We then derive the property on updating of beliefs. In the second part of the proof we show existence under a natural restriction on beliefs.

Part 1: We first derive the properties on updating. We have

$$\begin{aligned}{} & {} q_{it} (F_{it-1}, a_{j_{t-1},t-1},q_{it-1}) \nonumber \\{} & {} \quad = \frac{q_{it-1} P [a_{j_{t-1},t-1} | F_{it-1}, s=H] }{ q_{it-1} P [a_{j_{t-1},t-1} | F_{it-1}, s=H] + (1-q_{it-1}) P [a_{j_{t-1},t-1} | F_{it-1}, s=L]} { }. \end{aligned}$$

(12)

We can express the probability that the partner $j_{t-1}$ in match $t-1$ cooperated, by considering all the possible environments this individual might have faced in the past, in particular what his partner in match $t-2$, individual $k_{t-1}$ chose:

$$\begin{aligned}{} & {} P [a_{j_{t-1},t-1} = D | F_{it-1}, s = H] = \sum _{F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1}} \varPhi _{H} \\{} & {} \quad \left[ \beta ^{*}(1,q_{j_{t-1},t-1}) - \phi _{F} { } \mathbbm {1}_{\{F_{j_{t-1},t-2}=1\}} { } - { } \phi _{C} { } \mathbbm {1}_{\{a_{k_{t-1},t-2}=C\}} \right] \\{} & {} \quad \times { } P[F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1}]{}. \end{aligned}$$

Denote

$$\begin{aligned}{} & {} \gamma ^{*} \left( F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1} \right) \\{} & {} \quad = \beta ^{*}(1,q_{j_{t-1},t-1}) - \phi _{F} { } \mathbbm {1}_{\{F_{j_{t-1},t-2}=1\}} { } - { } \phi _{C} { } \mathbbm {1}_{\{a_{k_{t-1},t-2}=C\}} {}. \end{aligned}$$

and

$$\begin{aligned} R (x) \equiv \frac{ \sum _{F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1} } \varPhi _{H} \left[ \gamma ^{*} \left( x, a_{k_{t-1},t-2}, q_{j_{t-1},t-1} \right) \right] P[F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1}] }{ \sum _{F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1}} \varPhi _{L} \left[ \gamma ^{*} \left( x, a_{k_{t-1},t-2}, q_{j_{t-1},t-1} \right) \right] P[F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1}] } { }. \end{aligned}$$

Using expression (12), we have:

$$\begin{aligned} q_{it} (1, D,q_{it-1}) < q_{it} (0, D,q_{it-1}) \Leftrightarrow R(1) \ge R(0) { }. \end{aligned}$$

We then use all possible values of the vector $(F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1})$ in turn. Take such a value v for this vector and denote

$$\begin{aligned} a &\equiv \sum _{(F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1}) \ne v} \varPhi _{H} \left[ \gamma ^{*} \left( x, a_{k_{t-1},t-2}, q_{j_{t-1},t-1} \right) \right] \\ & \quad P[F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1}] { },\\ b &\equiv \sum _{(F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1}) \ne v} \varPhi _{L} \left[ \gamma ^{*} \left( x, a_{k_{t-1},t-2}, q_{j_{t-1},t-1} \right) \right] \\ & \quad P[F_{j_{t-1},t-2}, a_{k_{t-1},t-2}, q_{j_{t-1},t-1}] { }. \end{aligned}$$

We clearly have $a<b$. Furthermore, we can write

$$\begin{aligned} R (x) \equiv \frac{ a + \varPhi _{H} \left[ \gamma ^{*} \left( v \right) \right] P[v] }{ b + \varPhi _{L} \left[ \gamma ^{*} \left( v \right) \right] P[v] } \end{aligned}$$

We have $\beta ^{*} (0,q) > \beta ^{*} (1,q)$ imply that $\gamma ^{*} \left( 0, a_{k't-2}, q_{j't-1} \right) > \gamma ^{*} \left( 1, a_{k't-2}, q_{j't-1} \right)$. Thus, using Property 2 below, it implies that $R(1) \ge R(0)$ and thus $q_{it} (1, D,q_{it-1}) < q_{it} (0, D,q_{it-1})$.

Property 2

$\frac{a + p\varPhi _{H} \left[ x \right] }{ b + p\varPhi _{L} \left[ x \right] }$ where $b>a$ is increasing in x.

Proof

The derivative of the ratio is given by

$$\begin{aligned} \frac{\phi _{H}\left( b + p \varPhi _{L} \right) - \phi _{L} \left( a + p \varPhi _{H} \right) }{\left( b + p \varPhi _{L} \right) ^{2}} \end{aligned}$$

(13)

We showed in the proof of Property 1 that: $\phi _{H} \varPhi _{L} - \phi _{L}\varPhi _{H} >0$. Furthermore, we also showed that $\frac{\phi _{H}}{\phi _{L}}$ is increasing and since $a<b$ this implies: $\phi _{H} b - \phi _{L} a >0$. Combining these two results in condition (13) establishes Property 2.$\square$

Part 2: We show that an equilibrium exists if we assume that a player who has belief $q_{it}$ in match t believes that other players in match t and $t-1$ shared the same belief $q_{j_{t},t} = q_{k_{t},t} = q_{it}$.

If a stationary equilibrium exists, it is necessarily such that players use cutoff strategies where the cutoff is defined by:

$$\begin{aligned} \beta _{t}^{*}(F_{it},q_{it})= & {} { } \varPi _{1} - F { } \mathbbm {1}_{\{F_{it}=1\}} { } + { } p^{*}_{t}(F_{it}, q_{it}) \left[ \varPi _{2} { } - { } \frac{\delta }{1-\delta } \left( F { } \mathbbm {1}_{\{F_{it}=1\}} + \varPi _{3}\right) \right] \end{aligned}$$

We have:

$$\begin{aligned} p^{*}_{t}(F_{it}, q_{it}) &= P[a_{j_{t},t} = C | F_{it}, q_{it}] \\ &= q_{it} \sum _{(F_{j_{t},t-1}, a_{k_{t},t-1}, q_{j_{t},t-1}) } \left[ 1 - \varPhi _{H} \left[ \beta ^{*}(F_{j_{t},t},q_{j_{t},t-1}) \right. \right. \\ &\quad \left. \left.\, - \phi _{F} { } \mathbbm {1}_{\{F_{j_{t},t-1}=1\}} { } - { } \phi _{C} { } \mathbbm {1}_{\{a_{k_{t},t-1}=C\}} \right] \right] \\ &\quad \times { } P[F_{j_{t},t-1},a_{k_{t}t-1},q_{j_{t},t-1}| s=H] \\ & \quad + (1 - q_{it}) \sum _{(F_{j_{t},t-1}, a_{k_{t},t-1}, q_{j_{t},t-1}) } \left[ 1 - \varPhi _{L} \left[ \beta ^{*}(F_{j_{t},t},q_{j_{t},t-1}) \right. \right. \\ & \quad \left. \left. -\, \phi _{F} { } \mathbbm {1}_{\{F_{j_{t},t-1}=1\}} { } - { } \phi _{C} { } \mathbbm {1}_{\{a_{k_{t},t-1}=C\}} \right] \right] \\ & \quad \times { } P[F_{j_{t},t-1},a_{k_{t}t-1},q_{j_{t},t-1}| s=L] \end{aligned}$$

Furthermore, we have

$$\begin{aligned}{} & {} P[F_{j_{t},t-1},a_{k_{t},t-1},q_{j_{t},t-1} | s=H] { } = { } P[F_{j_{t},t-1}] { } \\{} & {} \quad P[a_{k_{t},t-1} |F_{j_{t},t-1}, s=H] { } f_{t}(q_{j_{t},t-1} |s=H) \\{} & {} \quad = \frac{1}{2} { } P[a_{k_{t},t-1} |F_{j_{t},t-1}, s=H] { } f_{t}(q_{j_{t},t-1} | s=H) \end{aligned}$$

we assumed that a player who had belief $q_{it-1}$ in match $t-1$ believes that all other players in that match share the same belief $q_{it-1}$. Under this restriction, we have $f_{t}(q_{j't-1} | s=.) = \mathbbm {1}_{(q_{j't-1} = q_{it-1})}$

$$\begin{aligned} P[1,D,q | s=H] { }= & {} \frac{1}{2} { } p^{*}_{t}(1, q) \end{aligned}$$

We get a similar expression as in the proof of Proposition 2:

$$\begin{aligned} p^{*}(F_{it},q_{it}) & = \left[ 1- \varPhi _{H} \left[ \beta ^{*}(F_{it},q_{it}) - \phi _{F} - \phi _{C} \right] \right] \nonumber \\ & \quad \frac{1}{2} p^{*}(1,q_{it}) { }+ { } \left[ 1- \varPhi _{H} \left[ \beta ^{*}(F_{it},q_{it}) - \phi _{F} \right] \right] \frac{1}{2} [1- p^{*}(1,q_{it})] \nonumber \\ & \quad + \left[ 1- \varPhi _{H} \left[ \beta ^{*}(F_{it},q_{it}) - \phi _{C} \right] \right] \frac{1}{2} p^{*}(0,q_{it}) { } + { } \nonumber \\ & \quad \left[ 1- \varPhi _{H} \left[ \beta ^{*}(F_{it},q_{it}) \right] \right] \frac{1}{2} [1- p^{*}(0,q_{it})] { }. \end{aligned}$$

(14)

This implies that for each belief q, there is a system of equation equivalent to system A in the proof if Proposition 2. We thus have a solution of this system for each value q.