Abstract
In a sequential-move, finitely repeated prisoners’ dilemma game (FRPD), cooperation can be sustained if the first-mover believes her opponent might be a behavioral type who plays a tit-for-tat strategy in every period. We test this theory by revealing second-mover histories from an earlier FRPD experiment to their current opponent. Despite eliminating the possibility of reputation-building, aggregate cooperation actually increases when histories are revealed. Cooperative histories lead to increased trust, but negative histories do not cause decreased trust. We develop a behavioral model to explain these findings.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
During the first block, subjects know that there will be an optional second experiment but know nothing about its nature.
In contrast to the finitely repeated case, experimental evidence has shown that cooperation in the infinitely repeated prisoners’ dilemma aligns well with theoretical predictions. For example, Roth and Murnighan (1978) and Murnighan and Roth (1983) study behavior in indefinitely repeated prisoners’ dilemma experiments and find behavioral differences predicted by standard folk theorem equilibria. More recently, Dal Bó (2005) finds experimental evidence that greater cooperation occurs in an indefinitely repeated prisoner’s dilemma with the same expected length as a finitely repeated control and Dal Bó and Frechette (2011) find evidence that subgame perfection is a necessary (but not sufficient) condition in supporting cooperation in an indefinitely repeated prisoners’ dilemma.
Jung et al. (1994) analyze the sequential equilibrium of a chain-store game that shared some features with Camerer and Weigelt’s borrower–lender game and also find discrepancies with the theory that cannot be resolved with an appeal to homemade beliefs. Similarly, both Brandts and Figueras (2003) and Tingley and Walter (2011) find higher rates of cooperation than predicted by reputation-building in shorter finitely repeated games.
Reuben and Suetens ensure that cooperation is not a rational strategy by setting the probability that the game terminates below the threshold required for cooperation to constitute a subgame perfect equilibrium.
Selten and Stoecker use parameter estimates from the first 20 supergames to predict the outcomes of the last five supergames and find strong agreement between the predictions and actual outcomes.
Subjects in Selten and Stoeker’s experiment participated in 6-person matching groups so could have learned their opponents’ types over 25 repetitions. Since subjects were told that they would play each opponent only once, however, they ruled this possibility out.
In an fMRI study of a 10-period trust game—which is similar to the FRPD game—King-Casas et al. (2005) find that second-movers’ brains eventually signal the intent to cooperate before the first-movers’ actions are revealed. They also become more accurate in predicting first-movers’ actions. This is consistent with the hypothesis that players build a model of their opponent over time, though the data are not informative about the content of that model. A related idea is explored by Kahn and Murnighan (1993), who conduct an experiment on FRPDs in which they explicitly induce uncertainty about opponents’ types by varying their pecuniary payoffs. They find that “weak” players (players for whom defection is not a dominant strategy in the stage game) are more cooperative than “strong” players (with typical prisoners’ dilemma payoffs), and that uncertainty about opponents’ payoffs increases cooperation for “weak” players.
Samuelson (1987) shows that cooperation can be sustained for at least some periods when the assumption that the number of periods is common knowledge is relaxed. Following this approach, Normann and Wallace (2012) experimentally compare repeated prisoners’ dilemma games with known, random, and ambiguous number of periods, finding no significant differences in cooperation. An experiment by Bruttel et al. (2012) studies an FRPD in which the number of periods is uncertain. They find that cooperation breaks down closer to the final round than in a baseline treatment with a commonly known finite horizon. They also find that many players cooperated after they were privately informed about the number of remaining periods. In the current study, the number of periods is publicly announced to all subjects to eliminate such uncertainty.
The repeated sequential-move game also has the advantage of tractability and yields a unique sequential equilibrium.
That is, similar to Kreps et al. (1982), the first-mover’s prior is common knowledge. This assumption could be relaxed. For example, one could model the first-mover’s prior as coming from a distribution of priors, in which case the second-mover’s optimal strategy will be a function of the first-mover’s expected prior. The distribution for this expectation may change as \(t\) increases too, but the intuition for the optimal strategies remains largely the same and revealing oneself as rational removes the uncertainty over the second-mover’s type.
Such beliefs are certainly justified, given that tit-for-tat play is often observed in experimental data; see Andreoni and Miller (1993), for example.
We will more simply refer to a player that has the objective of maximizing his own payoff as a rational player.
The second-mover cannot attempt to restore cooperation by playing C in response to D, for this would reveal his true rationality with certainty and result in defection in all subsequent periods.
An alternative model is one in which second-movers begin the game as tit-for-tat players but randomly “wake up” in some period and become rational as they realize the end of the game is approaching. This would be identical to the current model, except \(p_t\) would be decreasing rather than constant in early periods. Further details are available upon request.
Having subjects play multiple supergames in each block allows them to become familiar with the game and, more importantly, allows second-movers to reveal more information about their types in Block 1. In any single supergame, it is possible that an individual second-mover might face a very uncooperative first-mover, so that no information about the second-mover’s type would be revealed.
In a couple of sessions, less than 20 subjects participated. In these sessions, subjects played against a different subject from the other group until they had played against all of them once. In the remaining supergames of Block 2, subjects were matched randomly with one of the subjects had already faced in Block 1 but not yet faced in Block 2.
Experiments have also been conducted on the (one-shot) sequential prisoners’ dilemma, and they generally show little difference from simultaneous-move setups. Bolle and Ockenfels (1990) found little difference in cooperation levels between simultaneous and sequential one-shot prisoners’ dilemma using the strategy method to elicit second-mover strategies. Brandts and Charness (2000) found no significant difference in cooperation between the sequential one-shot prisoner’s dilemma using the strategy method and direct response. Blanco et al. (2011) used the strategy method with role uncertainty in several information conditions and belief-elicitation treatments to show that correlation between strategies in different roles is driven partially (but not completely) by a consensus effect. Clark and Sefton (2001) examined sequential prisoners’ dilemma games with varying levels of temptation and overall stakes in both the United States and the United Kingdom. They found substantial cooperation levels in early rounds, which diminished by the tenth and final round. They also found that second-movers were much more likely to cooperate if the first-mover cooperated, but this tendency also decreased across rounds and with higher temptation levels. Higher overall stakes lead to a slight increase in second-mover reciprocal cooperation in the United Kingdom, but a decrease in reciprocal cooperation in the Unite States.
For both first and second-movers in both treatments, round 10 cooperation/conditional cooperation is never significantly higher in Block 2 than in Block 1 (Wilcoxon signed-rank tests for data having within-group correlations (Larocque 2005) with the unit of observation being the subject-level average cooperation/conditional cooperation in round 10 across all supergames in a block).
Fig. 4 
Second-mover average conditional cooperation
The unit of observation in these tests is the average (conditional) cooperation for an individual first (second) mover across all periods of all Block 2 supergames. As we are testing whether there is higher cooperation in 2S than 1S, the latter \(p\) values are based on a one-sided test in which the null is \(\mu _{2S}=\mu _{1S}\) and the alternative is that \(\mu _{2S}>\mu _{1S}\). The two-sided \(p\) values are 0.126 and 0.141, respectively.
Table 2 Aggregate cooperation We perform a power calculation for the 1S test, assuming the effect size found in 2S and a 10 % significance level, and find that our power is 85 %. The standard threshold for acceptable power is 80 %.
In the “Appendix,” we present similar results in which players are classified only by their type from the last supergame they play.
The proportion of second-movers who are classified as Cooperators may be inflated relative to imitators because first-movers defected first in 41.5 % of the Block 1 games with Cooperators. It is possible that these second-movers were using a reputation-building strategy that is not revealed because of the first-movers’ defection; however, classifying them as Cooperators is still useful because second-movers are not revealed to first-movers as rational.
\(P\) values are from a McNemar test for data having within-group correlations (Durkalski et al. 2003). We calculate the power of the 1S test to be 78 %, assuming the effect size observed in 2S.
As the type effects are relative to second-mover types in 1S, the estimates do not simply reflect a restart effect.
For Trusting first-movers, the difference in cooperation between 2S and 1S is \(2.125\), but only with \(p=0.259\).
This explanation is also consistent with the rates of defecting first shown in Table 6: in 2S, Trusting first-movers facing imitator-type second-movers defect first 68.4 % of the time, compared to 42.0 % in 1S.
We decided not to elicit beliefs before or during gameplay because doing so is difficult and may itself affect beliefs and behavior. We thank an anonymous referee for the suggestion of eliciting beliefs about third parties.
This approach is similar to Radner (1986), in which players have arbitrary beliefs about the opponent’s trigger strategy choice in a simultaneous-move FRPD and choose a best-response trigger strategy given these beliefs.
Selten and Stoecker (1986) propose an alternative non-Bayesian model of learning from histories of play in FRPDs, which predicts a general pattern of behavior that is consistent with our data. In their model, a player defects one period earlier (or later) with some probability if her previous opponent defected earlier (or later) than she did, and she defects in the same period otherwise. This learning model does not include beliefs about other players nor does it assume optimizing behavior, but only an iterative Markov transition learning rule given a starting point and supergame outcome. Unlike our experiment, subjects in Selten and Stoecker (1986) are given no information about opponents’ histories of play in prior supergames, and it is not clear how strategies would be updated in their model when players see the current opponent’s history of play against others. In contrast to Selten and Stoecker, we model players as Bayesian optimizers in a framework that is general enough to accommodate the informational environment of our experiment as well as most other FRPD experiments.
Evidence of non-equilibrium behavior like this is abundant in the experimental literature on strategic sophistication. See Crawford et al. (2013) for a recent survey.
This observation implies that cooperation would be sustained at least as long for more optimistic beliefs.
We restrict \(\mu (s_{k}) \in (0,1)\) so that Bayes’ rule can always be used. Without changing the results of this section, we could instead assume players update via Bayes’ rule whenever possible and allow beliefs to be free when zero-probability events are observed, and an opponent’s history does not eliminate any strategies and assign a new belief of zero when a particular strategy may be eliminated based on the opponent’s history.
Calculation of the conditional probabilities \(p_{t}\) and the conditions in Proposition 1 for a uniform prior show that this is the case. Take the first-mover for example. Because \(p_{t}\) decreases in \(t\) for these beliefs, \(s_{k}\) is optimal if and only if \(p_{k} \ge 4\big /7\) holds because \(p_{k} \ge 4\big /7\) implies that \(p_{l} \ge 4\big /\left[ \sum _{i=k+1}^{l} \bigl (3 \prod _{j=i}^{l-1} p_{j}\bigr ) + 7 \prod _{i=k}^{l-1} p_{i}\right] \) holds for all \(l \ge k\). \(s_{2}\) is optimal because \(p_{k} \ge 4\big /7\) holds for all \(k \ge 2\) and \(p_{1} < 4\big /7\). The calculation is similar for the second-mover.
References
Ambrus, A., Pathak, P.A.: Cooperation over finite horizons: a theory and experiments. J. Public Econ. 95, 500–512 (2011)
Anderlini, L., Lagunoff, R.: Communication in dynastic repeated games: “whitewashes” and “coverups”. Econ. Theory 26, 265–299 (2005)
Andreoni, J.: Why free ride? Strategies and learning in public goods experiments. J. Public Econ. 37, 291–304 (1988)
Andreoni, J., Croson, R.: Partners vs. strangers: random rematching in public goods experiments, working Paper (1998)
Andreoni, J., Miller, J.H.: Rational cooperation in the finitely repeated prisoners’ dilemma: experimental evidence. Econ. J. 103(418), 570–585 (1993)
Blanco, M., Engelmann, D., Koch, A.K., Normann, H.T.: Preferences and beliefs in a sequential social dilemma: a within-subjects analysis, working Paper (2011)
Bolle, F., Ockenfels, P.: Prisoners’ dilemma as a game with incomplete information. J. Econ. Psychol. 11, 69–84 (1990)
Bolton, G.E., Brandts, J., Katok, E.: How strategy sensitive are contributions? Econ. Theory 15, 367–387 (2000)
Bolton, G.E., Katok, E., Ockenfels, A.: Cooperation among strangers with limited information about reputation. J. Public Econ. 89, 1457–1468 (2005)
Brandts, J., Charness, G.: Hot vs. cold: sequential responses and preference stability in experimental games. Exp. Econ. 2, 227–238 (2000)
Brandts, J., Charness, G.: The strategy versus the direct-response method: a first survey oif experimental comparisons. Exp. Econ. 14, 375–398 (2011)
Brandts, J., Figueras, N.: An exploration of reputation formation in experimental games. J. Econ. Behav. Organ. 50(1), 89–115 (2003)
Bruttel, L.V., Güth, W., Kamecke, U.: Finitely repeated prisoners’ dilemma experiments without a commonly known end. Int. J. Game Theory 41(1), 23–47 (2012)
Camera, G., Casari, M.: Cooperation among strangers under the shadow of the future. Am. Econ. Rev. 99, 979–1005 (2009)
Camerer, C., Weigelt, K.: Experimental tests of a sequential equilibrium reputation model. Econometrica 56(1), 1–36 (1988)
Chakravorti, B., Conley, J., Taub, B.: On uniquely implementing cooperation in the prisoners’ dilemma. Econ. Theory 8, 347–366 (1996)
Clark, K., Sefton, M.: The sequential prisoner’s dilemma: evidence on reciprocation. Econ. J. 111, 51–68 (2001)
Cooper, R., DeJong, D., Forsythe, R., Ross, T.W.: Cooperation without reputation: experimental evidence from prisoner’s dilemma games. Games Econ. Behav. 12, 187–218 (1996)
Crawford, V.P., Costa-Gomes, M.A., Iriberri, N.: Structural models of nonequilibrium strategic thinking: theory, evidence, and applications. J. Econ. Lit. 51(1), 5–62 (2013)
Bó, Dal: P.: Cooperation under the shadow of the future: experimental evidence from infinitely repeated games. Am. Econ. Rev. 95, 1591–1604 (2005)
Bó, Dal: P., Frechette, G.: The evolution of cooperation in repeated games: experimental evidence. Am. Econ. Rev. 101, 411–429 (2011)
Duffy, J., Ochs, J.: Cooperative behavior and the frequency of social interaction. Games Econ. Behav. 66, 758–812 (2009)
Duffy, J., Xie, H., Lee, Y.J.: Social norms, information, and trust among strangers: theory and evidence. Econ. Theory 52, 669–708 (2013)
Durkalski, V., Palesch, Y., Lipsitz, S., Rust, P.: Analysis of clustered matched-pair data. Stat. Med. 22, 2417–2428 (2003)
Embrey, M., Fréchette, G.R., Yuksel, S.: Cooperation in the finitely repeated prisoner’s dilemma, Working Paper (2014)
Engle-Warnick, J., Slonim, R.L.: Inferring repeated-game strategies from actions: evidence from trust game experiments. Econ. Theory 28, 603–632 (2006)
Gachter, S., Thoni, C.: Social learning and voluntary cooperation among like-minded people. J. Eur. Econ. Assoc. 3(2–3), 303–314 (2005)
Gong, B., Yang, C.L.: Reputation and cooperation: an experiment on prisoner’s dilemma with second-order information, working Paper (2010)
Healy, P.J.: Group reputations, stereotypes, and cooperation in a repeated labor market. Am. Econ. Rev. 97(5), 1751–1773 (2007)
Ho, T.H., Su, X.: Peer-induced fairness in games. Am. Econ. Rev. 99, 2022–2049 (2009)
Irlenbusch, B., Sliwka, D.: Incentives, decision frames, and motivation crowding out—an experimental investigation, IZA Discussion Paper No. 1758 (2005)
Jung, Y.J., Kagel, J.H., Levin, D.: On the existence of predatory pricing: an experimental study of reputation and entry deterrence in the chain-store game. RAND J. Econ. 25(1), 72–93 (1994)
Kagel, J.H., McGee, P.: Team versus individual play in finitely repeated prisoner dilemma games, Working Paper (2014)
Kahn, L.M., Murnighan, J.K.: Conjecture, uncertainty, and cooperation in prisoner’s dilemma games. J. Econ. Behav. Organ. 22, 91–117 (1993)
King-Casas, B., Tomlin, D., Anen, C., Camerer, C.F., Quartz, S.R., Montague, P.R.: Getting to know you: reputation and trust in a two-person economic exchange. Science 308, 78–83 (2005)
Kreps, D.M., Milgrom, P., Roberts, J., Wilson, R.: Rational cooperation in the finitely repeated prisoners’ dilemma. J. Econ. Theory 27, 245–252 (1982)
Larocque, D.: The Wicoxon signed-rank test for cluster correlated data. In: Duchesne, P., Rémillard, B. (eds.) Statistical modeling and analysis for complex data problems, pp. 309–323. Springer, US (2005)
Murnighan, J.K., Roth, A.E.: Expecting continued play in prisoner’s dilemma games. J. Confl. Resol. 27, 279–300 (1983)
Neral, J., Ochs, J.: The sequential equilibrium theory of reputation building: a further test. Econometrica 60(5), 1151–1169 (1992)
Nishihara, K.: A resolution of N-person prisoners’ dilemma. Econ. Theory 10, 531–540 (1997)
Normann, H.T., Wallace, B.: The impact of the termination rule on cooperation in a prisoner’s dilemma experiment. Int. J. Game Theory 41(3), 707–718 (2012)
Nowak, M.A., Sigmund, K.: Evolution of indirect reciprocity. Nature 437, 1291–1298 (2005)
Radner, R.: Can bounded rationality resolve the prisoners’ dilemma? In: Mas-Colell, A., Hildenbrand, W. (eds.) Contributions to mathematical economics, pp. 387–399. North-Holland, Amsterdam (1986)
Rao, J.N.K., Scott, A.J.: The analysis of categorical data from complex sample surveys: Chi-squared tests for goodness of fit and independence in two-way tables. J. Am. Stat. Assoc. 76, 221–230 (1981)
Rao, J.N.K., Scott, A.J.: On chi-squared tests for multiway contingency tables with cell proportions estimated from survey data. Ann. Stat. 12, 46–60 (1984)
Reuben, E., Suetens, S.: Revisiting strategic versus non-strategic cooperation. Exp. Econ. 15, 24–43 (2012)
Roe, B.E., Wu, S.Y.: Do the selfish mimic cooperators? Experimental evidence from finitely-repeated labor markets, working Paper (2009)
Roth, A.E., Murnighan, J.K.: Equilibrium behavior and repeated play of the prisoner’s dilemma. J. Math. Psychol. 11, 189–198 (1978)
Samuelson, L.: A note on uncertainty and cooperation in a finitely repeated prisoners’ dilemma. Int. J. Game Theory 16, 187–195 (1987)
Schwartz, S.T., Young, R.A., Zvinakis, K.: Reputation without repeated interaction: a role for public disclosures. Rev. Account. Stud. 5, 351–375 (2000)
Selten, R., Stoecker, R.: End behavior in sequences of finite repeated prisoner’s dilemma supergames: a learning theory approach. J. Econ. Behav. Organ. 7, 47–70 (1986)
Sribney, W.M.: Two-way contingency tables for survey or clustered data. Stata Tech. Bull. 45, 33–49 (1998)
Tingley, D.H., Walter, B.F.: The effect of repeated play on reputation building: an experimental approach. Int. Organ. 65, 343–365 (2011)
Acknowledgments
The authors thank Yaron Azrieli, Lucas Coffman, Glenn Dutcher, John Kagel, Semin Kim, Peter McGee, Xiangyu Qu, Arno Riedl, Dan Schley, Mike Sinkey, Tom Wilkening, and Chao Yang for their helpful comments and conversations. We are also grateful to two anonymous referees whose comments and insights have helped us to significantly improve the paper. Healy acknowledges financial support from National Science Foundation Grant #SES-0847406. Any opinions, findings, conclusions, or recommendations expressed are those of the authors and do not necessarily reflect the views of the Federal Trade Commission.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Appendix
Appendix
Proposition 4
Let \(p\in (0,1)\) be the common belief that the other player plays tit for tat and \(\overline{p}_t\) the period \(t\) posterior belief. The following is a sequential equilibrium for a sequential-move FRPD.
-
(a)
The second-mover plays tit for tat in round \(t\) with probability
$$\begin{aligned} q_{t}\left( p\right) =\min \left\{ \frac{p}{1-p}\frac{1-\bar{p}_{t+1}}{\bar{p} _{t+1}},1\right\} . \end{aligned}$$Otherwise, the second-mover defects in round \(t\).
-
(b)
The first-mover cooperates in round \(t\) if and only if \(t \ge t^{*}(p)\) where
$$\begin{aligned} t^{*}\left( p\right) =\min \left\{ t\in \mathbb {N}:p\ge \bar{p}_{t}\right\} \text { and } \bar{p}_{t} = \left( \frac{4}{7}\right) ^{t} \text { hold for all }t. \end{aligned}$$Otherwise, the first-mover defects in round \(t\).
Proof
It is easier notationally to derive the equilibrium by counting backwards with \(t=10\) representing the first round of the supreme. In the body of the paper, however, time is indexed forward with \(t=1\) representing the first round of the supreme. Now, in any period \(t\), the first-mover will cooperate if
where \(V_{t-1}\left( p\right) \) is the continuation value of the first-mover entering period \(t-1\) with belief \(p\). Let \(V_{0}\equiv 0\). Let \(\bar{p}_{t}\) be the smallest value of \(p_{t}\) satisfying this inequality. (We will show later that the inequality in fact grows in \(p_{t}\).)
The probability a selfish second-mover cooperates is the highest \(q\) such that the first-mover is willing to cooperate in periods \(t-1\) after observing cooperation in period \(t\). Thus, if \(\bar{p}_{t}\) is the lowest belief at which first-mover will cooperate in period \(t\), then \(q_{t}\left( p\right) \) solves
and so
For completeness, let \(q_{t}\left( 1\right) =1\) for all \(t\) and \(q_{t}\equiv 1 \) for any \(t\) where \(\bar{p}_{t-1}=0\). Since a selfish second-mover never cooperates in the last period, set \(q_{1}\left( p\right) =0\) for all \(p\). (This is equivalent to setting \(\bar{p}_{0}=1\).)
For any \(t>1\), consider the case where \(p_{t}\ge \bar{p}_{t-1}\). Here, \( q_{t}\left( p_{t}\right) =1\) (the second-mover cooperates with certainty) and
so the above inequality becomes
or
In other words, the first-mover always cooperates if \(p_{t}\ge \bar{p}_{t-1}\) . This proves that \(\bar{p}_{t}\le \bar{p}_{t-1}\).
Now, suppose \(p_{t}<\bar{p}_{t-1}\). Here,
and so
The above inequality becomes
After several steps of algebra, this reduces to
Since \(\bar{p}_{t}\) solves this inequality exactly, it has the property that
In \(t=1\), the first-mover cooperates if
and so
Thus,
or
Note that \(V_{1}\left( p_{1}\right) =4=V_{1}\left( 0\right) \) for any \( p_{1}\le \bar{p}_{1}\).
In \(t=2\), we know that if \(p_{2}\ge \bar{p}_{1}=4/7\), then the first-mover cooperates with certainty.
If \(p_{2}<\bar{p}_{1}\), then he will cooperate only if \(p_{2}\ge \bar{p}_{2}\) , where \(\bar{p}_{2}\) solves
The expression for \(V_{2}\left( p\right) \) is given by
which is equal to
Note that \(V_{2}\left( p_{2}\right) =8=V_{2}\left( 0\right) \) for any \( p_{2}\le \bar{p}_{2}\).
In \(t=3\), we know that if \(p_{3}\ge \bar{p}_{2}\), then the first-mover cooperates with certainty.
If \(p_{3}<\bar{p}_{2}\), then he will cooperate only if \(p_{3}\ge \bar{p}_{3}\), where \(\bar{p}_{3}\) solves
The expression for \(V_{3}\left( p\right) \) is given by
which is equal to
In general, we will have
and
Let
First-movers will cooperate in period \(T\) if \(p_{T}^{*}\ge \bar{p}_{T}\). Thereafter, they will cooperate as long as they have never seen a defection and will never cooperate after seeing a defection. In that case, beliefs will evolve according to the formula
Beliefs change to \(p_{t}^{*}=0\) if a defection is observed in any previous period. If \(p_{T}^{*}<\bar{p}_{T}\), then both players always defect and \(p_{t}^{*}=p_{T}^{*}\) for every period \(t\). The on-path continuation value of the first-mover will equal
where we set \(\bar{p}_{0}=1\).
Proof of Proposition 1
(a) Let the first-mover’s expected payoff in round \(s\) from the remaining rounds \(1,\ldots ,s\) given beliefs \(p_{1},\ldots ,p_{s}\) be denoted by \(V_{s}(p_{1},\ldots ,p_{s})\). The expected payoff for cooperating in round \(t\) is \(p_{t} (7 + V_{t-1}(p_{1},\ldots ,p_{t-1})) + (1 - p_{t}) V_{t-1}(0,\ldots ,0)\). The expected payoff for defecting in round \(t\), given that the second-mover will respond by defecting for at least one round, is at most \(4 + V_{t-1}(p_{1},\ldots ,p_{t} p_{t-1})\). Therefore, the first-mover plays tit for tat in period \(t\) if and only if the following inequality holds
We need to show that
if
and
otherwise.
The proof is by induction. First, we know that the first-mover cooperates in round \(1\) if and only if \(p_{1} 7 + (1 - p_{1}) 0 \ge 4\) holds. Therefore, if \(p_{1} \ge \frac{4}{7}\) holds, then we have \(V_{1}(p_{1}) = 7 p_{1}\), and if \(p_{1} < \frac{4}{7}\) holds, then we have \(V_{1}(p_{1}) = 4\), and the formula is true for \(t = 1\).
Now, assume that the formula holds for all rounds up to \(t-1\) and show that it holds for round \(t\). Assume that the following holds
if
and
otherwise.
The first-mover cooperates in round \(t\) if and only if
We have assumed that \(p_{l} \ge \frac{4}{\sum _{i=k+1}^{l} (3 \prod _{j=i}^{l-1} p_{j}) + 7 \prod _{i=k}^{l-1} p_{i}}\) holds for all \(l\) such that \(t-1 \ge l \ge k\). First, suppose also that \(p_{t}p_{t-1} \ge \frac{4}{\sum _{i=k+1}^{t-1} (3 \prod _{j=i}^{t-2} p_{j}) + 7 \prod _{i=k}^{t-2} p_{i}}\) holds. Then, the first-mover cooperates in round \(t\) if and only if
Now, suppose that \(p_{t}p_{t-1} < \frac{4}{\sum _{i=k+1}^{t-1} (3 \prod _{j=i}^{t-2} p_{j}) + 7 \prod _{i=k}^{t-2} p_{i}}\) holds. Then, the first-mover cooperates in round \(t\) if and only if
Hence, the first-mover cooperates in round \(t\) and \(V_{t}(p_{1},p_{2},\ldots ,p_{t}) = 4 (t-1) + \sum _{i=k+1}^{t} (3\prod _{j=i}^{t} p_{j}) + 7 \prod _{i=k}^{t} p_{i}\) if and only if \(p_{l} \ge \frac{4}{\sum _{i=k+1}^{l} (3 \prod _{j=i}^{l-1} p_{j}) + 7 \prod _{i=k}^{l-1} p_{i}}\) holds for all \(l\) such that \(t \ge l \ge k\). Otherwise, the first-mover defects in round \(t\) and \(V_{t}(p_{1},p_{2},\ldots ,p_{t}) = 4t\).
(b) Let the second-mover’s expected payoff in round \(s\) from the remaining rounds \(1,\ldots ,s-1\) given beliefs \(p_{1},\ldots ,p_{s-1}\) be denoted by \(V_{s}(p_{1},\ldots ,p_{s-1})\). The expected payoff for cooperating in round \(t\) is \(7 + p_{t} (7 + V_{t}(p_{1},\ldots ,p_{t-1})) + (1 - p_{t}) (4 + V_{t}(0,\ldots ,0))\). The expected payoff for defecting in round \(t\), given that the first-mover will respond by defecting for at least one round, is at most \(12 + V_{t}(p_{1},\ldots ,p_{t} p_{t-1})\). Therefore, the second-mover plays tit for tat in period \(t+1\) if and only if the following inequality holds
We need to show that
if
and
otherwise.
The proof is by induction. First, we know that defection is the dominant action for the second-mover in round \(1\). The second-mover cooperates in round \(2\) if and only if \(7 + p_{1} 12 + (1 - p_{1}) 4 \ge 12 + 4\) holds. Therefore, if \(p_{1} \ge \frac{5}{8}\) holds, then we have \(V_{2}(p_{1}) = 4 + 8 p_{1}\), and if \(p_{1} < \frac{5}{8}\) holds, then we have \(V_{2}(p_{1}) = 4\), and the formula is true for \(t = 1\).
Now, we assume that the formula holds for all rounds up to \(t-1\) and show that it holds for round \(t\). Assume that the following holds
if
and
otherwise.
The second-mover cooperates in round \(t+1\) if and only if
We have assumed that \(p_{l} \ge \frac{5}{\sum _{i=k+1}^{l} (3 \prod _{j=i}^{l-1} p_{j}) + 8 \prod _{i=k}^{l-1} p_{i}}\) holds for all \(l\) such that \(t-1 \ge l \ge k\). First, suppose that \(p_{t}p_{t-1} \ge \frac{5}{\sum _{i=k+1}^{t-1} (3 \prod _{j=i}^{t-2} p_{j}) + 8 \prod _{i=k}^{t-2} p_{i}}\) holds. Then, the second-mover cooperates in round \(t+1\) if and only if
Now, suppose that \(p_{t}p_{t-1} < \frac{5}{\sum _{i=k+1}^{t-1} (3 \prod _{j=i}^{t-2} p_{j}) + 8 \prod _{i=k}^{t-2} p_{i}}\) holds. Then, the second-mover cooperates in round \(t+1\) if and only if
Hence, the second-mover cooperates in round \(t+1\) and \(V_{t+1}(p_{1},p_{2},\ldots ,p_{t}) = 4 (t-1) + \sum _{i=k+1}^{t} (3\prod _{j=i}^{t} p_{j}) + 8 \prod _{i=k}^{t} p_{i}\) if \(p_{l} \ge \max \{\frac{1}{3},\frac{5}{\sum _{i=k+1}^{l} (3 \prod _{j=i}^{l-1} p_{j})\} + 8 \prod _{i=k}^{l-1} p_{i}}\) holds for all \(l\) such that \(t \ge l \ge k\). Otherwise, the second-mover defects in round \(t+1\) and \(V_{t+1}(p_{1},p_{2},\ldots ,p_{t}) = 4t\).
Proof of Proposition 2
By Proposition 1, the first-mover’s Block 1 strategy is \(s_{m}\) if and only if \(\mu \) is such that \(p_{k} \ge \frac{4}{\sum _{i=m+1}^{k} (3 \prod _{j=i}^{k-1} p_{j}) + 7 \prod _{i=m}^{k-1} p_{i}}\) holds for all \(k \in \{m,\ldots ,10\}\). This condition can be rewritten in terms of the prior beliefs \(\mu \) as
for all \(k \in \{m,\ldots ,10\}\). After several steps of algebra, the denominator of the right-hand side of the above inequality simplifies to \(\frac{1}{\sum _{i=1}^{k}\mu (s_{i})}((7 + 3(k - n))(\sum _{i=1}^{m}\mu (s_{i})) + 3\sum _{i=m+1}^{k}((k+1-i)\mu (s_{i})))\), and the condition can be simplified to
for all \(k \in \{m,\ldots ,10\}\).
Now, suppose that the first-mover’s prior beliefs satisfy \(\mu (s_{k+1}) \le (3\big /4) \sum _{i=1}^{k} \mu (s_{i})\) for all \(k \in \{m,\ldots ,10\}\). For \(k=m\), the above condition is satisfied trivially. We now show that the above condition is satisfied for \(k=m+r\) for any \(r \ge 1\). For any \(r \ge 1\), the inequality \(\mu (s_{m+r+1}) \le (3\big /4) \sum _{i=1}^{m+r} \mu (s_{i})\) can be rewritten as
where \(\delta = \frac{1}{4} \sum _{i=m+1}^{m+r} (3(1-m-r+i)+1) \mu (s_{i}) - \frac{3}{4}r \sum _{i=1}^{m} \mu (s_{i})\). If \(r = 1\), then \(\delta = \mu (s_{m+1}) - \frac{3}{4} \sum _{i=1}^{m} \mu (s_{i}) \le 0\) holds and the condition for the first-mover to play strategy \(s_{m}\) in Block 1 is satisfied. Now, suppose that \(r \ge 2\). We can rewrite \(\delta \) as follows:
where \(\gamma = \frac{1}{4} \sum _{i=m+1}^{m+r-2} (3(1-m-r+i)+1) \mu (s_{i})\). Note that if \(r \ge 2\), then \(\gamma < 0\). Therefore, we have the following
\(r \ge 2\) implies that \((\frac{3}{4})^{r-2} - r < 0\), so \(\delta < 0\) holds and the condition for the first-mover to play strategy \(s_{m}\) in Block 1 is satisfied.
Given that the second-mover always defected before her opponent by round \(n\) of Block 1 supergames, where \(m < n\), we have \(\tilde{\mu }(s_{k}) = 0\) for all \(k \le m\). Therefore, \(\tilde{p}_{l} = 0\) holds for all \(l \le m\). By Proposition 1, it follows that the first-mover’s Block 2 strategy is \(s_{m+t}\) for some \(t \ge 1\).
Proof of Proposition 3
(a) By Proposition 1, the first-mover’s Block 1 strategy is \(s_{n}\) if and only if \(\mu \) is such that \(p_{k} \ge \frac{4}{\sum _{i=n+1}^{k} (3 \prod _{j=i}^{k-1} p_{j}) + 7 \prod _{i=n}^{k-1} p_{i}}\) holds for all \(k \in \{n,\ldots ,10\}\). By similar logic to the proof of Proposition 2, if the first-mover’s prior beliefs satisfy \(\mu (s_{k+1}) \le (3\big /4) \sum _{i=1}^{k} \mu (s_{i})\) for all \(k \in \{n,\ldots ,10\}\), then the condition for the first-mover to play strategy \(s_{n}\) in Block 1 is satisfied. Given that the second-mover never defected before her opponent in rounds \(10,\ldots ,m\) of Block 1 supergames, where \(m < n\), we have \(\tilde{\mu }(s_{k}) = 0\) for all \(k > m\). Therefore, \(\tilde{p}_{l} = 1\) holds for all \(l \ge m\). By Proposition 1, it follows that the first-mover’s Block 2 strategy is \(s_{n-t}\) for some \(t \ge 1\).
(b) \(\mu (s_{11}) > 3\big /7\) implies that \(\mu (s_{11}) > (3\big /4) \sum _{i=1}^{10} \mu (s_{i})\) holds, so the condition for the first-mover to play strategy \(s_{11}\) in Block 1 is satisfied. Given that the second-mover never defected before her opponent in rounds \(10,\ldots ,m\) of Block 1 supergames, where \(m \le 10\), we have \(\tilde{\mu }(s_{k}) = 0\) for all \(k > m\). Therefore, \(\tilde{p}_{l} = 1\) holds for all \(l \ge m\). By Proposition 1, it follows that the first-mover’s Block 2 strategy is \(s_{11-t}\) for some \(t \ge 1\).
Rights and permissions
About this article
Cite this article
Cox, C.A., Jones, M.T., Pflum, K.E. et al. Revealed reputations in the finitely repeated prisoners’ dilemma. Econ Theory 58, 441–484 (2015). https://doi.org/10.1007/s00199-015-0863-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00199-015-0863-1