Abstract
This paper compares behavior under four different implementations of infinitely repeated games in the laboratory: the standard random termination method [proposed by Roth and Murnighan (J Math Psychol 17:189–198, 1978)] and three other methods that de-couple the expected number of rounds and the discount factor. Two of these methods involve a fixed number of repetitions with payoff discounting, followed by random termination [proposed by Sabater-Grande and Georgantzis (J Econ Behav Organ 48:37–50, 2002)] or followed by a coordination game [proposed in (Andersson and Wengström in J Econ Behav Organ 81:207–219, 2012; Cooper and Kuhn in Am Econ J Microecon 6:247–278, 2014a)]. We also propose a new method—block random termination—in which subjects receive feedback about termination in blocks of rounds. We find that behavior is consistent with the presence of dynamic incentives only with methods using random termination, with the standard method generating the highest level of cooperation. Subject behavior in the other two methods display two features: a higher level of stability in cooperation rates and less dependence on past experience. Estimates of the strategies used by subjects reveal that across implementations, even when the discount rate is the same, if interactions are expected to be longer defection increases and the use of the Grim strategy decreases.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Other methods, used mainly in other social sciences, involve not specifying the number of repetitions or announcing the number of repetitions, but playing for a very long time.
Zwick et al. (1992) study an infinite horizon game, an alternating bargaining game, with an exogenous termination probability and compare the results to prior experiments using payoff discounting. Results are quite similar even though the experiments use different procedures.
Note that one could also first have a fixed number of rounds without payoff discounting followed by random termination. Such a procedure has been implemented in some experiments (Feinberg and Husted 1993; Feinberg and Snyder 2002), but this changes the environment to a non-stationary one and, thus, for certain games, can introduce different equilibria. In Feinberg and Husted (1993), for example, which studies collusive behavior in duopoly markets, collusion levels are higher in the first part of a supergame (without random termination) relative to the second part (after random termination is introduced).
The Grim trigger strategy involves first cooperating, followed by cooperation as long as the other player cooperates, but defection forever if either player defects.
Andersson and Wengström (2012) use coordination games with two pareto ranked equilibria that allows agents to support cooperation in the preceding PD, but not necessarily the specific one used in this paper.
It has since been used by Wilson and Wu (2014).
As is usual for such a game, if
$$\delta \ge \frac{R-T}{P-T}$$(1)joint cooperation can be supported as part of a subgame perfect equilibrium.
Thus, the probability that the block to be played is the last, given that the previous block was not the last, is given by \(\sum \limits _{i=1}^{\rho }\left( 1-\delta \right) \delta ^{i-1}\) for \(\rho \ge 1\).
Sherstyuk et al. (2013) experimentally investigate the effect of paying only in the last round of a match (as opposed to all rounds) which eliminates the need to assume risk neutrality. They find no difference in behavior between the standard payment method and paying only in the last round of a match. Note also that Schley and Kagel (2013) find that behavior is not sensitive to presentation manipulation: i.e. if the payoffs are listed in cents or dollars does not affect cooperation rates.
This assumption is satisfied if subjects are playing strategies which are such that when they play against each other the resulting payoffs are equivalent to the set of payoffs achieved with only Grim and Always defect. For instance, Tit-For-Tat against AD gives the same payoff as Grim against AD.
Brandts and Charness (2000) find no difference between a “hot” and “cold” treatment in two one-shot games.
Dal Bó (2005) has a similar treatment (Pd1 with \(\delta = 0.5\)) where, given the continuation probability, \(\left( C,C\right)\) cannot be supported in equilibrium, but it is possible to construct an equilibrium in which players alternate between \(\left( D,C\right)\) and \(\left( C,D\right)\). He finds alternation between these two outcomes to be slightly higher in this treatment compared to another one with the same \(\delta\) but a different payoff structure, where this cannot be sustained in equilibrium. However, he concludes that there is only weak evidence to suggest that subjects play such an equilibrium.
Available at http://cess.nyu.edu/frechette/print/Frechette_2014b_inst.pdf. The computer interface was implemented using zTree Fischbacher (2007).
As \(\rho\) increases, the time in the laboratory required to conduct the alternative implementations becomes longer. Four seemed long enough without making the sessions with alternative methods prohibitively long.
In part two, R, the payoff to joint cooperation in round 4, is reduced to 10.1.
In part two, the payoff to \(\left( G,G\right)\) is reduced to 30.4.
Throughout the text, unless noted otherwise, the statistical tests are based on probit estimations allowing for clustering at the session level. For a discussion of potential sources of session-effects, see Fréchette (2012).
The tests in Table 5 include dummy regressors to control for the specific random sequence in a given session.
Length is redefined to be the number of rounds −3 in the case of D+RT to make the estimates comparable across treatments. Note, also, that length is the number of rounds used for payments in the case of BRT.
By cooperative strategies, in the context of this paper, we mean any strategy that starts with cooperation. Note that, although players may update their beliefs about other aspects of the strategy used by others, choices after round one are not exogenous of one own’s choice and, thus, using only round one avoids issues of endogeneity.
See for instance Dal Bó and Fréchette (2011), in which, the authors show that a learning model can account well for the aggregate evolution of that aspect of behavior over matches.
In the RT treatment, it starts at 40 % and ends at 18 %. In the other treatments, it starts close to 25 % and decreases to a rate between 7 and 10 %.
The increase in cooperation over the 12 matches (for treatments RT, D+RT, and BRT) is between 7 and 21 % points, depending on the treatments.
There are no sessions in the RT treatment where matches 1 and 2 last at least four rounds. Also, the first match to last at least four rounds in this treatment is the fifth match. Similarly, in the BRT treatment, the first match for which the fifth round within a match is observed is the fifth match. This is the reason why there are missing values in Fig. 3 for these treatments.
Our findings in Fig. 2 show that aggregate response to defection and cooperation exhibit similar differences in D+RT and in RT. The combination of these observations suggests that subjects are more likely to play miscoordinated strategies, such as variations of Tit-For-Tat, rather than Grim in this treatment. We postpone further discussion of this to Sect. 3.3.
This is done using an ordered probit (cooperation can decrease, stay the same, or increase) clustering by session.
We omit the earlier data to look at more-stable and more-experienced behavior, although the picture changes very little when we include all the data.
The drop at the end for RT can be explained by the fact that the sample of matches is changing as we look across rounds. In particular, there are 52 observations in round 10, but only 12 for round 11. To eliminate variations due to the fact that the matches that have x-many rounds vary, Fig. 5 presents a similar graph for all matches that lasted at least five rounds, but only looking at the first five rounds. In that case, the sample is of the same size for each round of a treatment and as can be seen similar patterns are observed.
We look at actions in matches besides the ones under consideration—e.g., in the column for matches 4–6: this is computed from matches 1–3 and 7–12.
In the case of the last column, including dummy variables for all but one treatment does not qualitatively change the results. Since both dummies are not statistically significant, nor are they jointly different from zero, they are not included. Table 10 reports the probit regression estimates, and Table 11 reports correlated random effects estimates.
Note that for matches 7–9, an F-test shows that the sum of the first three terms is significantly different from 0 (p < 0.01).This implies that outcomes of (C, C) and (D, D) in round one generate different levels of cooperation in round five. This suggests that, for a period of time, some subjects employed defection in the last round as a punishment strategy.
The figure is almost identical if, instead, the y-axis is computed for round five and all subsequent rounds.
See Embrey et al. (2016) for a review of behavior infinitely repeated PDs.
The interested reader can refer to the online appendix for a more detailed description of the estimation method.
Standard errors are obtained by bootstrapping. This is done by first drawing sessions and then subjects (both with replacement).
In our estimation, we include the 20 strategies considered in Fudenberg et al. (2012), which cover the commonly considered strategies in repeated prisoner’s dilemma experiments. We refer the reader to Table 13 for a description of these strategies. Our estimation results for the entire set of strategies can be found in Table 14.
In Table 14, to ensure that the treatment differences we observe are not driven by differences in the number of observations per match, we also estimate strategies for a subset of observations in D+RT and BRT, focusing only on the rounds in each match that are observed under all methods. For example if a match ended in round five in RT, we look only at the data from the first five rounds for the equivalent match in D+RT and BRT. The results are very similar when using all the data versus only this subset. For D+RT, the difference is that Grim and Win-Stay-Lose-Shift are statistically significant using the subset but not all the data, while 2-Tits-For-Tat and 2-Tits-For-2-Tats are statistically significant using all the data but not in the subset. This seems to suggest that identifying strategies with longer memory is more difficult with fewer choices per match. For BRT, Suspicious Tit-For-Tat is not significant using all the data but it is in the subset. On the other hand, 2-Tits-For-2-Tats is not significant in the subset, but significant for all the data. These are relatively small differences considering the number of strategies; and except in the case of Grim, each of these strategies represent less than 10 % of the data. Fudenberg et al. (2012) and Dal Bó and Fréchette (2015) have already pointed out that this method does not perform as well at identifying strategies that are present in small proportions.
This is in line with prior research indicating that in perfect monitoring environments, these three strategies can account for the majority of the data. As shown in Fudenberg et al. (2012), when moving to imperfect public monitoring, strategies become more lenient and more forgiving.
To do hypothesis testing between the treatments, we pool data from two treatments and rerun our estimation procedure, allowing for different distribution of strategies in the separate treatments, and use a Wald test. Point estimates for the distribution of strategies following this method are identical to the results we find when the estimation is done separately for each treatment.
These differences are much less pronounced if we include "suspicious" strategies amongst cooperative strategies. In that case we find 16, 26, and 25 % of defective but not suspicious strategies for RT, D+RT, and BRT respectively.
The easiest way to see this share of the population is to sum the population share playing strategies that start with defection and then subtract the share playing Always Defect.
The value of cooperation is often captured by the size of the basin of attraction of AD. Intuitively, if a subject is uncertain about whether he is playing against a subject following the Grim or AD strategy, the set of prior beliefs under which it is optimal to follow the Grim strategy relative to AD increases with \(\delta\).
Embrey et al. (2016) provides a meta-analysis of experimental research on the finitely repeated PD and analyzes how the interaction length affects cooperations rates.
Cooper and Kühn (2014a) find that free form communication generates cooperation primarily by allowing subjects to communicate explicit punishment threats in response to noncooperative actions. Consistent with this observation, they find that cooperation cannot be sustained when communication is structured and the limited message space does not allow for communication of contingent strategies. Cooper and Kühn (2014b) revisit the impact and nature of communication in their D+C design. They also manipulate the number of rounds played before the coordination game: one or two. As in Cooper and Kühn (2014a), their findings indicate that with free form communication cooperation emerges. Their results are consistent with ours in that, without communication, despite the fact that by the end of the experiment very few subjects in round one choose the high mutual payoff action, the vast majority of subjects do not choose the lowest payoff equilibrium in the coordination game. While lack of variation in round 1 play makes it difficult to look at contingent play in the coordination game, the fact that less than 5 % of subjects play the low payoff action in the coordination game suggests a limited link between choices in the coordination game and history of play in the first round. The same is not true, however, when they allow for free form communication. In that case, there is an important correlation between round one and round two choices, suggesting that subjects create and use dynamic incentives in these treatments.
We are wary of a cookbook approach to design and thus provide these with caution. Specific questions each have their own requirements that are difficult to anticipate in the abstract. Thus, these recommendations should be taken with a grain of salt.
Of these three methods, this one generates the lowest variance in expected payments. If controlling the budget is a priority, this implementation would be preferable. If this is a concern, however, the method that gives the most control on expected payment is the one presented in Sherstyuk et al. (2013) of using RT and paying only the last round. That method could be adapted to be used in combination with the BRT design.
It should be noted that this method generates a re-start effect (between blocks) that disappears with experience. For this reason, this method is better suited to experimental designs where many matches will be played.
References
Andersson, O., & Wengström, E. (2012). Credible communication and cooperation: Experimental evidence from multi-stage games. Journal of Economic Behavior & Organization, 81(1), 207–219.
Blonski, M., Ockenfels, P., & Spagnolo, G. (2011). Equilibrium selection in the repeated prisoner’s dilemma: Axiomatic approach and experimental evidence. The American Economic Journal: Microeconomics, 3(3), 164–192.
Brandts, J., & Charness, G. (2000). Hot versus cold: Sequential responses and preference stability in experimental games. Experimental Economics, 2(3), 227–238.
Cabral, L. M. B., Ozbay, E., & Schotter, A. (2014). Intrinsic and instrumental reciprocity: An experimental study. Games and Economic Behavior, 87, 100–121.
Cooper, D. J., & Kühn, K.-U. (2014a). Communication, renegotiation, and the scope for collusion. American Economic Journal: Microeconomics, 6(2), 247–278.
Cooper, D.J., & Kühn, K.U. (2014b). Cruel to be kind: An experimental study of communication, collusion, and threats to punish. Working paper.
Dal Bó, P. (2005). Cooperation under the shadow of the future: Experimental evidence from infinitely repeated games. The American Economic Review, 95, 1591–1604.
Dal Bó, P., & Fréchette, G. R. (2011). The evolution of cooperation in infinitely repeated games: Experimental evidence. The American Economic Review, 101(1), 411–429.
Dal Bó, P., & Fréchette, G. R. (2015). Strategy choice in the infinitely repeated prisoners dilemma. Working paper.
Dal Bó, P., & Fréchette, G. R. (2016). On the determinants of cooperation in infinitely repeated games: A survey. Journal of Economic Literature, forthcoming.
Dreber, A., Rand, D. G., Fudenberg, D., & Nowak, M. A. (2008). Winners don’t punish. Nature, 452, 348–351.
Embrey, M.S., Fréchette, G. R., & Yuksel, S. (2016). Cooperation in the finitely repeated prisoner’s dilemma. Working paper.
Engle-Warnick, J., & Slonim, R. L. (2006). Inferring repeated-game strategies from actions: Evidence from repeated trust game experiments. Economic Theory, 28(3), 603–632.
Feinberg, R., & Husted, T. A. (1993). An experimental test of discount-rate effects on collusive behaviour in duopoly markets. The Journal of Industrial Economics, 41, 153–160.
Feinberg, R., & Snyder, C. (2002). Collusion with secret price cuts: An experimental investigation. Economics Bulletin, 3(6), 1–11.
Fischbacher, U. (2007). z-tree: Zurich toolbox for readymade economic experiments. Experimental Economics, 10(2), 171–178.
Fréchette, G. R. (2012). Session-effects in the laboratory. Experimental Economics, 15(3), 485–498.
Fudenberg, D., Rand, D. G., & Dreber, A. (2012). Slow to anger and fast to forget: Leniency and forgiveness in an uncertain world. The American Economic Review, 102, 720–749.
Mailath, G. J., & Samuelson, L. (2006). Repeated games and reputations: Long-run relationships. New York: Oxford University Press.
Rand, D. G., Fudenberg, D., & Dreber, A. (2015). It is the thought that counts: The role of intentions in noisy repeated games. Journal of Economic Behavior & Organization, 116, 481–499.
Roth, A. E., & Murnighan, J. K. (1978). Equilibrium behavior and repeated play of the prisoner’s dilemma. Journal of Mathematical Psychology, 17, 189–198.
Sabater-Grande, G., & Georgantzis, N. (2002). Accounting for risk aversion in repeated prisoners’ dilemma games: An experimental test. Journal of Economic Behavior & Organization, 48(1), 37–50.
Schley, D. R., & Kagel, J. H. (2013). How economic rewards affect cooperation reconsidered. Economics Letters, 121(1), 124–127.
Sherstyuk, K., Tarui, N., & Saijo, T. (2013). Payment schemes in infinite-horizon experimental games. Experimental Economics, 16, 125–153.
Vespa, E.I. (2015). An experimental investigation of strategies in dynamic games. Working paper.
Wilson, A., & Wu, H. (2014). At-will Relationships: How an Option to Walk Away Affects Cooperation and Efficiency. Working Paper.
Zwick, R., Rapoport, A., & Howard, J. C. (1992). Two-person sequential bargaining behavior with exogenous breakdown. Theory and Decision, 32, 241–268.
Acknowledgments
We would like to thank two anonymous referees, David Cooper, Drew Fudenberg, Ryan Oprea, Andrew Schotter, Ekatarina Sherstyuk, Alistair Wilson, participants at the 2011 ESA Conference and the NYU experimental group for helpful comments, Emanuel Vespa for his help developing the ztree code to conduct the experiment; also CESS and the C.V. Starr Center for financial support. Fréchette gratefully acknowledges the support of the NSF via Grants SES-0924780, SES-1225779, and SES-1558857.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Fréchette, G.R., Yuksel, S. Infinitely repeated games in the laboratory: four perspectives on discounting and random termination. Exp Econ 20, 279–308 (2017). https://doi.org/10.1007/s10683-016-9494-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10683-016-9494-z