Strategy inference using the maximum likelihood estimation in the iterated prisoner’s dilemma game

Abstract

The iterated prisoner’s dilemma game has shown how cooperation evolves based on direct reciprocity. In the course of iterated interactions, a player can observe the co-player’s past actions, infer his or her strategy, and react to it. Among the above three processes, the fidelity of inference is relatively unexplored compared to the others. In this work, we explicitly construct an inference process between observation and reaction. Specifically, the focal player infers the co-player’s strategy by applying the maximum likelihood estimation to the observed sequence of actions. Our first finding is that the focal player’s inference is accurate when both players take only their last actions into consideration if the observed sequence is sufficiently long. To see the case in which inference must be inaccurate, we also set the focal player’s memory length to be shorter than the co-player’s. In this case, we choose a combination of Tit-for-tat and Anti-tit-for-tat (TA) as the co-player’s long-memory strategy. TA satisfies the following three conditions: (1) mutual cooperation is achieved when all players use the strategy, (2) the strategy exploits unconditional cooperation, and (3) a player using this strategy is not exploited repeatedly by any co-player. The short-memory player inaccurately infers TA as either Tit-for-tat, Win–Stay–Lose–Shift, or Grim trigger, depending on his or her own strategy. This work presents how a long-memory strategy is projected onto a short-memory one by inference with information loss. In addition, we suggest that each of those three well-known strategies could be a facet of a single successful strategy with a higher cognitive capacity.