Machine Learning

, Volume 67, Issue 1–2, pp 7–22 | Cite as

Slow emergence of cooperation for win-stay lose-shift on trees



We consider a group of agents on a graph who repeatedly play the prisoner’s dilemma game against their neighbors. The players adapt their actions to the past behavior of their opponents by applying the win-stay lose-shift strategy. On a finite connected graph, it is easy to see that the system learns to cooperate by converging to the all-cooperate state in a finite time. We analyze the rate of convergence in terms of the size and structure of the graph. Dyer et al. (2002) showed that the system converges rapidly on the cycle, but that it takes a time exponential in the size of the graph to converge to cooperation on the complete graph. We show that the emergence of cooperation is exponentially slow in some expander graphs. More surprisingly, we show that it is also exponentially slow in bounded-degree trees, where many other dynamics are known to converge rapidly.


Games on graphs Learning Prisoner’s dilemma game Win-Stay Lose-Shift Oriented percolation Emergence of cooperation 


  1. Aldous, D., & Fill, J. (2006). Reversible Markov Chains and Random Walks on Graphs, Book in Preparation.Google Scholar
  2. Axelrod, R. (1984). The Evolution of Cooperation. Basic Books.Google Scholar
  3. Berger, N., Kenyon, C., Mossel, E., & Peres, Y. (2005). Glauber dynamics on trees and hyperbolic graphs. Probab. Theory Related Fields, 131(3), 311–340. Extended abstract by Kenyon, Mossel and Peres appeared in Proceedings of 42nd IEEE Symposium on Foundations of Computer Science (FOCS) 2001, 568–578.MATHCrossRefMathSciNetGoogle Scholar
  4. Durrett, R. (1984). Oriented Percolation in Two Dimensions. The Annals of Probability, 12, 999–1040.MATHMathSciNetGoogle Scholar
  5. Durrett, R. (1996). Probability: theory and examples. Duxbury.Google Scholar
  6. Dyer, M., Goldberg, L. A., Greenhill, C., Istrate, G., & Jerrum, M. (2002). Convergence of the Iterated Prisoner's Dilemma Game. Combinatorics, Probability, and Computing, 11, 135–147.MATHCrossRefMathSciNetGoogle Scholar
  7. Friedman, J., Kahn, J., & Szemeredi, E. (1989). On the second eigenvalue in Random Regular Graphs. In Proceedings of ACM STOC.Google Scholar
  8. Fudenberg, D., & Levine, D. K. (1998). The Theory of Learning in Games. MIT Press.Google Scholar
  9. Kahale, N. (1995). Eigenvalues and Expansion of Regular Graphs. Journal of the ACM, 42, 1091–1106.MATHCrossRefMathSciNetGoogle Scholar
  10. Kittock, J. E. (1995). Emergent conventions and the structure of multi-agent systems. In L., Nadel and D., Stein (Eds.), Vol. VI of Santa Fe Institute Studies in the Sciences of Complexity Lecture, Santa Fe Institute, 1993 Lectures in Complex Systems: Proceedings of the 1993 Complex Systems Summer School, Addison-Wesley.Google Scholar
  11. Liggett, T. M. (1985). Interacting Particle Systems. Springer.Google Scholar
  12. Liggett, T. M. (1999). Stochastic Interacting Systems: Contact, Voter and Exclusion Processes. Springer.Google Scholar
  13. Nowak, M., & Sigmund, K. (1993). A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner's Dilemma game. Nature, 364, 56–58.CrossRefGoogle Scholar
  14. Shoham, Y., & Tennenholtz, M. (1993). Co-learning and the evolution of social activity. Mimeo.Google Scholar
  15. Shoham, Y., & Tennenholtz, M. (1997). On the emergence of social conventions: Modelling, analysis and simulations. Artificial Intelligence, 94, 139–166.MATHCrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of StatisticsUniversity of California, BerkeleyBerkeley

Personalised recommendations