The Success and Failure of Tag-Mediated Evolution of Cooperation

  • Austin McDonald
  • Sandip Sen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3898)


Use of tags to limit partner selection for playing has been shown to produce stable cooperation in agent populations playing the Prisoner’s Dilemma game. There is, however, a lack of understanding of how and why tags facilitate such cooperation. We start with an empirical investigation that identifies the key dynamics that result in sustainable cooperation in PD. Sufficiently long tags are needed to achieve this effect. A theoretical analysis shows that multiple simulation parameters including tag length, mutation rate and population size will have significant effect on sustaining cooperation. Experiments partially validate these observations. Additionally, we claim that tags only promote mimicking and not coordinated behavior in general, i.e., tags can promote cooperation only if cooperation requires identical actions from all group members. We illustrate the failure of the tag model to sustain cooperation by experimenting with domains where agents need to take complementary actions to maximize payoff.


Nash Equilibrium Multiagent System Average Payoff Multiagent Reinforcement Learning Proportionate Reproduction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Banerjee, B., Sen, S., Peng, J.: Fast concurrent reinforcement learners. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pp. 825–830 (2001)Google Scholar
  2. 2.
    Bowling, M., Veloso, M.: Rational and convergent learning in stochastic games. Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, 1021–1026 (2001)Google Scholar
  3. 3.
    Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 746–752. AAAI Press/MIT Press, Menlo Park (1998)Google Scholar
  4. 4.
    Hu, J., Wellman, M.P.: Multiagent reinforcement learning: Theoretical framework and an algorithm. In: Shavlik, J. (ed.) Proceedings of the Fifteenth International Conference on Machine Learning, pp. 242–250. Morgan Kaufmann, San Francisco (1998)Google Scholar
  5. 5.
    Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the Eleventh International Conference on Machine Learning, pp. 157–163. Morgan Kaufmann, San Mateo (1994)Google Scholar
  6. 6.
    Littman, M.L.: Friend-or-foe q-learning in general-sum games. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 322–328. Morgan Kaufmann, San Francisco (2001)Google Scholar
  7. 7.
    Littman, M.L., Stone, P.: Implicit negotiation in repeated games. In: Intelligent Agents VIII: Agent Theories, Architecture, And Languages, pp. 393–404 (2001)Google Scholar
  8. 8.
    Myerson, R.B.: Game Theory: Analysis of Conflict. Harvard University Press, Cambridge (1991)MATHGoogle Scholar
  9. 9.
    Nash, J.F.: Non-cooperative games. Annals of Mathematics 54, 286–295 (1951)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Brams, S.J.: Theory of Moves. Cambridge University Press, Cambridge (1994)MATHGoogle Scholar
  11. 11.
    Axelrod, R.: The Evolution of Cooperation. Basic Books, New York (1984)MATHGoogle Scholar
  12. 12.
    Dugatkin, L.A.: The evolution of cooperation: four paths to the evolution and maintenance of cooperative behavior. Bio Science 47, 355–361 (1997)Google Scholar
  13. 13.
    Sigmund, K., Nowak, M.A.: The alternating prisoner’s dilemma. Journal of Theoretical Biology 38, 262–275 (1994)Google Scholar
  14. 14.
    Stimpson, J.L., Goodrich, M.A., Walters, L.C.: Satisficing and learning cooperation in the prisoner’s dilemma. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pp. 535–540 (2001)Google Scholar
  15. 15.
    Trivers, R.: The evolution of reciprocal altruism. Quarterly Review of Biology 46, 35–57 (1972)CrossRefGoogle Scholar
  16. 16.
    Hales, D., Edmonds, B.: Can tags build working systems? frommabs to esoa. In: Engineering Self-Organising Systems. LNCS (LNAI), vol. 2977, pp. 186–194. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  17. 17.
    Hales, D., Edmonds, B.: Evolving social rationality for mas using "tags". In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, Melbourne, Australia, pp. 497–503. ACM Press, New York (2003)CrossRefGoogle Scholar
  18. 18.
    Holland, J.H., Holyoak, K., Nisbett, R., Thagard, P.: Induction: Processes of Inferences, Learning, and Discovery. MIT Press, Cambridge (1986)Google Scholar
  19. 19.
    Riolo, R.: The effects and evolution of tag-mediated selection of partners in populations playing the Iterated Prisoner’s Dilemma. In: Proceedings of the Seventh International Conference on Genetic Algorithms, pp. 378–385. Morgan Kaufmann Publishers, Inc., San Francisco (1997)Google Scholar
  20. 20.
    Riolo, R., Cohen, M.D., Axelrod, R.: Cooperation without reciprocity. Nature 414, 441–443 (2001)CrossRefGoogle Scholar
  21. 21.
    Allison, P.D.: The cultural evolution of beneficent norms. Social Forces 71, 279–301 (1992)CrossRefGoogle Scholar
  22. 22.
    Holland, J.: The effect of labels (tags) on social interactions. Technical ReportWorking Paper 93-10-064, Santa Fe Institute (1993)Google Scholar
  23. 23.
    Hales, D., Edmonds, B.: Change your tags fast! - a necessary condition for cooperation? In: Proceedings of the Joint Workshop on Multi-Agent and Multi-Agent-Based Simulation (2004)Google Scholar
  24. 24.
    Hales, D.: Self-organising, open and cooperative p2p societies - from tags to networks. In: Proceedings of the 2nd Workshop on Engineering Self-Organsing Applications (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Austin McDonald
    • 1
  • Sandip Sen
    • 1
  1. 1.Mathematics and Computer Science DepartmentUniversity of TulsaTulsaUSA

Personalised recommendations