Adaptive Agents in Coalition Formation Games

  • Alex K. Chavez
Part of the International Handbooks on Information Systems book series (INFOSYS)


Coalition formation games form an important subclass of mixed-motive strategic situations, in which players must negotiate competitively to secure contracts. This paper compares the performance of two learning mechanisms, reinforcement learning and counterfactual reasoning, for modeling play in such games. Previous work [CK04] found that while the former type of agent converged to theoretical solutions, they did so much more slowly than human subjects. The present work addresses this issue by allowing agents to update extensively based on counterfactual reasoning.


Reinforcement Learning Coalition Formation Coalition Structure Aspiration Level Mean Square Deviation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [AM64]
    R J. Aumann and M Maschler, The bargaining set for cooperative games, Advances in Game Theory (M Dresher, L S. Shapley, and A W. Tucker, eds.), Princeton University Press, 1964.Google Scholar
  2. [CH98]
    Colin F. Camerer and Teck-Hua Ho, Experience-weighted attraction learning in coordination games: Probability rules, heterogeneity, and time-variation, Journal of Mathematical Psychology 42 (1998), 305–326.PubMedGoogle Scholar
  3. [CH99]
    —, Experience-weighted attraction learning in normal form games, Econometrica 67 (1999), no. 4, 827–974.CrossRefGoogle Scholar
  4. [CK04]
    Alex K. Chavez and Steven O. Kimbrough, A model of human behavior in coalition formation games, Proceedings of the 4th Annual International Conference on Cognitive Modeling, July 2004.Google Scholar
  5. [DKL96]
    Garett O. Dworman, Steven O. Kimbrough, and James D. Laing, On automated discovery of models using genetic programming: Bargaining in a three-agent coalitions game, Journal of Management Information Systems 12 (1995-96), no. 3, 97–125.Google Scholar
  6. [DKL95a]
    —, Bargaining in a three-agent coalitions game: An application of genetic programming, Working Notes: AAAI-95 Fall Symposium Series, Genetic Programming (Boston, MA, November 10–12, 1995), AAAI, 1995, pp. 9–16.Google Scholar
  7. [DKL95b]
    —, On automated discovery of models using genetic programming in game-theoretic contexts, Proceedings of the Twenty-Eighth Annual Hawaii International Conference on System Sciences, Volume III: Information Systems: Decision Support and Knowledge-Based Systems (Los Alamitos, CA) (Jay F. Nunamaker, Jr. and Ralph H. Sprague, Jr., eds.), IEEE Computer Society Press, 1995, pp. 428–438.Google Scholar
  8. [DKL96]
    —, Bargaining by artificial agents in two coalition games: A study in genetic programming for electronic commerce, Genetic Programming 1996: Proceedings of the First Annual Genetic Programming Conference, July 28–31, 1996, Stanford University (John R. Koza, David E. Goldberg, David B. Fogel, and Rick L. Riolo, eds.), The MIT Press, 1996, pp. 54–62.Google Scholar
  9. [ER98]
    Ido Erev and Alvin E. Roth, Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria, The American Economic Review 88 (1998), no. 4, 848–881.Google Scholar
  10. [FL98]
    Drew Fudenberg and David K. Levine, The theory of learning in games, The MIT Press, Cambridge, MA, 1998.Google Scholar
  11. [Gal90]
    Charles R. Gallistel, The organization of learning, The MIT Press, Cambridge, MA, 1990.Google Scholar
  12. [GS93]
    Dhananjay K. Gode and Shyam Sunder, Allocative efficiency of markets with zero-intelligence traders: Market as a partial substitute for individual rationality, Journal of Political Economy 101 (1993), no. 1, 119–137.CrossRefGoogle Scholar
  13. [KLM96]
    Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore, Reinforcement learning: A survey, Journal of Artificial Intelligence Research 4 (1996), 237–285.Google Scholar
  14. [KR74]
    James P. Kahan and Amnon Rapoport, Test of the bargaining set and kernel models in three-person games, Game Theory as a Theory of Conflict Resolution (Anatol Rapoport, ed.), D. Reidel, Dordrecht, The Netherlands, 1974, pp. 119–160.Google Scholar
  15. [KR84]
    —, Theories of coalition formation, Lawrence Earlbaum Associates, Hillsdale, NJ, 1984.Google Scholar
  16. [LR57]
    R. Duncan Luce and Howard Raiffa, Games and decisions, John Wiley, New York, NY, 1957, Reprinted by Dover Books, 1989.Google Scholar
  17. [MF02]
    Michael W. Macy and Andreas Flache, Learning dynamics in social dilemmas, Proceedings of the National Academy of Science (PNAS) 99 (2002), no. suppl. 3, 7229–7236.Google Scholar
  18. [RE95]
    Alvin E. Roth and Ido Erev, Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term, Games and Economic Behavior 8 (1995), 164–212.CrossRefMathSciNetGoogle Scholar
  19. [SB98]
    Richar S. Sutton and Andrew G. Barto, Reinforcement learning: An introduction, The MIT Press, Cambridge, MA, 1998.Google Scholar
  20. [SC95]
    T. Sandholm and R. Crites, Multiagent reinforcement learning in iterated prisoner’s dilemma, Biosystems 37 (1995), 147–166, Special Issue on the Prisoner’s Dilemma.Google Scholar
  21. [SV99]
    R. Sarin and F. Vahid, Payoff assessments without probabilities: A simple dynamic model of choice, Games and Economic Behavior 28 (1999), 294–309.CrossRefGoogle Scholar
  22. [SV01]
    —, Predicting how people play games, Games and Economic Behavior 34 (2001), 104–122.CrossRefGoogle Scholar
  23. [Uhl90]
    Gerald R. Uhlich, Descriptive theories of bargaining, Springer-Verlag, Berlin, Germany, 1990.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Alex K. Chavez
    • 1
  1. 1.University of PennsylvaniaPhiladelphiaUSA

Personalised recommendations