On the Rate of Convergence of Fictitious Play

  • Felix Brandt
  • Felix Fischer
  • Paul Harrenstein
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6386)

Abstract

Fictitious play is a simple learning algorithm for strategic games that proceeds in rounds. In each round, the players play a best response to a mixed strategy that is given by the empirical frequencies of actions played in previous rounds. There is a close relationship between fictitious play and the Nash equilibria of a game: if the empirical frequencies of fictitious play converge to a strategy profile, this strategy profile is a Nash equilibrium. While fictitious play does not converge in general, it is known to do so for certain restricted classes of games, such as constant-sum games, non-degenerate 2×n games, and potential games. We study the rate of convergence of fictitious play and show that, in all the classes of games mentioned above, fictitious play may require an exponential number of rounds (in the size of the representation of the game) before some equilibrium action is eventually played. In particular, we show the above statement for symmetric constant-sum win-lose-tie games.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bellman, R.: On a new iterative algorithm for finding the solutions of games and linear programming problems. In: Research Memorandum P-473, The RAND Corporation (1953)Google Scholar
  2. 2.
    Berger, U.: Fictitious play in 2×n games. Journal of Economic Theory 120, 139–154 (2005)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Berger, U.: Brown’s original fictitious play. Journal of Economic Theory 135, 572–578 (2007)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Berger, U.: Learning in games with strategic complementarities revisited. Journal of Economic Theory 143, 292–301 (2008)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Berger, U.: The convergence of fictitious play in games with strategic complementarities: A comment. MPRA Paper No. 20241, Munich Personal RePEc Archive (2009)Google Scholar
  6. 6.
    Brandt, F., Fischer, F.: On the hardness and existence of quasi-strict equilibria. In: Monien, B., Schroeder, U.-P. (eds.) SAGT 2008. LNCS, vol. 4997, pp. 291–302. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Brown, G.W.: Iterative solutions of games by fictitious play. In: Koopmans, T.C. (ed.) Activity Analysis of Production and Allocation, pp. 374–376. John Wiley and Sons, Inc., Chichester (1951)Google Scholar
  8. 8.
    Conitzer, V.: Approximation guarantees for fictitious play. In: Proc. of 47th Annual Allerton Conference on Communication, Control and Computing (2009)Google Scholar
  9. 9.
    Conitzer, V., Sandholm, T.: AWESOME: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents. Machine Learning 67(1-2), 23–43 (2007)CrossRefGoogle Scholar
  10. 10.
    Dantzig, G.B.: A proof of the equivalence of the programming problem and the game problem. In: Koopmans, T.C. (ed.) Activity Analysis of Production and Allocation, pp. 330–335. John Wiley & Sons Inc., Chichester (1951)Google Scholar
  11. 11.
    Fudenberg, D., Levine, D.: Universal consistency and cautious fictitious play. Journal of Economic Dynamics and Control 19, 1065–1089 (1995)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Ganzfried, S., Sandholm, T.: Computing an approximate jam/fold equilibrium for 3-player no-limit texas hold’em tournaments. In: Proc. of 7th AAMAS Conference, pp. 919–925 (2008)Google Scholar
  13. 13.
    Gjerstad, S.: The rate of convergence of continuous fictitious play. Journal of Economic Theory 7, 161–178 (1996)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Hahn, S.: The convergence of fictitious play in 3×3 games with strategic complementarities. Economic Letters 64, 57–60 (1999)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Hannan, J.: Approximation to Bayes risk in repeated plays. In: Dresher, M., Tucker, A.W., Wolfe, P. (eds.) Contributions to the Theory of Games, vol. 3, pp. 97–139. Princeton University Press, Princeton (1957)Google Scholar
  16. 16.
    Luce, R.D., Raiffa, H.: Games and Decisions: Introduction and Critical Survey. John Wiley & Sons Inc., Chichester (1957)MATHGoogle Scholar
  17. 17.
    Miyasawa, K.: On the convergence of the learning process in a 2×2 nonzero sum two-person game. In: Research Memorandum 33, Econometric Research Program. Princeton University, Princeton (1961)Google Scholar
  18. 18.
    Monderer, D., Sela, A.: A 2×2 game without the fictitious play property. Games and Economic Behavior 14, 144–148 (1996)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Monderer, D., Shapley, L.S.: Potential games. Games and Economic Behavior 14(1), 124–143 (1996)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Monderer, D., Shapley, L.S.: Fictitious play property for games with identical interests. Journal of Economic Theory 68, 258–265 (1996)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Nachbar, J.H.: “Evolutionary” selection dynamics in games: Convergence and limit properties. International Journal of Game Theory 19, 59–89 (1990)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Powers, R., Shoham, Y.: New criteria and a new algorithm for learning in multi-agent systems. In: Advances in Neural Information Processing Systems, vol. 17, pp. 1089–1096. MIT Press, Cambridge (2004)Google Scholar
  23. 23.
    Rabinovich, Z., Gerding, E., Polukarov, M., Jennings, N.R.: Generalised fictitious play for a continuum of anonymous players. In: Proc. of 21st IJCAI, pp. 245–250 (2009)Google Scholar
  24. 24.
    Robinson, J.: An iterative method of solving a game. Annals of Mathematics 54(2), 296–301 (1951)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Shapley, L.: Some topics in two-person games. In: Dresher, M., Shapley, L.S., Tucker, A.W. (eds.) Advances in Game Theory. Annals of Mathematics Studies, vol. 52, pp. 1–29. Princeton University Press, Princeton (1964)Google Scholar
  26. 26.
    von Neumann, J.: Zur Theorie der Gesellschaftspiele. Mathematische Annalen 100, 295–320 (1928)MathSciNetCrossRefGoogle Scholar
  27. 27.
    von Neumann, J.: A numerical method to determine optimum strategy. Naval Research Logistics Quarterly 1(2), 109–115 (1954)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Zhu, W., Wurman, P.R.: Structural leverage and fictitious play in sequential auctions. In: Proc. of 18th AAAI Conference, pp. 385–390 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Felix Brandt
    • 1
  • Felix Fischer
    • 2
  • Paul Harrenstein
    • 1
  1. 1.Institut für InformatikTechnische Universität MünchenMünchenGermany
  2. 2.Harvard School of Engineering and Applied SciencesCambridgeUSA

Personalised recommendations