Theory of Computing Systems

, Volume 53, Issue 1, pp 41–52 | Cite as

On the Rate of Convergence of Fictitious Play

Article

Abstract

Fictitious play is a simple learning algorithm for strategic games that proceeds in rounds. In each round, the players play a best response to a mixed strategy that is given by the empirical frequencies of actions played in previous rounds. There is a close relationship between fictitious play and the Nash equilibria of a game: if the empirical frequencies of fictitious play converge to a strategy profile, this strategy profile is a Nash equilibrium. While fictitious play does not converge in general, it is known to do so for certain restricted classes of games, such as constant-sum games, non-degenerate 2×n games, and potential games. We study the rate of convergence of fictitious play and show that, in all the classes of games mentioned above, fictitious play may require an exponential number of rounds (in the size of the representation of the game) before some equilibrium action is eventually played. In particular, we show the above statement for symmetric constant-sum win-lose-tie games.

Keywords

Game theory Nash equilibrium Fictitious play Rate of convergence 

References

  1. 1.
    Bellman, R.: On a new iterative algorithm for finding the solutions of games and linear programming problems. Research Memorandum P-473, The RAND Corporation (1953) Google Scholar
  2. 2.
    Berger, U.: Fictitious play in 2×n games. J. Econ. Theory 120, 139–154 (2005) MATHCrossRefGoogle Scholar
  3. 3.
    Berger, U.: Brown’s original fictitious play. J. Econ. Theory 135, 572–578 (2007) MATHCrossRefGoogle Scholar
  4. 4.
    Berger, U.: Learning in games with strategic complementarities revisited. J. Econ. Theory 143, 292–301 (2008) MATHCrossRefGoogle Scholar
  5. 5.
    Berger, U.: The convergence of fictitious play in games with strategic complementarities: a comment. MPRA Paper No. 20241, Munich Personal RePEc Archive, 2009 Google Scholar
  6. 6.
    Brandt, F., Fischer, F.: On the hardness and existence of quasi-strict equilibria. In: Monien, B., Schroeder, U.-P. (eds.) Proceedings of the 1st International Symposium on Algorithmic Game Theory. Lecture Notes in Computer Science, vol. 4997, pp. 291–302. Springer, Berlin (2008) CrossRefGoogle Scholar
  7. 7.
    Brown, G.W.: Iterative solutions of games by fictitious play. In: Koopmans, T.C. (ed.) Activity Analysis of Production and Allocation, pp. 374–376. Wiley, New York (1951) Google Scholar
  8. 8.
    Conitzer, V.: Approximation guarantees for fictitious play. In: Proceedings of the 47th Annual Allerton Conference on Communication, Control, and Computing, pp. 636–643 (2009) Google Scholar
  9. 9.
    Conitzer, V., Sandholm, T.: AWESOME: a general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents. Mach. Learn. 67(1–2), 23–43 (2007) CrossRefGoogle Scholar
  10. 10.
    Dantzig, G.B.: A proof of the equivalence of the programming problem and the game problem. In: Koopmans, T.C. (ed.) Activity Analysis of Production and Allocation, pp. 330–335. Wiley, New York (1951) Google Scholar
  11. 11.
    Fudenberg, D., Levine, D.: Consistency and cautious fictitious play. J. Econ. Dyn. Control 19, 1065–1089 (1995) MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Ganzfried, S., Sandholm, T.: Computing an approximate jam/fold equilibrium for 3-player no-limit Texas Hold’em tournaments. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multi-Agent Systems, pp. 919–925 (2008) Google Scholar
  13. 13.
    Gjerstad, S.: The rate of convergence of continuous fictitious play. J. Econ. Theory 7, 161–178 (1996) MathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    Goldberg, P.W., Savani, R., Sørensen, T.B., Ventre, C.: On the approximation performance of fictitious play in finite games. Int. J. Game Theory (2013). doi:10.1007/s00182-012-0362-6 Google Scholar
  15. 15.
    Hahn, S.: The convergence of fictitious play in 3×3 games with strategic complementarities. Econ. Lett. 64, 57–60 (1999) MATHCrossRefGoogle Scholar
  16. 16.
    Hannan, J.: Approximation to Bayes risk in repeated plays. In: Dresher, M., Tucker, A.W., Wolfe, P. (eds.) Contributions to the Theory of Games, vol. 3, pp. 97–139. Princeton University Press, Princeton (1957) Google Scholar
  17. 17.
    Karlin, S.: Mathematical Methods and Theory in Games, Programming, and Economics, vol. 1–2. Addison-Wesley, Reading (1959) Google Scholar
  18. 18.
    Luce, R.D., Raiffa, H.: Games and Decisions: Introduction and Critical Survey. Wiley, New York (1957) MATHGoogle Scholar
  19. 19.
    Miyasawa, K.: On the convergence of the learning process in a 2×2 nonzero sum two-person game. Research Memorandum 33, Econometric Research Program, Princeton University, 1961 Google Scholar
  20. 20.
    Monderer, D., Sela, A.: A 2×2 game without the fictitious play property. Games Econ. Behav. 14, 144–148 (1996) MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Monderer, D., Shapley, L.S.: Potential games. Games Econ. Behav. 14(1), 124–143 (1996) MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    Monderer, D., Shapley, L.S.: Fictitious play property for games with identical interests. J. Econ. Theory 68, 258–265 (1996) MathSciNetMATHCrossRefGoogle Scholar
  23. 23.
    Nachbar, J.H.: “Evolutionary” selection dynamics in games: convergence and limit properties. Int. J. Game Theory 19, 59–89 (1990) MathSciNetMATHCrossRefGoogle Scholar
  24. 24.
    Powers, R., Shoham, Y.: New criteria and a new algorithm for learning in multi-agent systems. In: Advances in Neural Information Processing Systems 17, pp. 1089–1096. MIT Press, Cambridge (2004) Google Scholar
  25. 25.
    Rabinovich, Z., Gerding, E., Polukarov, M., Jennings, N.R.: Generalised fictitious play for a continuum of anonymous players. In: Proceedings of the 21st International Joint Conference on Artificial Intelligence, pp. 245–250 (2009) Google Scholar
  26. 26.
    Robinson, J.: An iterative method of solving a game. Ann. Math. 54(2), 296–301 (1951) MATHCrossRefGoogle Scholar
  27. 27.
    Shapiro, H.: Note on a computation model in the theory of games. Commun. Pure Appl. Math. 11, 587–593 (1958) MATHCrossRefGoogle Scholar
  28. 28.
    Shapley, L.: Some topics in two-person games. In: Dresher, M., Shapley, L.S., Tucker, A.W. (eds.) Advances in Game Theory. Annals of Mathematics Studies, vol. 52, pp. 1–29. Princeton University Press, Princeton (1964) Google Scholar
  29. 29.
    von Neumann, J.: Zur Theorie der Gesellschaftspiele. Math. Ann. 100, 295–320 (1928) MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    von Neumann, J.: A numerical method to determine optimum strategy. Nav. Res. Logist. Q. 1(2), 109–115 (1954) CrossRefGoogle Scholar
  31. 31.
    Zhu, W., Wurman, P.R.: Structural leverage and fictitious play in sequential auctions. In: Proceedings of the 18th National Conference on Artificial Intelligence, pp. 385–390 (2002) Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Institut für InformatikTechnische Universität MünchenGarchingGermany
  2. 2.Statistical LaboratoryUniversity of CambridgeCambridgeUK
  3. 3.Department of Computer ScienceUniversity of OxfordOxfordUK

Personalised recommendations