From Darwin to Poincaré and von Neumann: Recurrence and Cycles in Evolutionary and Algorithmic Game Theory

  • Victor Boone
  • Georgios PiliourasEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11920)


Replicator dynamics, the continuous-time analogue of Multiplicative Weights Updates, is the main dynamic in evolutionary game theory. In simple evolutionary zero-sum games, such as Rock-Paper-Scissors, replicator dynamic is periodic [39], however, its behavior in higher dimensions is not well understood. We provide a complete characterization of its behavior in zero-sum evolutionary games. We prove that, if and only if, the system has an interior Nash equilibrium, the dynamics exhibit Poincaré recurrence, i.e., almost all orbits come arbitrary close to their initial conditions infinitely often. If no interior equilibria exist, then all interior initial conditions converge to the boundary. Specifically, the strategies that are not in the support of any equilibrium vanish in the limit of all orbits. All recurrence results furthermore extend to a class of games that generalize both graphical polymatrix games as well as evolutionary games, establishing a unifying link between evolutionary and algorithmic game theory. We show that two degrees of freedom, as in Rock-Paper-Scissors, is sufficient to prove periodicity.



Georgios Piliouras acknowledges MOE AcRF Tier 2 Grant 2016-T2-1-170, grant PIE-SGP-AI-2018-01 and NRF 2018 Fellowship NRF-NRFF2018-07. This work was partially done while Victor Boone was a visitor at SUTD under the supervision of Georgios Piliouras. Victor Boone thanks Bruno Gaujal and Panayotis Mertikopoulos for helping to arrange the visit and for their overall guidance and mentorship.


  1. 1.
    Akin, E., Losert, V.: Evolutionary dynamics of zero-sum games. J. Math. Biol. 20, 231–258 (1984)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Arora, S., Hazan, E., Kale, S.: The multiplicative weights update method: a meta-algorithm and applications. Theory Comput. 8(1), 121–164 (2012)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bailey, J.P., Gidel, G., Piliouras, G.: Finite regret and cycles with fixed step-size via alternating gradient descent-ascent. arXiv e-prints arXiv:1907.04392, July 2019
  4. 4.
    Bailey, J.P., Piliouras, G.: Multiplicative weights update in zero-sum games. In: ACM Conference on Economics and Computation (2018)Google Scholar
  5. 5.
    Bailey, J.P., Piliouras, G.: Fast and Furious learning in zero-sum games: vanishing regret with non-vanishing step sizes. In: NeurIPS (2019)Google Scholar
  6. 6.
    Bailey, J.P., Piliouras, G.: Multi-agent learning in network zero-sum games is a Hamiltonian system. In: AAMAS (2019)Google Scholar
  7. 7.
    Barreira, L.: Poincare recurrence: old and new. In: XIVth International Congress on Mathematical Physics, pp. 415–422. World Scientific (2006)Google Scholar
  8. 8.
    Bomze, I.M.: Lotka-Volterra equation and replicator dynamics: new issues in classification. Biol. Cybern. 72(5), 447–453 (1995)CrossRefGoogle Scholar
  9. 9.
    Boone, V., Piliouras, G.: From Darwin to Poincaré and von Neumann: recurrence and cycles in evolutionary and algorithmic game theory. ArXiv (2019).
  10. 10.
    Cheung, Y.K., Piliouras, G.: Vortices instead of equilibria in minmax optimization: chaos and butterfly effects of online learning in zero-sum games. In: COLT (2019)Google Scholar
  11. 11.
    Conley, C.C.: Isolated Invariant Sets and The Morse Index, Number 38. American Mathematical Soc., Providence (1978)CrossRefGoogle Scholar
  12. 12.
    Gaunersdorfer, A., Hofbauer, J.: Fictitious play, shapley polygons, and the replicator equation. Games Econ. Behav. 11(2), 279–303 (1995)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Harper, M.: Escort evolutionary game theory. Phys. D 240(18), 1411–1415 (2011)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Hofbauer, J.: Evolutionary dynamics for bimatrix games: a Hamiltonian system? J. Math. Biol. 34, 675–688 (1996)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Hofbauer, J., Sigmund, K.: Evolutionary Games and Population Dynamics. Cambridge University Press, Cambridge (1998)CrossRefGoogle Scholar
  16. 16.
    Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106(4), 620 (1957)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Karev, G.P.: Replicator equations and the principle of minimal production of information. Bull. Math. Biol. 72(5), 1124–1142 (2010)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Kleinberg, R., Ligett, K., Piliouras, G., Tardos, É.: Beyond the Nash equilibrium barrier. In: Symposium on Innovations in Computer Science (ICS) (2011)Google Scholar
  19. 19.
    Kleinberg, R., Piliouras, G., Tardos, É.: Multiplicative updates outperform generic no-regret learning in congestion games. In: ACM Symposium on Theory of Computing (STOC) (2009)Google Scholar
  20. 20.
    Kwon, J., Mertikopoulos, P.: A continuous-time approach to online optimization. J. Dyn. Games 4, 125 (2017)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Mai, T., Panageas, I., Ratcliff, W., Vazirani, V.V., Yunker, P.: Cycles in zero sum differential games and biological diversity. In: ACM EC (2018)Google Scholar
  22. 22.
    Mertikopoulos, P., Papadimitriou, C., Piliouras, G.: Cycles in adversarial regularized learning. In: Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 2703–2717. SIAM (2018)CrossRefGoogle Scholar
  23. 23.
    Nagarajan, S.G., Mohamed, S., Piliouras, G.: Three body problems in evolutionary game dynamics: convergence, periodicity and limit cycles. In: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, pp. 685–693. International Foundation for Autonomous Agents and Multi-Agent Systems (2018)Google Scholar
  24. 24.
    von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press, Princeton (1944)zbMATHGoogle Scholar
  25. 25.
    Omidshafiei, S., et al.: \(\upalpha \)-rank: multi-agent evaluation by evolution. Sci. Rep. 9 (2019)Google Scholar
  26. 26.
    Ostrovski, G., van Strien, S.: Piecewise linear Hamiltonian flows associated to zero-sum games: transition combinatorics and questions on ergodicity. Regul. Chaotic Dyn. 16(1–2), 128–153 (2011)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Page, K.M., Nowak, M.A.: Unifying evolutionary dynamics. J. Theor. Biol. 219(1), 93–98 (2002)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Panageas, I., Piliouras, G.: Average case performance of replicator dynamics in potential games via computing regions of attraction. In: Proceedings of the 2016 ACM Conference on Economics and Computation, pp. 703–720. ACM (2016)Google Scholar
  29. 29.
    Papadimitriou, C., Piliouras, G.: From Nash equilibria to chain recurrent sets: an algorithmic solution concept for game theory. Entropy 20(10), 782 (2018)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Papadimitriou, C., Piliouras, G.: Game dynamics as the meaning of a game. ACM SIGecom Exch. 16(2), 53–63 (2019)CrossRefGoogle Scholar
  31. 31.
    Piliouras, G., Nieto-Granda, C., Christensen, H.I., Shamma, J.S.: Persistent patterns: multi-agent learning beyond equilibrium and utility. In: AAMAS, pp. 181–188 (2014)Google Scholar
  32. 32.
    Piliouras, G., Schulman, L.J.: Learning dynamics and the co-evolution of competing sexual species. In: ITCS (2018)Google Scholar
  33. 33.
    Piliouras, G., Shamma, J.S.: Optimization despite chaos: convex relaxations to complex limit sets via Poincaré recurrence. In: Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 861–873. SIAM (2014)Google Scholar
  34. 34.
    Poincaré, H.: Sur le problème des trois corps et les équations de la dynamique. Acta Math. 13, 1–270 (1890)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Sandholm, W.H.: Population Games and Evolutionary Dynamics. MIT Press, Cambridge (2010)zbMATHGoogle Scholar
  36. 36.
    Sandholm, W.H., Dokumacı, E., Lahkar, R.: The projection dynamic and the replicator dynamic. Games Econ. Behav. 64(2), 666–683 (2008)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Sato, Y., Akiyama, E., Farmer, J.D.: Chaos in learning a simple two-person game. Proc. Nat. Acad. Sci. 99(7), 4748–4751 (2002)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Weibull, J.W.: Evolutionary Game Theory. MIT Press, Cambridge (1995)zbMATHGoogle Scholar
  39. 39.
    Zeeman, E.C.: Population dynamics from game theory. In: Nitecki, Z., Robinson, C. (eds.) Global Theory of Dynamical Systems. LNM, vol. 819, pp. 471–497. Springer, Heidelberg (1980). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.ENS de LyonLyonFrance
  2. 2.SUTDSingaporeSingapore

Personalised recommendations