Advertisement

Beyond Local Nash Equilibria for Adversarial Networks

  • Frans A. OliehoekEmail author
  • Rahul Savani
  • Jose Gallego
  • Elise van der Pol
  • Roderich Groß
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1021)

Abstract

Save for some special cases, current training methods for Generative Adversarial Networks (GANs) are at best guaranteed to converge to a ‘local Nash equilibrium’ (LNE). Such LNEs, however, can be arbitrarily far from an actual Nash equilibrium (NE), which implies that there are no guarantees on the quality of the found generator or classifier. This paper proposes to model GANs explicitly as finite games in mixed strategies, thereby ensuring that every LNE is an NE. We use the Parallel Nash Memory as a solution method, which is proven to monotonically converge to a resource-bounded Nash equilibrium. We empirically demonstrate that our method is less prone to typical GAN problems such as mode collapse and produces solutions that are less exploitable than those produced by GANs and MGANs.

Notes

Acknowledgments

This research made use of a GPU donated by NVIDIA. F.A.O. is funded by EPSRC First Grant EP/R001227/1. This project had received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 758824—INFLUENCE).

References

  1. 1.
    Arjovsky, M., Bottou, L.: Towards principled methods for training generative adversarial networks. In: ICLR (2017)Google Scholar
  2. 2.
    Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: ICML (2017)Google Scholar
  3. 3.
    Arora, S., Zhang, Y.: Do GANs actually learn the distribution? An empirical study, ArXiv e-prints (2017)Google Scholar
  4. 4.
    Arora, S., Ge, R., Liang, Y., Ma, T., Zhang, Y.: Generalization and equilibrium in generative adversarial nets (GANs). In: ICML (2017)Google Scholar
  5. 5.
    Aubin, J.P.: Optima and Equilibria: An Introduction to Nonlinear Analysis, vol. 140. Springer, Heidelberg (1998).  https://doi.org/10.1007/978-3-662-03539-9CrossRefzbMATHGoogle Scholar
  6. 6.
    Bosanský, B., Kiekintveld, C., Lisý, V., Pechoucek, M.: An exact double-oracle algorithm for zero-sum extensive-form games with imperfect information. J. AI Res. 51, 829–866 (2014)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Brown, G.W.: Iterative solution of games by fictitious play. Act. Anal. Prod. Alloc. 13(1), 374–376 (1951)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Chintala, S.: How to train a GAN? Tips and tricks to make GANs work. https://github.com/soumith/ganhacks (2016). Accessed 08 Feb 2018
  9. 9.
    Danskin, J.M.: Fictitious play for continuous games revisited. Int. J. Game Theory 10(3), 147–154 (1981)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Foster, D.J., Li, Z., Lykouris, T., Sridharan, K., Tardos, E.: Learning in games: robustness of fast convergence. In: NIPS 29 (2016)Google Scholar
  11. 11.
    Fudenberg, D., Levine, D.K.: The Theory of Learning in Games. MIT Press, Cambridge (1998)zbMATHGoogle Scholar
  12. 12.
    Ge, H., Xia, Y., Chen, X., Berry, R., Wu, Y.: Fictitious GAN: training GANs with historical models. ArXiv e-prints (2018)Google Scholar
  13. 13.
    Ghosh, A., Kulharia, V., Namboodiri, V.P., Torr, P.H.S., Dokania, P.K.: Multi-agent diverse generative adversarial networks. ArXiv e-prints (2017)Google Scholar
  14. 14.
    Goodfellow, I., et al.: Generative adversarial nets. In: NIPS 27 (2014)Google Scholar
  15. 15.
    Grnarova, P., Levy, K.Y., Lucchi, A., Hofmann, T., Krause, A.: An online learning approach to generative adversarial networks. In: ICLR (2018)Google Scholar
  16. 16.
    Grnarova, P., Levy, K.Y., Lucchi, A., Perraudin, N., Hofmann, T., Krause, A.: Evaluating GANs via duality. arXiv e-prints (2018)Google Scholar
  17. 17.
    Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In: NIPS 30 (2017)Google Scholar
  18. 18.
    Hoang, Q., Nguyen, T.D., Le, T., Phung, D.Q.: Multi-generator generative adversarial nets. In: ICLR (2018)Google Scholar
  19. 19.
    Hsieh, Y.P., Liu, C., Cevher, V.: Finding mixed Nash equilibria of generative adversarial networks. ArXiv e-prints (2018)Google Scholar
  20. 20.
    Im, D.J., Ma, A.H., Taylor, G.W., Branson, K.: Quantitatively evaluating GANs with divergences proposed for training. In: ICLR (2018)Google Scholar
  21. 21.
    Jiwoong Im, D., Ma, H., Dongjoo Kim, C., Taylor, G.: Generative adversarial parallelization. ArXiv e-prints (2016)Google Scholar
  22. 22.
    Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. In: ICLR (2018)Google Scholar
  23. 23.
    Li, W., Gauci, M., Groß, R.: Turing learning: a metric-free approach to inferring behavior and its application to swarms. Swarm Intell. 10(3), 211–243 (2016)CrossRefGoogle Scholar
  24. 24.
    McMahan, H.B., Gordon, G.J., Blum, A.: Planning in the presence of cost functions controlled by an adversary. In: ICML (2003)Google Scholar
  25. 25.
    Nash, J.F.: Equilibrium points in N-person games. Proc. Natl. Acad. Sci. U. S. A. 36, 48–49 (1950)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Nemirovski, A., Juditsky, A., Lan, G., Shapiro, A.: Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19(4), 1574–1609 (2009)MathSciNetCrossRefGoogle Scholar
  27. 27.
    von Neumann, J.: Zur Theorie der Gesellschaftsspiele. Math. Ann. 100(1), 295–320 (1928)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Odena, A., Olah, C., Shlens, J.: Conditional image synthesis with auxiliary classifier GANs. In: ICML (2017)Google Scholar
  29. 29.
    Oliehoek, F.A., de Jong, E.D., Vlassis, N.: The parallel Nash memory for asymmetric games. In: Proceedings of the Genetic and Evolutionary Computation (GECCO) (2006)Google Scholar
  30. 30.
    Oliehoek, F.A., Savani, R., Gallego-Posada, J., Van der Pol, E., De Jong, E.D., Groß, R.: GANGs: generative adversarial network games. ArXiv e-prints (2017)Google Scholar
  31. 31.
    Oliehoek, F.A., Savani, R., Gallego-Posada, J., van der Pol, E., Gross, R.: Beyond local Nash equilibria for adversarial networks. ArXiv e-prints (2018)Google Scholar
  32. 32.
    Osborne, M.J., Rubinstein, A.: Nash equilibrium. In: A Course in Game Theory. The MIT Press (1994)Google Scholar
  33. 33.
    Rakhlin, A., Sridharan, K.: Online learning with predictable sequences. In: COLT (2013)Google Scholar
  34. 34.
    Rakhlin, A., Sridharan, K.: Optimization, learning, and games with predictable sequences. In: NIPS 26 (2013)Google Scholar
  35. 35.
    Ratliff, L.J., Burden, S.A., Sastry, S.S.: Characterization and computation of local Nash equilibria in continuous games. In: Annual Allerton Conference on Communication, Control, and Computing. IEEE (2013)Google Scholar
  36. 36.
    Salimans, T., Goodfellow, I.J., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. In: NIPS 29 (2016)Google Scholar
  37. 37.
    Shoham, Y., Leyton-Brown, K.: Multi-Agent Systems: Algorithmic, Game-Theoretic and Logical Foundations. Cambridge University Press, Cambridge (2008)CrossRefGoogle Scholar
  38. 38.
    Sinn, M., Rawat, A.: Non-parametric estimation of Jensen-Shannon divergence in generative adversarial network training. In: AISTATS (2018)Google Scholar
  39. 39.
    Theis, L., van den Oord, A., Bethge, M.: A note on the evaluation of generative models. In: ICLR (2016)Google Scholar
  40. 40.
    Unterthiner, T., Nessler, B., Klambauer, G., Heusel, M., Ramsauer, H., Hochreiter, S.: Coulomb GANs: provably optimal Nash equilibria via potential fields. In: ICLR (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Frans A. Oliehoek
    • 1
    Email author
  • Rahul Savani
    • 2
  • Jose Gallego
    • 3
  • Elise van der Pol
    • 3
  • Roderich Groß
    • 4
  1. 1.Delft University of TechnologyDelftThe Netherlands
  2. 2.University of LiverpoolLiverpoolUK
  3. 3.University of AmsterdamAmsterdamThe Netherlands
  4. 4.The University of SheffieldSheffieldThe Netherlands

Personalised recommendations