Advertisement

Analysis and Optimization of Deep Counterfactual Value Networks

  • Patryk Hopner
  • Eneldo Loza MencíaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11117)

Abstract

Recently a strong poker-playing algorithm called DeepStack was published, which is able to find an approximate Nash equilibrium during gameplay by using heuristic values of future states predicted by deep neural networks. This paper analyzes new ways of encoding the inputs and outputs of DeepStack’s deep counterfactual value networks based on traditional abstraction techniques, as well as an unabstracted encoding, which was able to increase the network’s accuracy.

Keywords

Poker Deep neural networks Game abstractions 

References

  1. 1.
    Bowling, M., Burch, N., Johanson, M., Tammelin, O.: Heads-up limit hold’em poker is solved. Commun. ACM 60(11), 81–88 (2017)CrossRefGoogle Scholar
  2. 2.
    Burch, N., Bowling, M.: CFR-D: solving imperfect information games using decomposition. CoRR abs/1303.4441 (2013). http://arxiv.org/abs/1303.4441
  3. 3.
    Ganzfried, S., Sandholm, T.: Endgame solving in large imperfect-information games. In: Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2015, pp. 37–45, Richland, SC (2015)Google Scholar
  4. 4.
    Gibson, R.: Regret minimization in games and the development of champion multiplayer computer poker-playing agents. Ph.D. thesis, University of Alberta (2014)Google Scholar
  5. 5.
    Gilpin, A., Sandholm, T., Sørensen, T.B.: A heads-up no-limit texas hold’em poker player: discretized betting models and automatically generated equilibrium-finding programs. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 2, AAMAS 2008, pp. 911–918. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC (2008)Google Scholar
  6. 6.
    Hopner, P.: Analysis and optimization of deep counterfactual value networks. Bachelor’s thesis, Technische Universität Darmstadt (2018). http://www.ke.tu-darmstadt.de/bibtex/publications/show/3078
  7. 7.
    Hopner, P., Loza Mencía, E.: Analysis and optimization of deep counterfactual value networks (2018). http://arxiv.org/abs/1807.00900
  8. 8.
    Johanson, M.: Measuring the size of large no-limit poker games. CoRR abs/1302.7008 (2013). http://arxiv.org/abs/1302.7008
  9. 9.
    Johanson, M., Bard, N., Lanctot, M., Gibson, R., Bowling, M.: Efficient nash equilibrium approximation through Monte Carlo counterfactual regret minimization. In: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 2, AAMAS 2012, pp. 837–846. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC (2012)Google Scholar
  10. 10.
    Johanson, M., Burch, N., Valenzano, R., Bowling, M.: Evaluating state-space abstractions in extensive-form games. In: Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems, AAMAS 2013, pp. 271–278. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC (2013)Google Scholar
  11. 11.
    Johanson, M.B.: Robust strategies and counter-strategies: from superhuman to optimal play. Ph.D. thesis, University of Alberta (2016). http://johanson.ca/publications/theses/2016-johanson-phd-thesis/2016-johanson-phd-thesis.pdf
  12. 12.
    Kanungo, T., Mount, D.M., Netanyahu, N.S., Piatko, C.D., Silverman, R., Wu, A.Y.: An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 881–892 (2002).  https://doi.org/10.1109/TPAMI.2002.1017616CrossRefzbMATHGoogle Scholar
  13. 13.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014). http://arxiv.org/abs/1412.6980
  14. 14.
    Moravcík, M., et al.: Deepstack: expert-level artificial intelligence in no-limit poker. CoRR abs/1701.01724 (2017). http://arxiv.org/abs/1701.01724
  15. 15.
    Moravcík, M., et al.: Supplementary materials for deepstack: expert-level artificial intelligence in no-limit poker (2017). https://www.deepstack.ai/
  16. 16.
    Nash, J.: Non-cooperative games. Ann. Math. 54(2), 286–295 (1951)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Noam Brown, T.S.: Libratus: the superhuman AI for no-limit poker. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, pp. 5226–5228 (2017)Google Scholar
  18. 18.
    Schnizlein, D.P.: State translation in no-limit poker. Master’s thesis, University of Alberta (2009)Google Scholar
  19. 19.
    Tammelin, O.: Solving large imperfect information games using CFR+. CoRR abs/1407.5042 (2014). http://arxiv.org/abs/1407.5042
  20. 20.
    Zinkevich, M., Johanson, M., Bowling, M., Piccione, C.: Regret minimization in games with incomplete information. In: Platt, J.C., Koller, D., Singer, Y., Roweis, S.T. (eds.) Advances in Neural Information Processing Systems 20, pp. 1729–1736. Curran Associates, Inc. (2008). http://papers.nips.cc/paper/3306-regret-minimization-in-games-with-incomplete-information.pdf

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Knowledge Engineering GroupTechnische Universität DarmstadtDarmstadtGermany

Personalised recommendations