Simulation and Performance Assessment of Poker Agents

  • Luís Filipe Teófilo
  • Rosaldo Rossetti
  • Luís Paulo Reis
  • Henrique Lopes Cardoso
  • Pedro Alves Nogueira
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7838)

Abstract

The challenge in developing agents for incomplete information games resides in the fact that the maximum utility decision for given information set is not always ascertainable. For large games like Poker, the agents’ strategies require opponent modeling, since Nash equilibrium strategies are hard to compute. In light of this, simulation systems are indispensable for accurate assessment of agents’ capabilities. Nevertheless, current systems do not accommodate the needs of computer poker research since they were designed mainly as an interface for human players competing against agents. In order to contribute towards improving computer poker research, a new simulation system was developed. This system introduces scientifically unexplored game modes with the purpose of providing a more realistic simulation environment, where the agent must play carefully to manage its initial resources. An evolutionary simulation feature was also included so as to provide support for the improvement of adaptive strategies. The simulator has built-in odds calculation, an agent development API, other platform agents and several variants support and an agent classifier with realistic game indicators including exploitability estimation. Tests and qualitative analysis have proven this simulator to be faster and better suited for thorough agent development and performance assessment.

Keywords

Poker Simulation Opponent Modeling Game Theory Incomplete Information Games Exploitability Agent Validation Gamblers ruin 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Billings, D.: Computer Poker. University of Alberta (1995)Google Scholar
  2. 2.
    Billings, D., et al.: Opponent modeling in poker. In: Proceedings of the National Conference on Artificial Intelligence, pp. 493–499. John Wiley & Sons Ltd. (1998)Google Scholar
  3. 3.
    Billings, D., et al.: Using selective-sampling simulations in poker. In: AAAI Syring Symposium Search Techniques for Problem Solving Under Uncertainty and Incomplete Information, pp. 1–6 (1999)Google Scholar
  4. 4.
    Van den Broeck, G., Driessens, K., Ramon, J.: Monte-Carlo Tree Search in Poker Using Expected Reward Distributions. In: Zhou, Z.-H., Washio, T. (eds.) ACML 2009. LNCS, vol. 5828, pp. 367–381. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  5. 5.
    Campbell, M., et al.: Deep Blue. Artificial Intelligence 134(1-2), 57–83 (2002)MATHCrossRefGoogle Scholar
  6. 6.
    Chen, B., Ankenman, J.: The Mathematics of Poker. Conjelco (2006)Google Scholar
  7. 7.
    Davidson, A., et al.: Poker Academy Pro - The Ultimate Poker Software, http://www.poker-academy.com/
  8. 8.
    Epstein, R.A.: The Theory of Gambling and Statistical Logic. Academic Press Inc. (1995)Google Scholar
  9. 9.
    Gilpin, A., Sandholm, T.: Better automated abstraction techniques for imperfect information games, with application to Texas Hold’em poker. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2007, p. 1. ACM Press (2007)Google Scholar
  10. 10.
    Johanson, M.: Robust Strategies and Counter-Strategies: Building a Champion Level Computer Poker Player. University of Alberta (2007) Google Scholar
  11. 11.
    Johanson, M., Bowling, M.: Data biased robust counter strategies. In: Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 264–271 (2009)Google Scholar
  12. 12.
    Quek, H., et al.: Evolving Nash-optimal poker strategies using evolutionary computation. Frontiers of Computer Science in China 3(1), 73–91 (2009)CrossRefGoogle Scholar
  13. 13.
    Rubin, J., Watson, I.: Case-based strategies in computer poker. AI Communications 25(1), 19–48 (2012)MathSciNetGoogle Scholar
  14. 14.
    Rubin, J., Watson, I.: Computer poker: A review. Artificial Intelligence 175(5-6), 958–987 (2011)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Schatzberg, D.: Open Meerkat Bot Simulation Testbed, http://code.google.com/p/opentestbed/
  16. 16.
    Sklansky, D.: The Theory of Poker: A Professional Poker Player Teaches You How to Think Like One. Two Plus Two (2007)Google Scholar
  17. 17.
    Teófilo, L.F., et al.: A Simulation System to Support Computer Poker Research. In: 13th International Workshop on Multi-Agent Based Simulation at AAMAS Workshop Proceedings, Valência, pp. 81–92 (2012)Google Scholar
  18. 18.
    Teófilo, L.F., Passos, N., Reis, L.P., Cardoso, H.L.: Adapting Strategies to Opponent Models in Incomplete Information Games: A Reinforcement Learning Approach for Poker. In: Kamel, M., Karray, F., Hagras, H. (eds.) AIS 2012. LNCS, vol. 7326, pp. 220–227. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  19. 19.
    Teófilo, L.F., et al.: Computer Poker Research at LIACC. In: Computer Poker Symposium. AAAI (2012)Google Scholar
  20. 20.
    Teófilo, L.F.: Estimating the Probability of Winning for Texas Hold’em Poker Agents. In: Proceedings of the 6th Doctoral Symposium on Informatics Engineering, pp. 129–140 (2011)Google Scholar
  21. 21.
    Teófilo, L.F., Reis, L.P.: Building a No Limit Texas Hold’em Poker Agent based on Game Logs using Supervised Learning. In: Kamel, M., Karray, F., Gueaieb, W., Khamis, A. (eds.) AIS 2011. LNCS (LNAI), vol. 6752, pp. 73–82. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  22. 22.
    Teófilo, L.F., Reis, L.P.: HoldemML: A framework to generate No Limit Hold’em Poker agents from human player strategies. In: 6th Iberian Conference on Information Systems and Technologies (CISTI 2011), pp. 755–760. IEEE (2011)Google Scholar
  23. 23.
    Zinkevich, M., et al.: A new algorithm for generating equilibria in massive zero-sum games. In: Proceedings of the 22nd National Conference on Artificial intelligence, AAAI 2007, vol. 1, pp. 788–793 (2007)Google Scholar
  24. 24.
    Zinkevich, M., Littman, M.L.: The 2006 AAAI Computer Poker Competition. Journal of International Computer Games Association 29, 166–167 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Luís Filipe Teófilo
    • 1
  • Rosaldo Rossetti
    • 1
  • Luís Paulo Reis
    • 2
  • Henrique Lopes Cardoso
    • 1
  • Pedro Alves Nogueira
    • 1
  1. 1.LIACC – Artificial Intelligence and Computer Science Lab., FEUP – Faculty of EngineeringUniversity of Porto – DEIPortugal
  2. 2.LIACC – Artificial Intelligence and Computer Science Lab., EEUM – School of EngineeringUniversity of Minho – DSIPortugal

Personalised recommendations