Similarity-Based Retrieval and Solution Re-use Policies in the Game of Texas Hold’em

  • Jonathan Rubin
  • Ian Watson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6176)

Abstract

In previous papers we have presented our autonomous poker playing agent (SARTRE) that uses a memory-based approach to create a betting strategy for two-player, limit Texas Hold’em. SARTRE participated in the 2009 IJCAI Computer Poker Competition where the system was thoroughly evaluated by challenging a range of other computerised opponents. Since the competition SARTRE has undergone case-based maintenance. In this paper we present results from the 2009 Computer Poker Competition and describe the latest modifications and improvements to the system. Specifically, we investigate two claims: the first that modifying the solution representation results in changes to the problem coverage and the second that different policies for re-using solutions leads to changes in performance. Three separate solution re-use policies for making betting decisions are introduced and evaluated. We conclude by presenting results of self-play experiments between the pre and post maintenance systems.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Billings, D., Papp, D., Schaeffer, J., Szafron, D.: Poker as Testbed for AI Research. In: Mercer, R.E. (ed.) Canadian AI 1998. LNCS, vol. 1418, pp. 228–238. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  2. 2.
    Rubin, J., Watson, I.: A Memory-Based Approach to Two-Player Texas Hold’em. In: Nicholson, A., Li, X. (eds.) AI 2009. LNCS, vol. 5866, pp. 465–474. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Rubin, J., Watson, I.: SARTRE: System Overview, A Case-Based Agent for Two-Player Texas Hold’em. In: Workshop on Case-Based Reasoning for Computer Games, Eighth International Conference on Case-Based Reasoning (ICCBR 2009), Springer, Heidelberg (2009)Google Scholar
  4. 4.
    Leake, D.B., Wilson, D.C.: Categorizing Case-Base Maintenance: Dimensions and Directions. In: Smyth, B., Cunningham, P. (eds.) EWCBR 1998. LNCS (LNAI), vol. 1488, pp. 196–207. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  5. 5.
  6. 6.
    Morris, P.: Introduction to game theory. Springer, New York (1994)MATHGoogle Scholar
  7. 7.
    Johanson, M. B.: Robust Strategies and Counter-Strategies: Building a Champion Level Computer Poker Player. Masters thesis, University of Alberta (2007)Google Scholar
  8. 8.
    Billings, D., Burch, N., Davidson, A., Holte, R.C., Schaeffer, J., Schauenberg, T., Szafron, D.: Approximating Game-Theoretic Optimal Strategies for Full-scale Poker. In: IJCAI 2003, Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp. 661–668. Morgan Kaufmann, San Francisco (2003)Google Scholar
  9. 9.
    Gilpin, A., Sandholm, T.: A Competitive Texas Hold’em Poker Player via Automated Abstraction and Real-Time Equilibrium Computation. In: Proceedings, The Twenty-First National Conference on Artificial Intelligence (AAAI 2006), pp. 1007–1013. AAAI Press, Menlo Park (2006)Google Scholar
  10. 10.
    Gilpin, A., Sandholm, T.: Better Automated Abstraction Techniques for Imperfect Information Games, with Application to Texas Hold’em Poker. In: 6th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2007), IFAAMAS, pp. 192–200 (2007)Google Scholar
  11. 11.
    Zinkevich, M., Johanson, M., Bowling, M.H., Piccione, C.: Regret Minimization in Games with Incomplete Information. In: Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, pp. 1729–1736. MIT Press, Cambridge (2007)Google Scholar
  12. 12.
  13. 13.
    Billings, D., Davidson, A., Schauenberg, T., Burch, N., Bowling, M., Holte, R.C., Schaeffer, J., Szafron, D.: Game-Tree Search with Adaptation in Stochastic Imperfect-Information Games. In: van den Herik, H.J., Björnsson, Y., Netanyahu, N.S. (eds.) CG 2004. LNCS, vol. 3846, pp. 21–34. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    Billings, D., Castillo, L. P., Schaeffer, J., Szafron, D.: Using Probabilistic Knowledge and Simulation to Play Poker. In: AAAI/IAAI, pp. 697–703. AAAI Press, Menlo Park (1999)Google Scholar
  15. 15.
    Schweizer, I., Panitzek, K., Park, S.H., Furnkranz, J.: An Exploitative Monte-Carlo Poker Agent. Technical report, Technische Universitat Darmstadt (2009)Google Scholar
  16. 16.
    Van den Broeck, G., Driessens, K., Ramon, J.: Monte-Carlo Tree Search in Poker Using Expected Reward Distributions. In: Zhou, Z.-H., Washio, T. (eds.) ACML 2009. LNCS, vol. 5828, pp. 367–381. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  17. 17.
    Johanson, M., Zinkevich, M., Bowling, M. H.: Computing Robust Counter-Strategies. In: Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, pp. 721–728. MIT Press, Cambridge (2007)Google Scholar
  18. 18.
    Johanson, M., Bowling, M.: Data Biased Robust Counter Strategies. In: Twelfth International Conference on Artificial Intelligence and Statistics, pp. 264–271 (2009)Google Scholar
  19. 19.
    Rubin, J., Watson, I.: Investigating the Effectiveness of Applying Case-Based Reasoning to the Game of Texas Hold’em. In: Proceedings of the Twentieth International Florida Artificial Intelligence Research Society Conference, pp. 417–422. AAAI Press, Menlo Park (2007)Google Scholar
  20. 20.
    Watson, I., Rubin, J.: CASPER: A Case-Based Poker-Bot. In: Wobcke, W., Zhang, M. (eds.) AI 2008. LNCS (LNAI), vol. 5360, pp. 594–600. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  21. 21.
    2008 Poker Bot Competition Summary, http://www.cs.ualberta.ca/~pokert/2008/results/
  22. 22.
    2009 Poker Bot Competition Summary, http://www.cs.ualberta.ca/~pokert/2009/results/

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Jonathan Rubin
    • 1
  • Ian Watson
    • 1
  1. 1.Department of Computer ScienceUniversity of AucklandNew Zealand

Personalised recommendations