Game-Tree Search with Adaptation in Stochastic Imperfect-Information Games

  • Darse Billings
  • Aaron Davidson
  • Terence Schauenberg
  • Neil Burch
  • Michael Bowling
  • Robert Holte
  • Jonathan Schaeffer
  • Duane Szafron
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3846)

Abstract

Building a high-performance poker-playing program is a challenging project. The best program to date, PsOpti, uses game theory to solve a simplified version of the game. Although the program plays reasonably well, it is oblivious to the opponent’s weaknesses and biases. Modeling the opponent to exploit predictability is critical to success at poker. This paper introduces Vexbot, a program that uses a game-tree search algorithm to compute the expected value of each betting option, and does real-time opponent modeling to improve its evaluation function estimates. The result is a program that defeats PsOpti convincingly, and poses a much tougher challenge for strong human players.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Horvitz, E., Breese, L., Heckerman, D., Hovel, D., Rommeke, K.: The Lumiere project: Bayesian user modeling for inferring the goals and needs of software users. In: UAI, pp. 256–265 (1998)Google Scholar
  2. 2.
    Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.): UM 2003. LNCS, vol. 2702. Springer, Heidelberg (2003)MATHGoogle Scholar
  3. 3.
    Weld, D., Anderson, C., Domingos, P., Etzioni, O., Lau, T., Gajos, K., Wolfman, S.: Automatically personalizing user interfaces. In: IJCAI, pp. 1613–1619 (2003)Google Scholar
  4. 4.
    Billings, D., Burch, N., Davidson, A., Holte, R., Schaeffer, J., Schauenberg, T., Szafron, D.: Approximating game-theoretic optimal strategies for full-scale poker. In: IJCAI, pp. 661–668 (2003)Google Scholar
  5. 5.
    Jansen, P.: Using Knowledge about the Opponent in Game-Tree Search. PhD thesis, Computer Science, Carnegie-Mellon University (1992)Google Scholar
  6. 6.
    Carmel, D., Markovitch, S.: Opponent modeling in adversary search. In: AAAI, pp. 120–125 (1996)Google Scholar
  7. 7.
    Iida, H., Uiterwijk, J.W.H.M., van den Herik, H.J., Herschberg, I.S.: Potential applications of opponent-model search. ICCA Journal 16, 201–208 (1993)Google Scholar
  8. 8.
    Billings, D., Davidson, A., Schaeffer, J., Szafron, D.: The challenge of poker. Artificial Intelligence 134, 201–240 (2002)MATHCrossRefGoogle Scholar
  9. 9.
    Dahl, F.: A reinforcement learning algorithm to simplified two-player Texas Hold’em poker. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 85–96. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  10. 10.
    Korb, K., Nicholson, A., Jitnah, N.: Bayesian poker. In: UAI, pp. 343–350 (1999)Google Scholar
  11. 11.
    Findler, N.: Studies in machine cognition using the game of poker. CACM 20, 230–245 (1977)MATHGoogle Scholar
  12. 12.
    von Neumann, J., Morgenstern, O.: The Theory of Games and Economic Behavior. Princeton University Press, Princeton (1944)Google Scholar
  13. 13.
    Kuhn, H.W.: A simplified two-person poker. Contributions to the Theory of Games 1, 97–103 (1950)Google Scholar
  14. 14.
    Billings, D., Papp, D., Schaeffer, J., Szafron, D.: Opponent modeling in poker. In: AAAI, pp. 493–499 (1998)Google Scholar
  15. 15.
    Billings, D., Peña, L., Schaeffer, J., Szafron, D.: Using probabilistic knowledge and simulation to play poker. In: AAAI, pp. 697–703 (1999)Google Scholar
  16. 16.
    Koller, D., Pfeffer, A.: Representations and solutions for game-theoretic problems. Artificial Intelligence, 167–215 (1997)Google Scholar
  17. 17.
    Buro, M.: Solving the oshi-zumo game. Advances in Computer Games 10, 361–366 (2004)MathSciNetGoogle Scholar
  18. 18.
    Billings, D.: The first international RoShamBo programming competition. International Computer Games Association Journal 23(3-8), 42–50 (2000)Google Scholar
  19. 19.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice-Hall, Englewood Cliffs (2003)Google Scholar
  20. 20.
    Billings, D.: Vexbot wins poker tournament. International Computer Games Association Journal 26, 281 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Darse Billings
    • 1
  • Aaron Davidson
    • 1
  • Terence Schauenberg
    • 1
  • Neil Burch
    • 1
  • Michael Bowling
    • 1
  • Robert Holte
    • 1
  • Jonathan Schaeffer
    • 1
  • Duane Szafron
    • 1
  1. 1.Department of Computing ScienceUniversity of AlbertaEdmonton, AlbertaCanada

Personalised recommendations