Monte-Carlo Tree Search in Poker Using Expected Reward Distributions
We investigate the use of Monte-Carlo Tree Search (MCTS) within the field of computer Poker, more specifically No-Limit Texas Hold’em. The hidden information in Poker results in so called miximax game trees where opponent decision nodes have to be modeled as chance nodes. The probability distribution in these nodes is modeled by an opponent model that predicts the actions of the opponents. We propose a modification of the standard MCTS selection and backpropagation strategies that explicitly model and exploit the uncertainty of sampled expected values. The new strategies are evaluated as a part of a complete Poker bot that is, to the best of our knowledge, the first exploiting no-limit Texas Hold’em bot that can play at a reasonable level in games of more than two players.
Unable to display preview. Download preview PDF.
- 1.Gilpin, A., Sandholm, T., Sørensen, T.: A heads-up no-limit Texas Hold’em poker player: discretized betting models and automatically generated equilibrium-finding programs. In: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems, International Foundation for Autonomous Agents and Multiagent Systems Richland, SC, vol. 2, pp. 911–918 (2008)Google Scholar
- 2.Billings, D.: Algorithms and assessment in computer poker. PhD thesis, Edmonton, Alta, Canada (2006)Google Scholar
- 4.Russell, S., Norvig, P.: Artificial intelligence: A modern approach. Prentice Hall, Englewood Cliffs (2003)Google Scholar
- 5.Suffecool, K.: Cactus kev’s poker hand evaluator (July 2007), http://www.suffecool.net/poker/evaluator.html
- 7.Wang, Y., Witten, I.: Induction of model trees for predicting continuous classes (1996)Google Scholar
- 9.Gelly, S., Wang, Y.: Exploration exploitation in go: UCT for Monte-Carlo go. In: Twentieth Annual Conference on Neural Information Processing Systems, NIPS 2006 (2006)Google Scholar
- 11.Van Lishout, F., Chaslot, G., Uiterwijk, J.: Monte-Carlo Tree Search in Backgammon. In: Computer Games Workshop, pp. 175–184 (2007)Google Scholar