Living Reference Work Entry

Handbook of Digital Games and Entertainment Technologies

pp 1-30

Date: Latest Version

Monte-Carlo Tree Search in Board Games

  • Mark H. M. WinandsAffiliated withDepartment of Data Science and Knowledge Engineering, Maastricht University Email author 

Abstract

Monte-Carlo Tree Search (MCTS) is a best-first search method guided by the results of Monte-Carlo simulations. It is based on randomized exploration of the search space. Using the results of previous explorations, the method gradually builds up a game tree in memory and successively becomes better at accurately estimating the values of the most promising moves. MCTS has substantially advanced the state of the art in board games such as Go, Amazons, Hex, Chinese Checkers, Kriegspiel, and Lines of Action.

This chapter gives an overview of popular and effective enhancements for board game playing MCTS agents. First, it starts by describing the structure of MCTS and giving pseudocode. It also addresses how to adjust MCTS to prove the game-theoretic value of a board position. Next, popular enhancements such as RAVE, progressive bias, progressive widening, and prior knowledge, which improve the simulation in the tree part of MCTS, are discussed in detail. Subsequently, enhancements such as MAST, N-Grams, and evaluation function-based strategies are explained for improving the simulation outside the tree. As modern computers have nowadays multiple cores, this chapter mentions techniques to parallelize MCTS in a straightforward but effective way. Finally, approaches to deal with imperfect information and stochasticity in an MCTS context are discussed as well.

Keywords

Minimax search techniques Monte-Carlo Tree Search (MCTS) MCTS MCTS-solver Tree and Root Parellization Chance nodes Multi-Armed Bandit (MAB) problem Playout Simulation strategy Expansion strategy Backpropagation Final move selection strategies Domain-independent Rapid Action-Value Estimator (RAVE) Progressive Bias (PB) Implicit Minimax Progressive Widening Move-Average Sampling Technique (MAST) N-Gram Selection Technique (NST) Greedy strategy Root parallelization Tree parallelization Determinization Upper Confidence Bounds applied to Trees (UCT)