Encyclopedia of Computer Graphics and Games

Living Edition
| Editors: Newton Lee

Monte-Carlo Tree Search

  • Mark H. M. Winands
Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-08234-9_12-1

Synonyms

Definition

Monte-Carlo Tree Search (MCTS) (Coulom 2007; Kocsis et al. 2006) is a best-first search method that does not require a positional evaluation function. It is based on a randomized exploration of the search space. Using the results of previous explorations, the algorithm gradually builds up a game tree in memory and successively becomes better at accurately estimating the values of the most promising moves. MCTS consists of four strategic steps, repeated as long as there is time left (Chaslot et al. 2008b). The steps, outlined in Fig. 1, are as follows:
This is a preview of subscription content, log in to check access.

References and Further Reading

  1. Abramson, B.: Expected-outcome: A general model of static evaluation. IEEE Trans. Pattern Anal. Mach. Intell. 12(2), 182–193 (1990)CrossRefGoogle Scholar
  2. Arneson, B., Hayward, R.B., Henderson, P.: Monte Carlo Tree Search in Hex. IEEE Trans. Comput. Intell. AI Games 2(4), 251–258 (2010)CrossRefGoogle Scholar
  3. Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)MATHCrossRefGoogle Scholar
  4. Balla, R.K., Fern A.: UCT for tactical assault planning in real-time strategy games. In: Boutilier, C. (ed.) Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence (IJCAI-09), pp. 40–45. AAAI Press, Menlo Park, CA, USA (2009)Google Scholar
  5. Billings, D., Peña, L., Schaeffer, J., Szafron, D.: Using probabilistic knowledge and simulation to play poker. In: Hendler, J., Subramanian, D. (eds) Proceedings of the Sixteenth National Conference on Artificial Intelligence and Eleventh Conference on Innovative Applications of Artificial Intelligence, pp. 697–703. AAAI Press/The MIT Press, Menlo Park, CA, USA (1999)Google Scholar
  6. Björnsson, Y., Finnsson, H.: CadiaPlayer: A simulation-based General Game Player. IEEE Trans. Comput. Intell. AI Games 1(l), 4–15 (2009)CrossRefGoogle Scholar
  7. Bouzy, B., Helmstetter, B.: Monte-Carlo Go developments. In: van den Herik, H.J., Iida, H., Heinz, E.A. (eds.) Advances in Computer Games 10: Many Games, Many Challenges. IFIP Advances in Information and Communication Technology, vol. 135, pp. 159–174. Kluwer, Boston (2004)CrossRefGoogle Scholar
  8. Browne, C.B., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P.I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., Colton, S.: A survey of Monte Carlo Tree Search methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)CrossRefGoogle Scholar
  9. Cazenave, T., Saffidine, A.: Score bounded Monte-Carlo Tree Search. In: van den Herik, H.J., Iida, H., Plaat, A. (eds.) Computers and Games (CG 2010). Lecture Notes in Computer Science, vol. 6515, pp. 93–104. Springer, Berlin (2011)CrossRefGoogle Scholar
  10. Chaslot, G.M.J.-B., Winands, M.H.M., van den Herik, H.J.: Parallel Monte-Carlo Tree Search. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) Computers and Games (CG 2008). Lecture Notes in Computer Science, vol. 5131, pp. 60–71. Springer, Berlin (2008a)CrossRefGoogle Scholar
  11. Chaslot, G.M.J.-B., Winands, M.H.M., van den Herik, H.J., Uiterwijk, J.W.H.M., Bouzy, B.: Progressive strategies for Monte-Carlo Tree Search. New Math. Nat. Comput. 4(3), 343–357 (2008b)MATHMathSciNetCrossRefGoogle Scholar
  12. Childs, B.E., Brodeur, J.H., Kocsis, L.: Transpositions and move groups in Monte Carlo Tree Search. In: Hingston, P., Barone, L. (eds.) Proceedings of the 2008 IEEE Symposium on Computational Intelligence and Games, pp. 389–395. IEEE, Piscataway, NJ, USA (2008)Google Scholar
  13. Ciancarini, P., Favini, G.P.: Monte Carlo Tree Search in Kriegspiel. AI J. 174(11), 670–684 (2010)MathSciNetGoogle Scholar
  14. Coulom, R.: Efficient selectivity and backup operators in Monte-Carlo Tree Search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M. (eds.) Computers and Games (CG 2006). Lecture Notes in Computer Science, vol. 4630, pp. 72–83. Springer, Berlin (2007)CrossRefGoogle Scholar
  15. Cowling, P.I., Powley, E.J., Whitehouse, D.: Information set Monte Carlo Tree Search. IEEE Trans. Comput. Intell. AI Games 4(2), 120–143 (2012)CrossRefGoogle Scholar
  16. Enzenberger, M., Müller, M.: A lock-free multithreaded Monte-Carlo Tree Search algorithm. In: van den Herik, H.J., Spronck, P. (eds.) Advances in Computer Games (ACG 2009). Lecture Notes in Computer Science (LNCS), vol. 6048, pp. 14–20. Springer, Berlin (2010)CrossRefGoogle Scholar
  17. Enzenberger, M., Müller, M., Arneson, B., Segal, R.: Fuego – an open-source framework for board games and Go engine based on Monte Carlo Tree Search. IEEE Trans. Comput. Intell AI Games 2(4), 259–270 (2010)CrossRefGoogle Scholar
  18. Gelly, S., Kocsis, L., Schoenauer, M., Sebag, M., Silver, D., Szepesvári, C., Teytaud, O.: The grand challenge of computer Go: Monte Carlo Tree Search and extensions. Commun. ACM 55(3), 106–113 (2012)CrossRefGoogle Scholar
  19. Ginsberg, M.L.: GIB: Steps toward an expert-level bridge-playing program. In: Dean, T. (ed.) Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI-99), vol. 1, pp. 584–589. Morgan Kaufmann, San Francisco, CA, USA (1999)Google Scholar
  20. Hennes, D., Izzo, D.: Interplanetary trajectory planning with Monte Carlo Tree Search. In: Yang, Q., Wooldridge, M. (eds.) Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015), pp. 769–775. AAAI Press, Menlo Park, CA, USA (2015)Google Scholar
  21. Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo Planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) Machine Learning: ECML 2006. Lecture Notes in Artificial Intelligence, vol. 4212, pp. 282–293. Springer, Berlin (2006)Google Scholar
  22. Lorentz, R.J.: Amazons discover Monte-Carlo. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) Computers and Games (CG 2008). Lecture Notes in Computer Science, vol. 5131, pp. 13–24. Springer, Berlin (2008)CrossRefGoogle Scholar
  23. Nguyen, K.Q., Thawonmas, R.: Monte Carlo Tree Search for collaboration control of Ghosts in Ms. Pac-Man. IEEE Trans. Comput. Intell. AI Games 5(1), 57–68 (2013)CrossRefGoogle Scholar
  24. Nijssen, J.A.M., Winands, M.H.M.: Enhancements for multi-player Monte-Carlo Tree Search. In: van den Herik, H.J., Iida, H., Plaat, A. (eds.) Computers and Games (CG 2010). Lecture Notes in Computer Science, vol. 6151, pp. 238–249. Springer, Berlin (2011)CrossRefGoogle Scholar
  25. Nijssen, J.A.M., Winands, M.H.M.: Monte Carlo Tree Search for the hide-and-seek game Scotland Yard. Trans. Comput. Intell. AI Games 4(4), 282–294 (2012)CrossRefGoogle Scholar
  26. Pepels, T., Winands, M.H.M., Lanctot, M.: Real-time Monte Carlo Tree Search in Ms Pac-Man. IEEE Trans. Comput. Intell. AI Games 6(3), 245–257 (2014)CrossRefGoogle Scholar
  27. Perez, D., Samothrakis, S., Lucas, S.M.: Knowledge-based fast evolutionary MCTS for general video game playing. In: Proceedings of the IEEE Conference on Computational Intelligence and Games (CIG 2014), pp. 68–75 (2014)Google Scholar
  28. Ruijl, B., Vermaseren, J., Plaat, A. van den Herik, H.J.: Combining simulated annealing and Monte Carlo Tree Search for expression simplification. In: ICAART 2014, pp. 724–731 (2014)Google Scholar
  29. Schadd, M.P.D., Winands, M.H.M., Tak, M.J.W., Uiterwijk, J.W.H.M.: Single-player Monte-Carlo Tree Search for SameGame. Knowl.-Based Syst. 34, 3–11 (2012)CrossRefGoogle Scholar
  30. Sheppard, B.: World-championship-caliber Scrabble. Artif. Intell. 134(1–2), 241–275 (2002)MATHCrossRefGoogle Scholar
  31. Sturtevant, N.R.: An analysis of UCT in multi-player games. ICGA J. 31(4), 195–208 (2008)Google Scholar
  32. Tak, M.J.W., Winands, M.H.M., Björnsson, Y.: N-Grams and the last-good-reply policy applied in general game playing. IEEE Trans. Comput. Intell. AI Games 4(2), 73–83 (2012)CrossRefGoogle Scholar
  33. Tesauro, G., Galperin, G.R.: On-line policy improvement using Monte-Carlo search. In: Mozer, M.C., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, pp. 1068–1074. MIT Press, Cambridge, MA, USA (1997)Google Scholar
  34. Winands, M.H.M., Björnsson, Y., Saito, J.-T.: Monte Carlo Tree Search in Lines of Action. IEEE Trans. Comput. Intell. AI Games 2(4), 239–250 (2010)CrossRefGoogle Scholar
  35. Zhu, G., Lizotte, D., Hoey, J.: Scalable approximate policies for Markov decision process models of hospital elective admissions. Artif. Intell. Med. 61(1), 21–34 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Data Science and Knowledge EngineeringMaastricht UniversityMaastrichtThe Netherlands