Score Bounded Monte-Carlo Tree Search

  • Tristan Cazenave
  • Abdallah Saffidine
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6515)

Abstract

Monte-Carlo Tree Search (MCTS) is a successful algorithm used in many state of the art game engines. We propose to improve a MCTS solver when a game has more than two outcomes. It is for example the case in games that can end in draw positions. In this case it improves significantly a MCTS solver to take into account bounds on the possible scores of a node in order to select the nodes to explore. We apply our algorithm to solving Seki in the game of Go and to Connect Four.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Victor Allis, L.: A knowledge-based approach of connect-four the game is solved: White wins. Masters thesis, Vrije Universitat Amsterdam, Amsterdam, The Netherlands (October 1988)Google Scholar
  2. 2.
    Berliner, H.J.: The B\(^{\mbox{*}}\) tree search algorithm: A best-first proof procedure. Artif. Intell. 12(1), 23–40 (1979)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Cazenave, T.: A phantom-go program. In: van den Herik, H.J., Hsu, S.-C., Hsu, T.-s., Donkers, H.H.L.M(J.) (eds.) CG 2005. LNCS, vol. 4250, pp. 120–125. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  4. 4.
    Cazenave, T.: Reflexive monte-carlo search. In: Computer Games Workshop, Amsterdam, The Netherlands, pp. 165–173 (2007)Google Scholar
  5. 5.
    Cazenave, T.: Nested monte-carlo search. In: IJCAI, pp. 456–461 (2009)Google Scholar
  6. 6.
    Chaslot, G., Chatriot, L., Fiter, C., Gelly, S., Hoock, J.-B., Perez, J., Rimmel, A., Teytaud, O.: Combiner connaissances expertes, hors-ligne, transientes et en ligne pour l’exploration Monte-Carlo. Apprentissage et MC. Revue d’Intelligence Artificielle 23(2-3), 203–220 (2009)CrossRefGoogle Scholar
  7. 7.
    Coulom, R.: Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M(J.) (eds.) CG 2006. LNCS, vol. 4630, pp. 72–83. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Coulom, R.: Computing Elo ratings of move patterns in the game of Go. ICGA Journal 30(4), 198–208 (2007)Google Scholar
  9. 9.
    Finnsson, H., Björnsson, Y.: Simulation-based approach to general game playing. In: AAAI, pp. 259–264 (2008)Google Scholar
  10. 10.
    Gelly, S., Silver, D.: Combining online and offline knowledge in UCT. In: ICML, pp. 273–280 (2007)Google Scholar
  11. 11.
    Gelly, S., Silver, D.: Achieving master level play in 9 x 9 computer go. In: AAAI, pp. 1537–1540 (2008)Google Scholar
  12. 12.
    Hart, P., Nilsson, N., Raphael, B.: A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Syst. Sci. Cybernet. 4(2), 100–107 (1968)CrossRefGoogle Scholar
  13. 13.
    Kocsis, L., Szepesvári, C.: Bandit based monte-carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    Kozelek, T.: Methods of MCTS and the game Arimaa. Master’s thesis, Charles University in Prague (2009)Google Scholar
  15. 15.
    Lorentz, R.J.: Amazons discover monte-carlo. In: Computers and Games, pp. 13–24 (2008)Google Scholar
  16. 16.
    Niu, X., Kishimoto, A., Müller, M.: Recognizing seki in computer go. In: ACG, pp. 88–103 (2006)Google Scholar
  17. 17.
    Schadd, M.P.D., Winands, M.H.M., van den Herik, H.J., Chaslot, G.M.J.-B., Uiterwijk, J.W.H.M.: Single-player monte-carlo tree search. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) CG 2008. LNCS, vol. 5131, pp. 1–12. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  18. 18.
    Winands, M.H.M., Björnsson, Y.: Evaluation function based Monte-Carlo LOA. In: Advances in Computer Games (2009)Google Scholar
  19. 19.
    Winands, M.H.M., Björnsson, Y., Saito, J.-T.: Monte-carlo tree search solver. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) CG 2008. LNCS, vol. 5131, pp. 25–36. Springer, Heidelberg (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Tristan Cazenave
    • 1
  • Abdallah Saffidine
    • 1
  1. 1.LAMSADEUniversité Paris-DauphineParisFrance

Personalised recommendations