Improving Monte–Carlo Tree Search in Havannah

  • Richard J. Lorentz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6515)

Abstract

Havannah is a game played on an hexagonal board of hexagons where the base of the board typically ranges from four to ten hexagons. The game is known to be difficult to program. We study an MCTS-based approach to programming Havannah using our program named Wanderer. We experiment with five techniques of the basic MCTS algorithms and demonstrate that at normal time controls of approximately 30 seconds per move Wanderer can make quite strong moves with bases of size four or five, and play a reasonable game with bases of size six or seven. At longer time controls (ten minutes per move) Wanderer (1) appears to play nearly perfectly with base four, (2) is difficult for most humans to beat at base five, and (3) gives a good game at bases six and seven. Future research focuses on larger board sizes.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Akl, S.G., Newborn, M.M.: The Principal Continuation and the Killer Heuristic. In: 1977 ACM Annual Conference Proceedings, pp. 466–473. ACM Press, New York (1977)Google Scholar
  2. 2.
    Bruegmann, B.: Monte-Carlo go (1993) (unpublished manuscript) Google Scholar
  3. 3.
    Cazenave, T.: Iterative widening. In: 17th International Joint Conference on Artificial Intelligence (IJCAI 2001), pp. 523–528 (2001)Google Scholar
  4. 4.
    Chaslot, G.M.J.B., Winands, M.H.M., Uiterwijk, J.W.H.M., van den Herik, H.J., Bouzy, B.: Progressive strategies for Monte-Carlo tree search. In: Wang, P., et al. (eds.) Proceedings of the 10th Joint Conference on Information Sciences, pp. 655–661 (2007)Google Scholar
  5. 5.
    Coulom, R.: Efficient selectivity and backup operators in monte-carlo tree search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M(J.) (eds.) CG 2006. LNCS, vol. 4630, pp. 72–83. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  6. 6.
    Coulom, R.: Computing Elo Ratings of Move Patterns in the Game of Go. In: van den Herik, H.J., Uiterwijk, J.W.H.M., Winands, M., Schadd, M. (eds.) Computer Games Workshop, Amsterdam, The Netherlands, pp. 113–124 (2007)Google Scholar
  7. 7.
    de Koning, J.: Personal communicationGoogle Scholar
  8. 8.
    Drake, P., Pouliot, A., Schreiner, N., Vanberg, B.: The proximity heuristic and an opening book in Monte-Carlo Go (2007) (submitted)Google Scholar
  9. 9.
    Drake, P., Uurtamo, S.: Move Ordering vs Heavy Playouts: Where Should Heuristics Be Applied in Monte Carlo Go? In: Proceedings of the 3rd North American Game-On Conference (2007)Google Scholar
  10. 10.
    Gelly, S., Silver, D.: Combining online and offline knowledge in UCT. In: ICML 2007: Proceedings of the 24th International Conference on Machine Learning, pp. 273–280. ACM Press, New York (2007)Google Scholar
  11. 11.
    Gelly, S., Wang, Y.: Exploration exploitation in go: UCT for Monte-Carlo go. In: Twentieth Annual Conference on Neural Information Processing Systems (2006)Google Scholar
  12. 12.
    Gelly, S., Wang, Y., Munos, R., Teytaud, O.: Modification of UCT with patterns in Monte-Carlo Go. Technical Report 6062, INRIA, France (2006)Google Scholar
  13. 13.
    Hartmann, D.: On the importance of self-contained papers. Journal of the International Computer Games Association 30(4), 223–224 (2009)Google Scholar
  14. 14.
    Kocsis, L., Szepesvári, C.: Bandit based monte-carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. 15.
    Mnih, V., Szepesvári, C., Audibert, J.-Y.: Empirical Bernstein stopping. In: ICML 2008: Proceedings of the 25th International Conference on Machine Learning, pp. 672–679. ACM, New York (2008)CrossRefGoogle Scholar
  16. 16.
    Teytaud, F., Teytaud, O.: Creating an Upper-Confidence-Tree Program for Havannah. In: van den Herik, H.J., Spronck, P. (eds.) ACG 2009. LNCS, vol. 6048, pp. 65–74. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Winands, M.H.M., Björnsson, Y.: Evaluation Function Based Monte-Carlo LOA. In: van den Herik, H.J., Spronck, P. (eds.) ACG 2009. LNCS, vol. 6048, pp. 33–44. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  18. 18.
  19. 19.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Richard J. Lorentz
    • 1
  1. 1.Department of Computer ScienceCalifornia State UniversityNorthridgeUSA

Personalised recommendations