Creating an Upper-Confidence-Tree Program for Havannah

  • Fabien Teytaud
  • Olivier Teytaud
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6048)


Monte-Carlo Tree Search and Upper Confidence Bounds provided huge improvements in computer-Go. In this paper, we test the generality of the approach by experimenting on the game, Havannah, which is known for being especially difficult for computers. We show that the same results hold, with slight differences related to the absence of clearly known patterns for the game of Havannah, in spite of the fact that Havannah is more related to connection games like Hex than to territory games like Go.


Legal Move Bandit Problem Professional Player Exploration Term Multiarmed Bandit 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wikipedia, Havannah (2009)Google Scholar
  2. 2.
    Schmittberger, R.W.: New Rules for Classic Games. Wiley, Chichester (1992)Google Scholar
  3. 3.
    Chaslot, G., Saito, J.T., Bouzy, B., Uiterwijk, J.W.H.M., van den Herik, H.J.: Monte-Carlo Strategies for Computer Go. In: Schobbens, P.Y., Vanhoof, W., Schwanen, G. (eds.) Proceedings of the 18th BeNeLux Conference on Artificial Intelligence, Namur, Belgium, pp. 83–91 (2006)Google Scholar
  4. 4.
    Coulom, R.: Efficient selectivity and backup operators in monte-carlo tree search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M(J.) (eds.) CG 2006. LNCS, vol. 4630, pp. 72–83. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  5. 5.
    Kocsis, L., Szepesvari, C.: Bandit-based monte-carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Wang, Y., Gelly, S.: Modifications of UCT and sequence-like simulations for Monte-Carlo Go. In: IEEE Symposium on Computational Intelligence and Games, Honolulu, Hawaii, pp. 175–182 (2007)Google Scholar
  7. 7.
    Bruegmann, B.: Monte carlo go (1993) (Unpublished)Google Scholar
  8. 8.
    Gelly, S., Silver, D.: Combining online and offline knowledge in uct. In: ICML 2007: Proceedings of the 24th international conference on Machine learning, New York, NY, USA, pp. 273–280. ACM Press, New York (2007)CrossRefGoogle Scholar
  9. 9.
    Coulom, R.: Computing elo ratings of move patterns in the game of go. In: Computer Games Workshop, Amsterdam, The Netherlands (2007)Google Scholar
  10. 10.
    Chaslot, G., Winands, M., Uiterwijk, J., van den Herik, H., Bouzy, B.: Progressive strategies for monte-carlo tree search. In: Wang, P. (ed.) Proceedings of the 10th Joint Conference on Information Sciences (JCIS 2007), pp. 655–661. World Scientific Publishing Co. Pte. Ltd., Singapore (2007)CrossRefGoogle Scholar
  11. 11.
    Lee, C.S., Wang, M.H., Chaslot, G., Hoock, J.B., Rimmel, A., Teytaud, O., Tsai, S.R., Hsu, S.C., Hong, T.P.: The computational intelligence of mogo revealed in taiwan’s computer go tournaments. IEEE Transactions on Computational Intelligence and AI in Games (2009) (accepted)Google Scholar
  12. 12.
    Gelly, S., Hoock, J.B., Rimmel, A., Teytaud, O., Kalemkarian, Y.: The parallelization of monte-carlo planning. In: Proceedings of the International Conference on Informatics in Control, Automation and Robotics (ICINCO 2008), pp. 198–203 (2008) (to appear)Google Scholar
  13. 13.
    Chaslot, G., Winands, M., van den Herik, H.: Parallel Monte-Carlo Tree Search. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) CG 2008. LNCS, vol. 5131. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Cazenave, T., Jouandeau, N.: On the parallelization of UCT. In: Proceedings of CGW 2007, pp. 93–101 (2007)Google Scholar
  15. 15.
    Kato, H., Takeuchi, I.: Parallel monte-carlo tree search with simulation servers. In: 13th Game Programming Workshop, GPW 2008 (November 2008)Google Scholar
  16. 16.
    Audouard, P., Chaslot, G., Hoock, J.B., Perez, J., Rimmel, A., Teytaud, O.: Grid coevolution for adaptive simulations; application to the building of opening books in the game of go. In: Proceedings of EvoGames (2009)Google Scholar
  17. 17.
    Lai, T., Robbins, H.: Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics 6, 4–22 (1985)zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite time analysis of the multiarmed bandit problem. Machine Learning 47(2/3), 235–256 (2002)zbMATHCrossRefGoogle Scholar
  19. 19.
    Audibert, J.Y., Munos, R., Szepesvari, C.: Use of variance estimation in the multi-armed bandit problem. In: NIPS 2006 Workshop on On-line Trading of Exploration and Exploitation (2006)Google Scholar
  20. 20.
    Mnih, V., Szepesvári, C., Audibert, J.Y.: Empirical Bernstein stopping. In: ICML 2008: Proceedings of the 25th international conference on Machine learning, New York, NY, USA, pp. 672–679. ACM, New York (2008)CrossRefGoogle Scholar
  21. 21.
    Wang, Y., Audibert, J.Y., Munos, R.: Algorithms for infinitely many-armed bandits. In: Advances in Neural Information Processing Systems., vol. 21 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Fabien Teytaud
    • 1
  • Olivier Teytaud
    • 1
  1. 1.TAO (Inria), LRI, UMR 8623(CNRS - Univ. Paris-Sud)OrsayFrance

Personalised recommendations