Bandit-Based Genetic Programming

  • Jean-Baptiste Hoock
  • Olivier Teytaud
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6021)

Abstract

We consider the validation of randomly generated patterns in a Monte-Carlo Tree Search program. Our bandit-based genetic programming (BGP) algorithm, with proved mathematical properties, outperformed a highly optimized handcrafted module of a well-known computer-Go program with several world records in the game of Go.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Audibert, J.-Y., Munos, R., Szepesvari, C.: Use of variance estimation in the multi-armed bandit problem. In: NIPS 2006 Workshop on On-line Trading of Exploration and Exploitation (2006)Google Scholar
  2. 2.
    Audouard, P., Chaslot, G., Hoock, J.-B., Perez, J., Rimmel, A., Teytaud, O.: Grid coevolution for adaptive simulations; application to the building of opening books in the game of go. In: Proceedings of EvoGames (2009)Google Scholar
  3. 3.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite time analysis of the multiarmed bandit problem. Machine Learning 47(2/3), 235–256 (2002)MATHCrossRefGoogle Scholar
  4. 4.
    Auger, A., Teytaud, O.: Continuous lunches are free plus the design of optimal optimization algorithms. Algorithmica (2009)Google Scholar
  5. 5.
    Bruegmann, B.: Monte carlo go (1993)Google Scholar
  6. 6.
    Cazenave, T., Jouandeau, N.: On the parallelization of UCT. In: Proceedings of CGW 2007, pp. 93–101 (2007)Google Scholar
  7. 7.
    Chaslot, G., Saito, J.-T., Bouzy, B., Uiterwijk, J.W.H.M., van den Herik, H.J.: Monte-Carlo Strategies for Computer Go. In: Schobbens, P.-Y., Vanhoof, W., Schwanen, G. (eds.) Proceedings of the 18th BeNeLux Conference on Artificial Intelligence, Namur, Belgium, pp. 83–91 (2006)Google Scholar
  8. 8.
    Chaslot, G., Winands, M., Uiterwijk, J., van den Herik, H., Bouzy, B.: Progressive strategies for monte-carlo tree search. In: Wang, P., et al. (eds.) Proceedings of the 10th Joint Conference on Information Sciences (JCIS 2007), pp. 655–661. World Scientific Publishing Co. Pte. Ltd., Singapore (2007)CrossRefGoogle Scholar
  9. 9.
    Chaslot, G., Winands, M., van den Herik, H.: Parallel Monte-Carlo Tree Search. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) CG 2008. LNCS, vol. 5131, pp. 60–71. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Coulom, R.: Efficient selectivity and backup operators in monte-carlo tree search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M(J.) (eds.) CG 2006. LNCS, vol. 4630, pp. 72–83. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  11. 11.
    Coulom, R.: Computing elo ratings of move patterns in the game of go. In: Computer Games Workshop, Amsterdam, The Netherlands (2007)Google Scholar
  12. 12.
    Crasmaru, M.: On the complexity of Tsume-Go. In: van den Herik, H.J., Iida, H. (eds.) CG 1998. LNCS, vol. 1558, pp. 222–231. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  13. 13.
    Gelly, S., Hoock, J.B., Rimmel, A., Teytaud, O., Kalemkarian, Y.: The parallelization of monte-carlo planning. In: Proceedings of the International Conference on Informatics in Control, Automation and Robotics ICINCO 2008, pp. 198–203 (2008) (To appear)Google Scholar
  14. 14.
    Gelly, S., Silver, D.: Combining online and offline knowledge in UCT. In: ICML 2007: Proceedings of the 24th international conference on Machine learning, pp. 273–280. ACM Press, New York (2007)CrossRefGoogle Scholar
  15. 15.
    Holland, J.H.: Genetic algorithms and the optimal allocation of trials. SIAM J. Comput. 2(2), 88–105 (1973)MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Kato, H., Takeuchi, I.: Parallel monte-carlo tree search with simulation servers. In: 13th Game Programming Workshop (GPW 2008) (November 2008)Google Scholar
  17. 17.
    Kocsis, L., Szepesvari, C.: Bandit-based monte-carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  18. 18.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by means of Natural Evolution. MIT Press, Massachusetts (1992)Google Scholar
  19. 19.
    Lai, T., Robbins, H.: Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics 6, 4–22 (1985)MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Lee, C.-S., Wang, M.-H., Chaslot, G., Hoock, J.-B., Rimmel, A., Teytaud, O., Tsai, S.-R., Hsu, S.-C., Hong, T.-P.: The Computational Intelligence of MoGo Revealed in Taiwan’s Computer Go Tournaments. IEEE Transactions on Computational Intelligence and AI in games (2009)Google Scholar
  21. 21.
    Lichtenstein, D., Sipser, M.: Go is polynomial-space hard. J. ACM 27(2), 393–401 (1980)MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Mnih, V., Szepesvári, C., Audibert, J.-Y.: Empirical Bernstein stopping. In: ICML 2008: Proceedings of the 25th international conference on Machine learning, pp. 672–679. ACM, New York (2008)CrossRefGoogle Scholar
  23. 23.
    Robson, J.M.: The complexity of go. In: IFIP Congress, pp. 413–417 (1983)Google Scholar
  24. 24.
    Rolet, P., Sebag, M., Teytaud, O.: Optimal active learning through billiards and upper confidence trees in continous domains. In: Proceedings of the ECML conference (2009)Google Scholar
  25. 25.
    Wang, Y., Gelly, S.: Modifications of UCT and sequence-like simulations for Monte-Carlo Go. In: IEEE Symposium on Computational Intelligence and Games, Honolulu, Hawaii, pp. 175–182 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Jean-Baptiste Hoock
    • 1
  • Olivier Teytaud
    • 1
  1. 1.TAO (Inria), LRIUMR 8623(CNRS - Univ. Paris-Sud)OrsayFrance

Personalised recommendations