Advertisement

Sequential Halving for Partially Observable Games

Conference paper
  • 429 Downloads
Part of the Communications in Computer and Information Science book series (CCIS, volume 614)

Abstract

This paper investigates Sequential Halving as a selection policy in the following four partially observable games: Go Fish, Lost Cities, Phantom Domineering, and Phantom Go. Additionally, H-MCTS is studied, which uses Sequential Halving at the root of the search tree, and UCB elsewhere. Experimental results reveal that H-MCTS performs the best in Go Fish, whereas its performance is on par in Lost Cities and Phantom Domineering. Sequential Halving as a flat Monte-Carlo Search appears to be the stronger technique in Phantom Go.

Keywords

Game State Selection Policy Card Game Lost City Test Domain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Arneson, B., Hayward, R., Henderson, P.: Monte-Carlo tree search in Hex. IEEE Trans. Comput. Intell. AI Games 2(4), 251–258 (2010)CrossRefGoogle Scholar
  2. 2.
    Audibert, J., Bubeck, S., Munos, R.: Best arm identification in multi-armed bandits. In: Proceedings of the 23rd Conference on Learning Theory, pp. 41–53 (2010)Google Scholar
  3. 3.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)CrossRefzbMATHGoogle Scholar
  4. 4.
    Balla, R.K., Fern, A.: UCT for tactical assault planning in real-time strategy games. In: Boutilier, C. (ed.) Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI), pp. 40–45 (2009)Google Scholar
  5. 5.
    Bouzy, B., Helmstetter, B.: Monte-Carlo Go developments. In: van den Herik, H.J., Iida, H., Heinz, E.A. (eds.) Advances in Computer Games. IFIP, vol. 135, pp. 159–174. Springer, New York (2004)CrossRefGoogle Scholar
  6. 6.
    Browne, C., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P.I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., Colton, S.: A survey of Monte-Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)CrossRefGoogle Scholar
  7. 7.
    Bubeck, S., Munos, R., Stoltz, G.: Pure exploration in finitely-armed and continuous-armed bandits. Theor. Comput. Sci. 412(19), 1832–1852 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Buro, M., Long, J., Furtak, T., Sturtevant, N.: Improving state evaluation, inference, and search in trick-based card games. In: Boutilier, C. (ed.) Proceedings of the 21st International Joint Conference on Artificial Intelligence, IJCAI 2009, Pasadena, CA, USA, pp. 1407–1413 (2009)Google Scholar
  9. 9.
    Cazenave, T.: A phantom-go program. In: van den Herik, H.J., Hsu, S.-C., Hsu, T., Donkers, H.H.L.M.J. (eds.) CG 2005. LNCS, vol. 4250, pp. 120–125. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Cazenave, T.: Sequential halving applied to trees. IEEE Trans. Comput. Intell. AI Games 7(1), 102–105 (2015)CrossRefGoogle Scholar
  11. 11.
    Ciancarini, P., Favini, G.: Monte Carlo tree search in Kriegspiel. AI J. 174(11), 670–6684 (2010)MathSciNetGoogle Scholar
  12. 12.
    Coulom, R.: Efficient selectivity and backup operators in Monte-Carlo tree search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M.J. (eds.) CG 2006. LNCS, vol. 4630, pp. 72–83. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  13. 13.
    Cowling, P., Powley, E., Whitehouse, D.: Information set Monte Carlo tree search. IEEE Trans. Comput. Intell. AI Games 4(2), 120–143 (2012)CrossRefGoogle Scholar
  14. 14.
    Feldman, Z., Domshlak, C.: Simple regret optimization in online planning for Markov decision processes. J. Artif. Intell. Res. (JAIR) 51, 165–205 (2014)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Ginsberg, M.: Gib: Steps toward an expert-level bridge-playing program. In: Dean, T. (ed.) Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI 1999), vol. 1, pp. 584–589. Morgan Kaufmann (1999)Google Scholar
  16. 16.
    Karnin, Z., Koren, T., Somekh, O.: Almost optimal exploration in multi-armed bandits. In: Proceedings of the International Conference on Machine Learning, pp. 1238–1246 (2013)Google Scholar
  17. 17.
    Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  18. 18.
    Nijssen, J.A.M., Winands, M.H.M.: Monte-Carlo tree search for the hide-and-seek game Scotland Yard. Trans. Comput. Intell. AI Games 4(4), 282–294 (2012)CrossRefGoogle Scholar
  19. 19.
    Pepels, T., Winands, M.H.M., Lanctot, M.: Real-time Monte-Carlo tree search in Ms Pac-Man. IEEE Trans. Comp. Intell. AI Games 6(3), 245–257 (2014)CrossRefGoogle Scholar
  20. 20.
    Pepels, T., Cazenave, T., Winands, M.H.M., Lanctot, M.: Minimizing simple and cumulative regret in Monte-Carlo tree search. In: Cazenave, T., Winands, M.H.M., Björnsson, Y. (eds.) CGW 2014. CCIS, vol. 504, pp. 1–15. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  21. 21.
    Powley, E.J., Whitehouse, D., Cowling, P.I.: Monte Carlo tree search with macro-actions and heuristic route planning for the physical travelling salesman problem. In: IEEE Conference on Computational Intelligence and Games, pp. 234–241. IEEE (2012)Google Scholar
  22. 22.
    Rimmel, A., Teytaud, O., Lee, C., Yen, S., Wang, M., Tsai, S.: Current frontiers in computer Go. IEEE Trans. Comput. Intell. AI Games 2(4), 229–238 (2010)CrossRefGoogle Scholar
  23. 23.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 3rd edn. Prentice-Hall Inc., Upper Saddle River (2010)zbMATHGoogle Scholar
  24. 24.
    Sheppard, B.: World-championship-caliber Scrabble. Artif. Intell. 134(1–2), 241–275 (2002)CrossRefzbMATHGoogle Scholar
  25. 25.
    Tolpin, D., Shimony, S.: MCTS based on simple regret. In: Proceedings of the Association for the Advancement Artificial Intelligence, pp. 570–576 (2012)Google Scholar
  26. 26.
    Winands, M.H.M., Björnsson, Y., Saito, J.T.: Monte Carlo tree search in lines of action. IEEE Trans. Comp. Intell. AI Games 2(4), 239–250 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Data Science and Knowledge EngineeringMaastricht UniversityMaastrichtThe Netherlands
  2. 2.LAMSADE - Université Paris-DauphineParisFrance

Personalised recommendations