Creating Action Heuristics for General Game Playing Agents

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 614)


Monte-Carlo Tree Search (MCTS) is the most popular search algorithm used in General Game Playing (GGP) nowadays mainly because of its ability to perform well in the absence of domain knowledge. Several approaches have been proposed to add heuristics to MCTS in order to guide the simulations. In GGP those approaches typically learn heuristics at runtime from the results of the simulations. Because of peculiarities of GGP, it is preferable that these heuristics evaluate actions rather than game positions. We propose an approach that generates heuristics that estimate the usefulness of actions by analyzing the game rules as opposed to the simulation results. We present results of experiments that show the potential of our approach.


Goal Condition Heuristic Function Game State Game Tree General Game 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Chaslot, G.M.J.B., Winands, M.H.M., Uiterwijk, J.W.H.M., van den Herik, H.J., Bouzy, B.: Progressive strategies for Monte-Carlo tree search. In: Proceedings of the 10th Joint Conference on Information Sciences (JCIS 2007), pp. 655–661. World Scientific Publishing Co. Pte. Ltd (2007)Google Scholar
  2. 2.
    Clune, J.: Heuristic evaluation functions for general game playing. In: AAAI 2007, pp. 1134–1139. AAAI Press (2007)Google Scholar
  3. 3.
    Coulom, R.: Efficient selectivity and backup operators in monte-carlo tree search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M.J. (eds.) CG 2006. LNCS, vol. 4630, pp. 72–83. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Finnsson, H.: Generalized Monte-Carlo tree search extensions for general game playing. In: Twenty-Sixth AAAI Conference on Artificial Intelligence (2012)Google Scholar
  5. 5.
    Finnsson, H., Björnsson, Y.: Simulation-based approach to general game playing. In: AAAI 2008, pp. 259–264. AAAI Press (2008)Google Scholar
  6. 6.
    Finnsson, H., Björnsson, Y.: Learning simulation control in general game playing agents. In: AAAI 2010, pp. 954–959. AAAI Press (2010)Google Scholar
  7. 7.
    Finnsson, H., Björnsson, Y.: Cadiaplayer: search-control techniques. KI 25(1), 9–16 (2011)Google Scholar
  8. 8.
    Gelly, S., Silver, D.: Combining online and offline knowledge in UCT. In: Proceedings of the 24th International Conference on Machine Learning. ACM International Conference Proceeding Series, vol. 227, pp. 273–280 (2007)Google Scholar
  9. 9.
    Genesereth, M.R., Love, N., Pell, B.: General game playing: overview of the AAAI competition. AI Mag. 26(2), 62–72 (2005)Google Scholar
  10. 10.
    Haufe, S., Schiffel, S., Thielscher, M.: Automated verification of state sequence invariants in general game playing. Artif. Intell. 187–188, 1–30 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Kocsis, L., Szepesvári, C.: Bandit based monte-carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Kuhlmann, G., Dresner, K., Stone, P.: Automatic heuristic construction in a complete general game player. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence, pp. 1457–1462 (2006)Google Scholar
  13. 13.
    Lanctot, M., Winands, M.H.M., Pepels, T., Sturtevant, N.R.: Monte Carlo tree search with heuristic evaluations using implicit minimax backups. In: 2014 IEEE Conference on Computational Intelligence and Games (CIG 2014), pp. 341–348. IEEE (2014)Google Scholar
  14. 14.
    Love, N., Hinrichs, T., Haley, D., Schkufza, E., Genesereth, M.: General game playing: Game description language specification. Technical report, Stanford University.
  15. 15.
    Michulke, D., Schiffel, S.: Admissible distance heuristics for general games. In: Filipe, J., Fred, A. (eds.) ICAART 2012. CCIS, vol. 358, pp. 188–203. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  16. 16.
    Pepels, T., Tak, M.J.W., Lanctot, M., Winands, M.H.M.: Quality-based rewards for Monte-Carlo tree search simulations. In: 21st European Conference on Artificial Intelligence (ECAI 2014), pp. 705–710. IOS Press (2014)Google Scholar
  17. 17.
    Reiter, R.: Knowledge in Action: Logical Foundations for Specifying and Implementing Dynamical Systems, pp. 61–73. Massachusetts Institute of Technology, Cambridge (2001)Google Scholar
  18. 18.
    Schiffel, S., Björnsson, Y.: Efficiency of GDL reasoners. IEEE Trans. Comput. Intell. AI Games 6(4), 343–354 (2014)CrossRefGoogle Scholar
  19. 19.
    Schiffel, S., Thielscher, M.: Fluxplayer: a successful general game player. In: Proceedings of the 22nd AAAI Conference on Artificial Intelligence (AAAI-07), pp. 1191–1196. AAAI Press (2007)Google Scholar
  20. 20.
    Sharma, S., Kobti, Z., Goodwin, S.D.: Knowledge generation for improving simulations in UCT for general game playing. In: Wobcke, W., Zhang, M. (eds.) AI 2008. LNCS (LNAI), vol. 5360, pp. 49–55. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  21. 21.
    Tak, M.J.W., Winands, M.H.M., Björnsson, Y.: N-grams and the last-good-reply policy applied in general game playing. IEEE Trans. Comput. Intell. AI in Games 4(2), 73–83 (2012)CrossRefGoogle Scholar
  22. 22.
    Waledzik, K., Mandziuk, J.: An automatically generated evaluation function in general game playing. IEEE Trans. Comput. Intell. AI Games 6(3), 258–270 (2014)CrossRefGoogle Scholar
  23. 23.
    Winands, M.H.M., Björnsson, Y., Saito, J.T.: Monte-Carlo tree search in lines of action. IEEE Trans. Comput. Intell. AI Games 2(4), 239–250 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Faculty of Information TechnologyBrno University of TechnologyBrnoCzech Republic
  2. 2.School of Computer ScienceReykjavík UniversityReykjavíkIceland

Personalised recommendations