Efficient Heuristic Policy Optimisation for a Challenging Strategic Card Game

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12104)


Turn-based multi-action adversarial games are challenging scenarios in which each player turn consists of a sequence of atomic actions. The order in which an AI agent runs these atomic actions may hugely impact the outcome of the turn. One of the main challenges of game artificial intelligence is to design a heuristic function to help agents to select the optimal turn to play, given a particular state of the game. In this paper, we report results using the recently developed N-Tuple Bandit Evolutionary Algorithm to tune the heuristic function parameters. For evaluation, we measure how the tuned heuristic function affects the performance of the state-of-the-art evolutionary algorithm Online Evolution Planning. The multi-action adversarial strategy card game Legends of Code and Magic was used as a testbed. Results indicate that the N-Tuple Bandit Evolutionary Algorithm can effectively tune the heuristic function parameters to improve the performance of the agent.


Game artificial intelligence Board and card games solving Learning in games Multi-action games Heuristic policy optimisation 



The authors want to thank J. Kowalski for his help solving doubts about the LOCM rules and N. Justensen and H. Baier for their help in better understanding their algorithms.


  1. 1.
    Baier, H., Cowling, P.I.: Evolutionary MCTS for multi-action adversarial games. In: Preceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018 (2018).
  2. 2.
    Bursztein, E.: I am a legend: hacking hearthstone using statistical learning methods. In: 2016 IEEE Conference on Computational Intelligence and Games (CIG) (2016).
  3. 3.
    Choe, J.S.B., Kim, J.K.: Enhancing Monte Carlo tree search for playing hearthstone. In: Proceedings of the 1st Conference on Games, GOG 2019 (2019)Google Scholar
  4. 4.
    Cowling, P., Powley, E., Whitehouse, D.: Information set Monte Carlo tree search. IEEE Trans. Comput. Intell. AI Games 4, 120–143 (2012). Scholar
  5. 5.
    Dockhorn, A., Frick, M., Akkaya, Ü., Kruse, R.: Predicting opponent moves for improving hearthstone AI. In: Medina, J., et al. (eds.) IPMU 2018. CCIS, vol. 854, pp. 621–632. Springer, Cham (2018). Scholar
  6. 6.
    Justesen, N., Mahlmann, T., Togelius, J.: Online evolution for multi-action adversarial games. In: Squillero, G., Burelli, P. (eds.) EvoApplications 2016. LNCS, vol. 9597, pp. 590–603. Springer, Cham (2016). Scholar
  7. 7.
    Kowalski, J., Miernik, R.: Legends of code and magic (2019). Accessed 26 May 2019
  8. 8.
    Lucas, S.M., et al.: Efficient evolutionary methods for game agent optimisation: model-based is best. In: AAAI Workshop on Games and Simulations for Artificial Intelligence (2019)Google Scholar
  9. 9.
    Lucas, S.M., Liu, J., Perez-Liebana, D.: The n-tuple bandit evolutionary algorithm for game agent optimisation. In: Proceedings of IEEE Congress on Evolutionary Computation, CEC 2018 (2018)Google Scholar
  10. 10.
    Myerson, R.: Game Theory: Analysis of Conflict. Harvard University Press, Cambridge (1997)zbMATHGoogle Scholar
  11. 11.
    Perez Liebana, D., Samothrakis, S., Lucas, S., Rohlfshagen, P.: Rolling horizon evolution versus tree search for navigation in single-player real-time games. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, GECCO 2013, pp. 351–358 (2013).
  12. 12.
    Santos, A., Santos, P.A., Melo, F.S.: Monte Carlo tree search experiments in hearthstone. In: 2017 IEEE Conference on Computational Intelligence and Games, CIG 2017, pp. 272–279 (2017)Google Scholar
  13. 13.
    Zhang, S., Buro, M.: Improving hearthstone AI by learning high-level rollout policies and bucketing chance node events. In: 2017 IEEE Conference on Computational Intelligence and Games, CIG 2017, pp. 309–316 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Institute of New Imaging TechnologiesUniversity Jaume ICastellónSpain
  2. 2.Queen Mary University of LondonLondonUK

Personalised recommendations