Efficient Heuristic Policy Optimisation for a Challenging Strategic Card Game
- 550 Downloads
Turn-based multi-action adversarial games are challenging scenarios in which each player turn consists of a sequence of atomic actions. The order in which an AI agent runs these atomic actions may hugely impact the outcome of the turn. One of the main challenges of game artificial intelligence is to design a heuristic function to help agents to select the optimal turn to play, given a particular state of the game. In this paper, we report results using the recently developed N-Tuple Bandit Evolutionary Algorithm to tune the heuristic function parameters. For evaluation, we measure how the tuned heuristic function affects the performance of the state-of-the-art evolutionary algorithm Online Evolution Planning. The multi-action adversarial strategy card game Legends of Code and Magic was used as a testbed. Results indicate that the N-Tuple Bandit Evolutionary Algorithm can effectively tune the heuristic function parameters to improve the performance of the agent.
KeywordsGame artificial intelligence Board and card games solving Learning in games Multi-action games Heuristic policy optimisation
The authors want to thank J. Kowalski for his help solving doubts about the LOCM rules and N. Justensen and H. Baier for their help in better understanding their algorithms.
- 1.Baier, H., Cowling, P.I.: Evolutionary MCTS for multi-action adversarial games. In: Preceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018 (2018). https://doi.org/10.1109/CIG.2018.8490403
- 2.Bursztein, E.: I am a legend: hacking hearthstone using statistical learning methods. In: 2016 IEEE Conference on Computational Intelligence and Games (CIG) (2016). https://doi.org/10.1109/CIG.2016.7860416
- 3.Choe, J.S.B., Kim, J.K.: Enhancing Monte Carlo tree search for playing hearthstone. In: Proceedings of the 1st Conference on Games, GOG 2019 (2019)Google Scholar
- 7.Kowalski, J., Miernik, R.: Legends of code and magic (2019). https://jakubkowalski.tech/Projects/LOCM/. Accessed 26 May 2019
- 8.Lucas, S.M., et al.: Efficient evolutionary methods for game agent optimisation: model-based is best. In: AAAI Workshop on Games and Simulations for Artificial Intelligence (2019)Google Scholar
- 9.Lucas, S.M., Liu, J., Perez-Liebana, D.: The n-tuple bandit evolutionary algorithm for game agent optimisation. In: Proceedings of IEEE Congress on Evolutionary Computation, CEC 2018 (2018)Google Scholar
- 11.Perez Liebana, D., Samothrakis, S., Lucas, S., Rohlfshagen, P.: Rolling horizon evolution versus tree search for navigation in single-player real-time games. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, GECCO 2013, pp. 351–358 (2013). https://doi.org/10.1145/2463372.2463413
- 12.Santos, A., Santos, P.A., Melo, F.S.: Monte Carlo tree search experiments in hearthstone. In: 2017 IEEE Conference on Computational Intelligence and Games, CIG 2017, pp. 272–279 (2017)Google Scholar
- 13.Zhang, S., Buro, M.: Improving hearthstone AI by learning high-level rollout policies and bucketing chance node events. In: 2017 IEEE Conference on Computational Intelligence and Games, CIG 2017, pp. 309–316 (2017)Google Scholar