Abstract
Game agents are efficiently implemented through different AI techniques, such as neural network, reinforcement learning, and evolutionary search. Although there are many works for each approach, we present a critical analysis and comparison between them, suggesting a common benchmark and parameter configurations. The evolutionary strategy implements the NeuroEvolution of Augmenting Topologies algorithm, while the reinforcement learning agent leverages Q-Learning and Proximal Policy Optimization. We formulate and empirically compare this set of solutions using the Flappy Bird game as a test scenario. We also compare different representations of state and reward functions for each method. All methods were able to generate agents that can play the game, where the NEAT algorithm had the best results, reaching the goal of never losing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Yannakakis, G.N., Togelius, J.: Artificial Intelligence and Games, vol. 2. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-63519-4
Silver, D., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Vinyals, O., et al.: AlphaStar: mastering the real-time strategy game StarCraft II (2019). https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/
Samsuden, M.A., Diah, N.M., Rahman, N.A.: A review paper on implementing reinforcement learning technique in optimising games performance. In: 2019 IEEE 9th International Conference on System Engineering and Technology (ICSET), pp. 258–263. IEEE (2019)
Injadat, M.N., Moubayed, A., Nassif, A.B., Shami, A.: Machine learning towards intelligent systems: applications, challenges, and opportunities. Artifi. Intell. Rev. 54(5), 3299–3348 (2021). https://doi.org/10.1007/s10462-020-09948-w
Unity. https://unity.com/. Accessed 19 June 2021
Pygame. https://www.pygame.org/. Accessed 19 June 2021
Mishra, Y., Kumawat, V., Selvakumar, K.: Performance analysis of flappy bird playing agent using neural network and genetic algorithm. In: Gani, A.B., Das, P.K., Kharb, L., Chahal, D. (eds.) ICICCT 2019. CCIS, vol. 1025, pp. 253–265. Springer, Singapore (2019). https://doi.org/10.1007/978-981-15-1384-8_21
Vu, T., Tran, L.: FlapAI bird: training an agent to play flappy bird using reinforcement learning techniques. arXiv preprint arXiv:2003.09579 (2020)
Hosu, I., Urzica, A.: Comparative analysis of existing architectures for general game agents. In: 2015 17th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), pp. 257–260 (2015)
Jeerige, A., Bein, D., Verma, A.: Comparison of deep reinforcement learning approaches for intelligent game playing. In: 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0366–0371 (2019)
Mirjalili, S.: Genetic algorithm. In: Evolutionary Algorithms and Neural Networks. SCI, vol. 780, pp. 43–55. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-93025-1_4
Marsland, S.: Machine Learning - An Algorithmic Perspective. Chapman and Hall/CRC Machine Learning and Pattern Recognition Series. CRC Press (2009)
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)
Lanham, M.: Learn Unity ML-Agents - Fundamentals of Unity Machine Learning: Incorporate New Powerful ML Algorithms such as Deep Reinforcement Learning for Games. Packt Publishing, Birmingham (2018)
Goulart, Í., Paes, A., Clua, E.: Learning how to play Bomberman with deep reinforcement and imitation learning. In: van der Spek, E., Göbel, S., Do, E.Y.-L., Clua, E., Baalsrud Hauge, J. (eds.) ICEC-JCSG 2019. LNCS, vol. 11863, pp. 121–133. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-34644-7_10
Schulman, J., et al.: Proximal policy optimization algorithms (2017)
Pecenin, M., Maidl, A., Weingaertner, D.: Optimization of halide image processing schedules with reinforcement learning. In: Anais do XX Simpósio em Sistemas Computacionais de Alto Desempenho, pp. 37–48. SBC, Porto Alegre (2019)
McIntyre, A., et al.: Neat-python. https://github.com/CodeReclaimers/neat-python
Unity ML-Agents PPO hyperparameters configurations. https://github.com/Unity-Technologies/ml-agents/blob/release_15_docs/docs/Training-Configuration-File.md. Accessed 19 June 2021
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 IFIP International Federation for Information Processing
About this paper
Cite this paper
Thurler, L., Montes, J., Veloso, R., Paes, A., Clua, E. (2021). AI Game Agents Based on Evolutionary Search and (Deep) Reinforcement Learning: A Practical Analysis with Flappy Bird. In: Baalsrud Hauge, J., C. S. Cardoso, J., Roque, L., Gonzalez-Calero, P.A. (eds) Entertainment Computing – ICEC 2021. ICEC 2021. Lecture Notes in Computer Science(), vol 13056. Springer, Cham. https://doi.org/10.1007/978-3-030-89394-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-89394-1_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89393-4
Online ISBN: 978-3-030-89394-1
eBook Packages: Computer ScienceComputer Science (R0)
