Abstract
Recently, games like Pac-Man have been hotbeds for neuroevolution research and NEAT has emerged as one of the leading techniques in the game playing domain [14] . In the context of the snake game, the goal of this paper is to enhance neuroevolution strategies with better fitness functions for effective goal finding. We develop greedy and non-greedy fitness functions, and demonstrate the effectiveness of these functions in both environments with and without dynamic obstacles. We then present an alternate implementation using the NEAT algorithm combined with Novelty Search to increase the genetic diversity of the agent population and explore the problem space without specifying direct objectives. These conclusions suggest that even with a low number of simple inputs, and simple fitness functions, agents are quickly able to achieve a novice amount of expertise in the Snake game using NEAT.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cuccu, G., Gomez, F.: When novelty is not enough. In: Di Chio, C., et al. (eds.) EvoApplications 2011. LNCS, vol. 6624, pp. 234–243. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20525-5_24
Ehlis, T., Hattan, J., Sikora, D.: Application of genetic programming to the snake game. Gamedev. Net 175 (2000)
Eiter, T., Mannila, H.: Computing discrete Frechet Distance. Technical Report CD-TR 94/64, p. 8 (1994)
Floreano, D., Dürr, P., Mattiussi, C.: Neuroevolution: from architectures to learning. Evol. Intell. 1(1), 47–62 (2008)
Gomes, J., Mariano, P., Christensen, A.L.: Devising effective novelty search algorithms: a comprehensive empirical study. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 943–950 (2015)
Hausknecht, M., Lehman, J., Miikkulainen, R., Stone, P.: A neuroevolution approach to general Atari game playing. IEEE Trans. Comput. Intell. AI Games 6(4), 355–366 (2014)
Kistemaker, S., Whiteson, S.: Critical factors in the performance of novelty search. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, pp. 965–972 (2011)
Lehman, J., Stanley, K.O.: Abandoning objectives: evolution through the search for novelty alone. Evol. Comput. 19(2), 189–223 (2011)
Lockhart, C.: Application of temporal difference learning to the game of Snake. Ph.D. thesis, University of Louisville (2010)
Ma, B., Tang, M., Zhang, J.: Exploration of reinforcement learning to snake (2016)
Ollion, C., Doncieux, S.: Why and how to measure exploration in behavioral space. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation - GECCO 2011, p. 267. ACM Press, Dublin (2011)
Risi, S., Togelius, J.: Neuroevolution in games: state of the art and open challenges. IEEE Trans. Comput. Intell. AI Games 9(1), 25–41 (2017)
Rodzin, S., Rodzina, O., Rodzina, L.: Neuroevolution: problems, algorithms, and experiments. In: 2016 IEEE 10th International Conference on Application of Information and Communication Technologies (AICT), pp. 1–4. IEEE (2016)
Samothrakis, S., Perez-Liebana, D., Lucas, S.M., Fasli, M.: Neuroevolution for general video game playing. In: 2015 IEEE Conference on Computational Intelligence and Games (CIG) (2015)
Stanley, K.O., Bryant, B.D., Miikkulainen, R.: Real-time neuroevolution in the nero video game. IEEE Trans. Evol. Comput. 9(6), 653–668 (2005)
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)
Stanton, C., Clune, J.: Curiosity search: producing generalists by encouraging individuals to continually explore and acquire skills throughout their lifetime. PLOS ONE 11(9), 1–20 (2016)
Urbano, P., Georgiou, L.: Improving grammatical evolution in Santa Fe Trail using novelty search. In: Advances in Artificial Life, ECAL 2013, pp. 917–924. MIT Press, September 2013
Wei, Z., Wang, D., Zhang, M., Tan, A.H., Miao, C., Zhou, Y.: Autonomous agents in snake game via deep reinforcement learning. In: 2018 IEEE International Conference on Agents (ICA), pp. 20–25. IEEE, Singapore, July 2018
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Vignesh Kumar, K., Sourav, R., Shunmuga Velayutham, C., Balasubramanian, V. (2020). Fitness Function Design for Neuroevolution in Goal-Finding Game Environments. In: Hernes, M., Wojtkiewicz, K., Szczerbicki, E. (eds) Advances in Computational Collective Intelligence. ICCCI 2020. Communications in Computer and Information Science, vol 1287. Springer, Cham. https://doi.org/10.1007/978-3-030-63119-2_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-63119-2_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63118-5
Online ISBN: 978-3-030-63119-2
eBook Packages: Computer ScienceComputer Science (R0)