Orthogonally Evolved AI to Improve Difficulty Adjustment in Video Games
Abstract
Computer games are most engaging when their difficulty is well matched to the player’s ability, thereby providing an experience in which the player is neither overwhelmed nor bored. In games where the player interacts with computer-controlled opponents, the difficulty of the game can be adjusted not only by changing the distribution of opponents or game resources, but also through modifying the skill of the opponents. Applying evolutionary algorithms to evolve the artificial intelligence that controls opponent agents is one established method for adjusting opponent difficulty. Less-evolved agents (i.e., agents subject to fewer generations of evolution) make for easier opponents, while highly-evolved agents are more challenging to overcome. In this publication we test a new approach for difficulty adjustment in games: orthogonally evolved AI, where the player receives support from collaborating agents that are co-evolved with opponent agents (where collaborators and opponents have orthogonal incentives). The advantage is that game difficulty can be adjusted more granularly by manipulating two independent axes: by having more or less adept collaborators, and by having more or less adept opponents. Furthermore, human interaction can modulate (and be informed by) the performance and behavior of collaborating agents. In this way, orthogonally evolved AI both facilitates smoother difficulty adjustment and enables new game experiences.
Keywords
Difficulty adjustment Coevolution Evolutionary computation Markov NetworksNotes
Acknowledgments
We would like to thank Chris Adami for insightful comments and discussion of the project.
References
- 1.Yannakakis, G.N.: AI in Computer Games (2006)Google Scholar
- 2.Browne, C.: Evolutionary Game Design. Springer, Heidelberg (2011)CrossRefGoogle Scholar
- 3.Spronck, P., Sprinkhuizen-Kuyper, I., Postma, E.: Difficulty scaling of game AI. In: Intelligent Games (2004)Google Scholar
- 4.Hunicke, R., Chapman, V.: AI for dynamic difficulty adjustment in games. In: Challenges in Game Artificial Intelligence AAAI (2004)Google Scholar
- 5.Overholtzer, C.A., Levy, S.D.: Evolving AI opponents in a first-person-shooter video game. In: AAAI Proceedings of the 20th National Conference on Artificial Intelligence (2005)Google Scholar
- 6.Cole, N., Louis, S.J., Miles, C.: Using a genetic algorithm to tune first-person shooter bots. Trans. IRE Prof. Group Audio 1, 131–139 (2004)Google Scholar
- 7.Tan, T.G., Anthony, P., Teo, J., Ong, J.H.: Neural network ensembles for video game AI using evolutionary multi-objective optimization. In: Transactions of the IRE Professional Group on Audio, pp. 605–610, December 2011Google Scholar
- 8.Yau, Y.J., Teo, J., Anthony, P.: Pareto evolution and co-evolution in cognitive game AI synthesis. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds.) EMO 2007. LNCS, vol. 4403, pp. 227–241. Springer, Heidelberg (2007)CrossRefGoogle Scholar
- 9.Yau, Y.J., Teo, J., Anthony, P.: Pareto evolution and co-evolution in cognitive neural agents synthesis for Tic-Tac-Toe. In: IEEE Symposium on Computational Intelligence and Games, pp. 304–311. IEEE (2007)Google Scholar
- 10.Mayer, H.A., Maier, P.: Coevolution of neural go players in a cultural environment. Trans. IRE Prof. Group Audio 2, 1012–1017 (2005)Google Scholar
- 11.Lubberts, A., Miikkulainen, R.: Co-evolving a go-playing neural network. In: Algorithms Upon Themselves (2001)Google Scholar
- 12.Chellapilla, K., Fogel, D.B.: Evolving an expert checkers playing program without using human expertise. IEEE Trans. Evol. Comput. 5(4), 422–428 (2001)CrossRefGoogle Scholar
- 13.Chellapilla, K., Fogel, D.B.: Evolution, neural networks, games, and intelligence. In: Proceedings of the IEEE, pp. 1471–1496 (1999)Google Scholar
- 14.Lim, C.U., Baumgarten, R., Colton, S.: Evolving behaviour trees for the commercial game DEFCON. In: Di Chio, C., Cagnoni, S., Cotta, C., Ebner, M., Esparcia-Alcazar, A.I., Goh, C.-K., Merelo, J.J., et al. (eds.) EvoApplicatons 2010. LNCS, vol. 6024, pp. 100–110. Springer, Heidelberg (2010)CrossRefGoogle Scholar
- 15.Hagelbäck, J., Johansson, S.J.: Using multi-agent potential fields in real-time strategy games. In: AAMAS 2008: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, May 2008Google Scholar
- 16.Priesterjahn, S., Kramer, O., Weimer, A., Goebels, A.: Evolution of human-competitive agents in modern computer games. In: IEEE International Conference on Evolutionary Computation, pp. 777–784. IEEE, November 2005–2006Google Scholar
- 17.van Valen, L.: A new evolutionary law. Evol. Theor. 1, 1–30 (1973)Google Scholar
- 18.Bell, G.: The Masterpiece of Nature: The Evolution and Genetics of Sexuality. CUP Archive (1982)Google Scholar
- 19.Olson, R.S., Knoester, D.B., Adami, C.: Critical interplay between density-dependent predation and evolution of the selfish herd. In: GECCO 2013: Proceeding of the 15th Annual Conference on Genetic and Evolutionary Computation Conference, ACM Request Permissions, July 2013Google Scholar
- 20.Yannakakis, G.N., Hallam, J.: Evolving opponents for interesting interactive computer games. In: From Animals to Animats (2004)Google Scholar
- 21.Grand, S., Cliff, D., Malhotra, A.: Creatures: artificial life autonomous software agents for home entertainment. In: AGENTS 1997: Proceedings of the 1st International Conference on Autonomous Agents, ACM, February 1997Google Scholar
- 22.Pollack, J., Blair, A.: Co-evolution in the successful learning of backgammon strategy. Mach. Learn. 32, 225–240 (1998)CrossRefzbMATHGoogle Scholar
- 23.Stanley, K.O., Bryant, B.D., Miikkulainen, R.: Evolving neural network agents in the NERO video game. In: Proceedings of the IEEE (2005)Google Scholar
- 24.Hastings, E.J., Guha, R.K., Stanley, K.O.: Evolving content in the galactic arms race video game. In: IEEE Symposium on Computational Intelligence and Games (CIG), pp. 241–248. IEEE (2009)Google Scholar
- 25.DeLooze, L.L., Viner, W.R.: Fuzzy Q-learning in a nondeterministic environment: developing an intelligent Ms. Pac-Man agent. In: CIG 2009: Proceedings of the 5th International Conference on Computational Intelligence and Games. IEEE Press, September 2009Google Scholar
- 26.Handa, H.: Constitution of Ms. PacMan player with critical-situation learning mechanism. Int. J. Knowl. Eng. Soft Data Paradig. 2(3), 237–250 (2010)CrossRefGoogle Scholar
- 27.Tong, C.K., Hui, O.J., Teo, J., On, C.K.: The evolution of gamebots for 3D first person shooter (FPS). Transactions of the IRE Professional Group on Audio, pp. 21–26, September 2011Google Scholar
- 28.Agapitos, A., Togelius, J., Lucas, S.M., Schmidhuber, J., Konstantinidis, A.: Generating diverse opponents with multiobjective evolution. In: IEEE Symposium on Computational Intelligence and Games, CIG 2008, pp. 135–142. IEEE (2008)Google Scholar
- 29.Olson, R.S., Hintze, A., Dyer, F.C., Knoester, D.B., Adami, C.: Predator confusion is sufficient to evolve swarming behaviour. J. Roy. Soc. Interface 10(85), 20130305 (2013)CrossRefGoogle Scholar
- 30.Marstaller, L., Hintze, A., Adami, C.: The evolution of representation in simple cognitive networks. Neural Comput. 25(8), 2079–2107 (2013)MathSciNetCrossRefGoogle Scholar
- 31.Hamilton, W.D.W.: Geometry for the selfish herd. J. Theor. Biol. 31(2), 295–311 (1971)CrossRefGoogle Scholar
- 32.Lenski, R.E., Ofria, C., Pennock, R.T., Adami, C.: The evolutionary origin of complex features. Nature 423(6), 139–144 (2003)CrossRefGoogle Scholar
- 33.Fry, B., Reas, C.: Processing Library for Visual Arts and DesignGoogle Scholar
- 34.Toner, J., Tu, Y.: Flocks, herds, and schools: a quantitative theory of flocking. Trans. IRE Prof. Group Audio (April 1998)Google Scholar