Advertisement

Orthogonally Evolved AI to Improve Difficulty Adjustment in Video Games

  • Arend HintzeEmail author
  • Randal S. Olson
  • Joel Lehman
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9597)

Abstract

Computer games are most engaging when their difficulty is well matched to the player’s ability, thereby providing an experience in which the player is neither overwhelmed nor bored. In games where the player interacts with computer-controlled opponents, the difficulty of the game can be adjusted not only by changing the distribution of opponents or game resources, but also through modifying the skill of the opponents. Applying evolutionary algorithms to evolve the artificial intelligence that controls opponent agents is one established method for adjusting opponent difficulty. Less-evolved agents (i.e., agents subject to fewer generations of evolution) make for easier opponents, while highly-evolved agents are more challenging to overcome. In this publication we test a new approach for difficulty adjustment in games: orthogonally evolved AI, where the player receives support from collaborating agents that are co-evolved with opponent agents (where collaborators and opponents have orthogonal incentives). The advantage is that game difficulty can be adjusted more granularly by manipulating two independent axes: by having more or less adept collaborators, and by having more or less adept opponents. Furthermore, human interaction can modulate (and be informed by) the performance and behavior of collaborating agents. In this way, orthogonally evolved AI both facilitates smoother difficulty adjustment and enables new game experiences.

Keywords

Difficulty adjustment Coevolution Evolutionary computation Markov Networks 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

We would like to thank Chris Adami for insightful comments and discussion of the project.

References

  1. 1.
    Yannakakis, G.N.: AI in Computer Games (2006)Google Scholar
  2. 2.
    Browne, C.: Evolutionary Game Design. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  3. 3.
    Spronck, P., Sprinkhuizen-Kuyper, I., Postma, E.: Difficulty scaling of game AI. In: Intelligent Games (2004)Google Scholar
  4. 4.
    Hunicke, R., Chapman, V.: AI for dynamic difficulty adjustment in games. In: Challenges in Game Artificial Intelligence AAAI (2004)Google Scholar
  5. 5.
    Overholtzer, C.A., Levy, S.D.: Evolving AI opponents in a first-person-shooter video game. In: AAAI Proceedings of the 20th National Conference on Artificial Intelligence (2005)Google Scholar
  6. 6.
    Cole, N., Louis, S.J., Miles, C.: Using a genetic algorithm to tune first-person shooter bots. Trans. IRE Prof. Group Audio 1, 131–139 (2004)Google Scholar
  7. 7.
    Tan, T.G., Anthony, P., Teo, J., Ong, J.H.: Neural network ensembles for video game AI using evolutionary multi-objective optimization. In: Transactions of the IRE Professional Group on Audio, pp. 605–610, December 2011Google Scholar
  8. 8.
    Yau, Y.J., Teo, J., Anthony, P.: Pareto evolution and co-evolution in cognitive game AI synthesis. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds.) EMO 2007. LNCS, vol. 4403, pp. 227–241. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  9. 9.
    Yau, Y.J., Teo, J., Anthony, P.: Pareto evolution and co-evolution in cognitive neural agents synthesis for Tic-Tac-Toe. In: IEEE Symposium on Computational Intelligence and Games, pp. 304–311. IEEE (2007)Google Scholar
  10. 10.
    Mayer, H.A., Maier, P.: Coevolution of neural go players in a cultural environment. Trans. IRE Prof. Group Audio 2, 1012–1017 (2005)Google Scholar
  11. 11.
    Lubberts, A., Miikkulainen, R.: Co-evolving a go-playing neural network. In: Algorithms Upon Themselves (2001)Google Scholar
  12. 12.
    Chellapilla, K., Fogel, D.B.: Evolving an expert checkers playing program without using human expertise. IEEE Trans. Evol. Comput. 5(4), 422–428 (2001)CrossRefGoogle Scholar
  13. 13.
    Chellapilla, K., Fogel, D.B.: Evolution, neural networks, games, and intelligence. In: Proceedings of the IEEE, pp. 1471–1496 (1999)Google Scholar
  14. 14.
    Lim, C.U., Baumgarten, R., Colton, S.: Evolving behaviour trees for the commercial game DEFCON. In: Di Chio, C., Cagnoni, S., Cotta, C., Ebner, M., Esparcia-Alcazar, A.I., Goh, C.-K., Merelo, J.J., et al. (eds.) EvoApplicatons 2010. LNCS, vol. 6024, pp. 100–110. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  15. 15.
    Hagelbäck, J., Johansson, S.J.: Using multi-agent potential fields in real-time strategy games. In: AAMAS 2008: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, May 2008Google Scholar
  16. 16.
    Priesterjahn, S., Kramer, O., Weimer, A., Goebels, A.: Evolution of human-competitive agents in modern computer games. In: IEEE International Conference on Evolutionary Computation, pp. 777–784. IEEE, November 2005–2006Google Scholar
  17. 17.
    van Valen, L.: A new evolutionary law. Evol. Theor. 1, 1–30 (1973)Google Scholar
  18. 18.
    Bell, G.: The Masterpiece of Nature: The Evolution and Genetics of Sexuality. CUP Archive (1982)Google Scholar
  19. 19.
    Olson, R.S., Knoester, D.B., Adami, C.: Critical interplay between density-dependent predation and evolution of the selfish herd. In: GECCO 2013: Proceeding of the 15th Annual Conference on Genetic and Evolutionary Computation Conference, ACM Request Permissions, July 2013Google Scholar
  20. 20.
    Yannakakis, G.N., Hallam, J.: Evolving opponents for interesting interactive computer games. In: From Animals to Animats (2004)Google Scholar
  21. 21.
    Grand, S., Cliff, D., Malhotra, A.: Creatures: artificial life autonomous software agents for home entertainment. In: AGENTS 1997: Proceedings of the 1st International Conference on Autonomous Agents, ACM, February 1997Google Scholar
  22. 22.
    Pollack, J., Blair, A.: Co-evolution in the successful learning of backgammon strategy. Mach. Learn. 32, 225–240 (1998)CrossRefzbMATHGoogle Scholar
  23. 23.
    Stanley, K.O., Bryant, B.D., Miikkulainen, R.: Evolving neural network agents in the NERO video game. In: Proceedings of the IEEE (2005)Google Scholar
  24. 24.
    Hastings, E.J., Guha, R.K., Stanley, K.O.: Evolving content in the galactic arms race video game. In: IEEE Symposium on Computational Intelligence and Games (CIG), pp. 241–248. IEEE (2009)Google Scholar
  25. 25.
    DeLooze, L.L., Viner, W.R.: Fuzzy Q-learning in a nondeterministic environment: developing an intelligent Ms. Pac-Man agent. In: CIG 2009: Proceedings of the 5th International Conference on Computational Intelligence and Games. IEEE Press, September 2009Google Scholar
  26. 26.
    Handa, H.: Constitution of Ms. PacMan player with critical-situation learning mechanism. Int. J. Knowl. Eng. Soft Data Paradig. 2(3), 237–250 (2010)CrossRefGoogle Scholar
  27. 27.
    Tong, C.K., Hui, O.J., Teo, J., On, C.K.: The evolution of gamebots for 3D first person shooter (FPS). Transactions of the IRE Professional Group on Audio, pp. 21–26, September 2011Google Scholar
  28. 28.
    Agapitos, A., Togelius, J., Lucas, S.M., Schmidhuber, J., Konstantinidis, A.: Generating diverse opponents with multiobjective evolution. In: IEEE Symposium on Computational Intelligence and Games, CIG 2008, pp. 135–142. IEEE (2008)Google Scholar
  29. 29.
    Olson, R.S., Hintze, A., Dyer, F.C., Knoester, D.B., Adami, C.: Predator confusion is sufficient to evolve swarming behaviour. J. Roy. Soc. Interface 10(85), 20130305 (2013)CrossRefGoogle Scholar
  30. 30.
    Marstaller, L., Hintze, A., Adami, C.: The evolution of representation in simple cognitive networks. Neural Comput. 25(8), 2079–2107 (2013)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Hamilton, W.D.W.: Geometry for the selfish herd. J. Theor. Biol. 31(2), 295–311 (1971)CrossRefGoogle Scholar
  32. 32.
    Lenski, R.E., Ofria, C., Pennock, R.T., Adami, C.: The evolutionary origin of complex features. Nature 423(6), 139–144 (2003)CrossRefGoogle Scholar
  33. 33.
    Fry, B., Reas, C.: Processing Library for Visual Arts and DesignGoogle Scholar
  34. 34.
    Toner, J., Tu, Y.: Flocks, herds, and schools: a quantitative theory of flocking. Trans. IRE Prof. Group Audio (April 1998)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Michigan State UniversityMichiganUSA
  2. 2.University of PennsylvaniaPennsylvaniaUSA
  3. 3.IT University of CopenhagenCopenhagenDenmark

Personalised recommendations

Cite paper

Buy options