Advertisement

Machine Learning

, Volume 63, Issue 3, pp 217–248 | Cite as

Adaptive game AI with dynamic scripting

  • Pieter Spronck
  • Marc Ponsen
  • Ida Sprinkhuizen-Kuyper
  • Eric Postma
Article

Abstract

Online learning in commercial computer games allows computer-controlled opponents to adapt to the way the game is being played. As such it provides a mechanism to deal with weaknesses in the game AI, and to respond to changes in human player tactics. We argue that online learning of game AI should meet four computational and four functional requirements. The computational requirements are speed, effectiveness, robustness and efficiency. The functional requirements are clarity, variety, consistency and scalability. This paper investigates a novel online learning technique for game AI called ‘dynamic scripting’, that uses an adaptive rulebase for the generation of game AI on the fly. The performance of dynamic scripting is evaluated in experiments in which adaptive agents are pitted against a collection of manually-designed tactics in a simulated computer roleplaying game. Experimental results indicate that dynamic scripting succeeds in endowing computer-controlled opponents with adaptive performance. To further improve the dynamic-scripting technique, an enhancement is investigated that allows scaling of the difficulty level of the game AI to the human player’s skill level. With the enhancement, dynamic scripting meets all computational and functional requirements. The applicability of dynamic scripting in state-of-the-art commercial games is demonstrated by implementing the technique in the game Neverwinter Nights. We conclude that dynamic scripting can be successfully applied to the online adaptation of game AI in commercial computer games.

Keywords

Computer game Reinforcement learning Dynamic scripting 

References

  1. Adamatzky, A. (2000). CREATURES - artificial life, autonomous agents and gaming environment. Kybernetes: The International Journal of Systems & Cybernetics 29:2.Google Scholar
  2. Allen, M., Suliman, H., Wen, Z., Gough, N., & Mehdi, Q. (2001). Directions for future game development. In: Q. Mehdi, N. Gough, and D. Al-Dabass (eds.): Proceedings of the Second International Conference on Intelligent Games and Simulation (pp. 22–32). SCS Europe Bvba.Google Scholar
  3. Brockington, M., & Darrah, M. (2002). How not to implement a basic scripting language. In: S. Rabin (ed.): AI Game Programming Wisdom (pp. 548–554). Charles River Media, Inc.Google Scholar
  4. Buro, M. (2003). RTS games as test-bed for real-time AI research. In: K. Chen, S. Chen, H. Cheng, D. Chiu, S. Das, R. Duro, Z. Jiang, N. Kasabov, E. Kerre, H. Leong, Q. Li, M. Lu, M. Grana Romay, D. Ventura, P. Wang, and J. Wu (eds.): Proceedings of the 7th Joint Conference on Information Science (JCIS 2003) (pp. 481–484).Google Scholar
  5. Buro, M. (2004). Call for AI research in RTS games. In: Proceedings of the AAAI-04 Workshop on Challenges in Game AI (pp. 139–142). AAAI Press.Google Scholar
  6. Charles, D., & Black, M. (2004). Dynamic player modelling: A framework for player-centric digital games. In: Q. Mehdi, N. Gough, S. Natkin, and D. Al- Dabass (eds.): Computer Games: Artificial Intelligence, Design and Education (CGAIDE 2004) (pp. 29–35). University of Wolverhampton.Google Scholar
  7. Charles, D., & Livingstone, D. (2004). AI: The missing link in game interface design. In: M. Rauterberg (ed.): Entertainment Computing – ICEC 2004 (pp. 351–354). Springer-Verlag.Google Scholar
  8. Cohen, P. (1995). Empirical Methods for Artificial Intelligence. MIT Press.Google Scholar
  9. Dahlbom, A. (2004). An Adaptive AI for Real-Time Strategy Games, M.Sc. thesis. Högskolan i Skövde.Google Scholar
  10. Demasi, P., & Cruz, A. (2002). Online coevolution for action games. International Journal of Intelligent Games and Simulation 2:2, 80–88.Google Scholar
  11. Evans, R. (2002). Varieties of learning. In: S. Rabin (ed.): AI Game Programming Wisdom (pp. 567–578). Charles River Media, Inc.Google Scholar
  12. Forbus, K., & Laird, J. (2002). AI and the entertainment industry. IEEE Intelligent Systems 17:4, 15–16.Google Scholar
  13. Gold, J. (2004). Object-oriented Game Development. Addison-Wesley.Google Scholar
  14. Graepel, T., Herbrich, R., & Gold, J. (2004). Learning to fight. In: Q. Mehdi, N. Gough, S. Natkin, and D. Al-Dabass (eds.): Computer Games: Artificial Intelligence, Design and Education (CGAIDE 2004). (pp. 193–200). University of Wolverhampton.Google Scholar
  15. Iida, H., Handa, K., & Uiterwijk, J. (1995). Tutoring strategies in game-tree search. ICCA Journal 18:4, 191–204.Google Scholar
  16. Johnson, S. (2004). Adaptive AI: A practical example. In: S. Rabin (ed.): AI Game Programming Wisdom 2, 639–647. Charles River Media, Inc.Google Scholar
  17. Laird, J., & Van Lent, M. (2001). Human-level’s AI killer application: Interactive Computer Games. Artificial Intelligence Magazine 22:2, 15–26.Google Scholar
  18. Lidén, L. (2004). Artificial Stupidity: The art of making intentional mistakes. In: S. Rabin (ed.): AI Game Programming Wisdom 2, 41–48. Charles River Media, Inc.Google Scholar
  19. Madeira, C., Corruble, V., Ramalho, G., & Ratitch, B. (2004). Bootstrapping the learning process for the semi-automated design of challenging game AI. In: D. Fu, S. Henke, and J. Orkin (eds.): Proceedings of the AAAI-04 Workshop on Challenges in Game Artificial Intelligence. (pp. 72–76). AAAI Press.Google Scholar
  20. Manslow, J. (2002). Learning and adaptation. In: S. Rabin (ed.): AI Game Programming Wisdom. (pp. 557–566). Charles River Media, Inc.Google Scholar
  21. Michalewicz, Z., & Fogel, D. (2000) How To Solve It: Modern Heuristics. Springer- Verlag.Google Scholar
  22. Nareyek, A. (2002). Intelligent agents for computer games. In: T. Marsland and I. Frank (eds.): Computers and Games, Second International Conference, CG 2000, Vol. 2063 of Lecture Notes in Computer Science. (pp. 414–422). Springer-Verlag.Google Scholar
  23. Ponsen, M., Muñoz-Avila, H., Spronck, P., & Aha, D. (2005). Automatically acquiring adaptive real-time strategy game opponents using evolutionary learning. In: Proceedings of the Twentieth National Conference on Artificial Intelligence and the Seventeenth Innovative Applications of Artificial Intelligence. (pp. 1535-1540), AAAI Press.Google Scholar
  24. Ponsen, M., & Spronck, P. (2004). Improving adaptive game AI with evolutionary learning. In: Q. Mehdi, N. Gough, S. Natkin, and D. Al-Dabass (eds.): Computer Games: Artificial Intelligence, Design and Education (CGAIDE 2004) (pp. 389–396). University of Wolverhampton.Google Scholar
  25. Rabin, S. (2004). Promising game AI techniques. In: S. Rabin (ed.): AI Game Programming Wisdom 2, 15–27. Charles River Media, Inc.Google Scholar
  26. Schaeffer, J. (2001). A gamut of games. Artificial Intelligence Magazine 22:3, 29–46.Google Scholar
  27. Scott, B. (2002). The illusion of intelligence. In: S. Rabin (ed.): AI Game Programming Wisdom. (pp.16–20). Charles River Media, Inc.Google Scholar
  28. Spronck, P., Sprinkhuizen-Kuyper, I., & Postma, E. (2003). Improving opponent intelligence through offline evolutionary learning. International Journal of Intelligent Games and Simulation 2:1, 20–27.Google Scholar
  29. Spronck, P. (2005). Adaptive Game AI, Ph.D. thesis. Universitaire Pers Maastricht.Google Scholar
  30. Sutton, R., & Barto, A. (1998). Reinforcement Learning: An Introduction. MIT Press.Google Scholar
  31. Tomlinson, S. (2003). Working at thinking about playing or a year in the life of a games AI programmer. In: Q. Mehdi, N. Gough, and S. Natkin (eds.): Proceedings of the 4th International Conference on Intelligent Games and Simulation (GAME-ON 2003). (pp. 5–12). EUROSIS.Google Scholar
  32. Tozour, P. (2002a). The evolution of game AI. In: S. Rabin (ed.): AI Game Programming Wisdom. (pp. 3–15). Charles River Media, Inc.Google Scholar
  33. Tozour, P. (2002b). The perils of AI scripting. In: S. Rabin (ed.): AI Game Programming Wisdom. (pp. 541–547). Charles River Media, Inc.Google Scholar
  34. Woodcock, S. (2000). The future of game AI: A personal view. Game Developer Magazine 7:8.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  • Pieter Spronck
    • 1
  • Marc Ponsen
    • 1
  • Ida Sprinkhuizen-Kuyper
    • 1
  • Eric Postma
    • 1
  1. 1.Institute for Knowledge and Agent TechnologyUniversiteit MaastrichtMaastrichtThe Netherlands

Personalised recommendations