Adapting to Human Gamers Using Coevolution

  • Phillipa M. Avery
  • Zbigniew Michalewicz
Part of the Studies in Computational Intelligence book series (SCI, volume 263)


No matter how good a computer player is, given enough time human players may learn to adapt to the strategy used, and routinely defeat the computer player. A challenging task is to mimic this human ability to adapt, and create a computer player that can adapt to its opposition’s strategy. By having an adaptive strategy for a computer player, the challenge it provides is ongoing. Additionally, a computer player that adapts specifically to an individual human provides a more personal and tailored game play experience. To address this need we have investigated the creation of such a computer player. By creating a computer player that changes its strategy with influence from the human strategy, we have shown that the holy grail of gaming – an individually tailored gaming experience, is indeed possible. We designed the computer player for the game of TEMPO, a zero sum military planning game. The player was created through a process that reverse engineers the human strategy and uses it to coevolve the computer player.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    The Planning, Programming, and Budgeting System (PPBS) directive. Tech. rep., USA Department of Defense (1984)Google Scholar
  2. 2.
    TEMPO military planning game – explanation and rules for players. Tech. rep. (2003)Google Scholar
  3. 3.
    Aha, D., Molineaux, M., Ponsen, M.: Learning to win: Case-based plan selection in a real-time strategy game. In: Muñoz-Ávila, H., Ricci, F. (eds.) ICCBR 2005. LNCS (LNAI), vol. 3620, pp. 5–20. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Avery, P., Greenwood, G., Michalewicz, Z.: Coevolving strategic intelligence. In: IEEE Proceedings for Congress on Evolutionary Computation, Hong Kong, China (2008)Google Scholar
  5. 5.
    Becker, K.: Teaching with games: the minesweeper and asteroids experience. Journal of Computing Sciences in Colleges 17, 23–33 (2001)Google Scholar
  6. 6.
    Carberry, S.: Techniques for plan recognition. In: User Modeling and User-Adapted Interaction, vol. 11, pp. 31–48. Kluwer Academic Publishers, Dordrecht (2001)Google Scholar
  7. 7.
    Charles, D., McNeill, M., McAlister, M., Black, M., Moore, A., Stringer, K., Kucklich, J., Kerr, A.: Player-centered game design: Player modeling and adaptive digital games. In: DiGRA 2005 Conference: Changing Views – Worlds in Play (2005)Google Scholar
  8. 8.
    Davis, I.L.: Strategies for strategy game AI. In: AAAI 1999 Spring Symposia, pp. 24–27 (1999)Google Scholar
  9. 9.
    Epstein, S.L.: Game playing: The next moves. In: Proceedings of the Sixteenth National Conference on Artificial Intelligence, pp. 987–993. AAAI Press, Menlo Park (1999)Google Scholar
  10. 10.
    Funes, P., Pollack, J.: Measuring progress in coevolutionary competition. In: Animals to Animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior, pp. 450–459. MIT Press, Cambridge (2000)Google Scholar
  11. 11.
    Funes, P., Sklar, E., Juillé, H., Pollack, J.: Animal-animat coevolution: Using the animal population as fitness function. In: Animals to Animats 5: Proceedings of the Fifth International Conference on Simulation of Adaptive Behavior, pp. 525–533. MIT Press, Cambridge (1998)Google Scholar
  12. 12.
    Giordano, J.C., Reynolds Jr., P.F., Brogan, D.C.: Exploring the constraints of human behavior representation. In: Proceedings of the 2004 Winter Simulation Conference, pp. 912–920 (2004)Google Scholar
  13. 13.
    Johnson, R.W., Michalewicz, Z., Melich, M.E., Schmidt, M.: Coevolutionary tempo game. In: Congress on Evolutionary Computation, Portland, Oregon, vol. 2, pp. 1610–1617 (2004)Google Scholar
  14. 14.
    Louis, S.J., McDonnell, J.: Learning with case-injected genetic algorithms. IEEE Transactions on Evolutionary Computation 8, 316–328 (2004)CrossRefGoogle Scholar
  15. 15.
    Louis, S.J., McDonnell, J.: Playing to train: Case injected genetic algorithms for strategic computer gaming. In: GECCO (2004)Google Scholar
  16. 16.
    Louis, S.J., Miles, C.: Combining case-based memory with genetic algorithm search for competent game AI. In: ICCBR Workshops, pp. 193–205 (2005)Google Scholar
  17. 17.
    Louis, S.J., Miles, C.: Playing to learn: Case-injected genetic algorithms for learning to play computer games. IEEE Transactions on Evolutionary Computation 9, 669–681 (2005)CrossRefGoogle Scholar
  18. 18.
    Miles, C., Louis, S., Cole, N., McDonnell, J.: Learning to play like a human: case injected genetic algorithms for strategic computer gaming. In: Congress on Evolutionary Computation, vol. 2, pp. 1441–1448 (2004)Google Scholar
  19. 19.
    Ponsen, M., Muoz-Avila, H., Spronck, P., Aha, D.: Automatically generating game tactics via evolutionary learning. AI Magazine 27, 75–84 (2006)Google Scholar
  20. 20.
    Ponsen, M., Spronck, P.: Improving adaptive game AI with evolutionary learning. In: Computer Games: Artificial Intelligence, Design and Education, pp. 389–396 (2004)Google Scholar
  21. 21.
    Ponsen, M., Spronck, P., Muoz-Avila, H., Aha, D.: Knowledge acquisition for adaptive game AI. Science of Computer Programming 67, 59–75 (2007)MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Scott, B.: AI game programming wisdom. In: Rabin, S. (ed.) The Illusion of Intelligence, pp. 16–20. Charles River Media (2002)Google Scholar
  23. 23.
    Spronck, P.: Adaptive game AI. Ph.D. thesis, Maastricht University (2005)Google Scholar
  24. 24.
    Spronck, P., Ponsen, M., Sprinkhuizen-Kuyper, I., Postma, E.: Adaptive game AI with dynamic scripting. Machine Learning 63, 217–248 (2006)CrossRefGoogle Scholar
  25. 25.
    Spronck, P., Sprinkhuizen-Kuyper, I., Postma, E.: Online adaptation of computer game opponent AI. In: Proceedings of the 15th Belgium-Netherlands Conference on Artificial Intelligence, vol. 2003, pp. 291–298 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Phillipa M. Avery
    • 1
  • Zbigniew Michalewicz
    • 2
    • 3
  1. 1.Department of Computer ScienceUniversity of Adelaide
  2. 2.School of Computer ScienceUniversity of Adelaide, South Australia; also at the Institute of Computer Science, Polish Academy of SciencesWarsawPoland
  3. 3.Polish-Japanese Institute of Information TechnologyWarsawPoland

Personalised recommendations