Computational intelligence and games: Challenges and opportunities

  • Simon M. LucasEmail author


The last few decades have seen a phenomenal increase in the quality, diversity and pervasiveness of computer games. The worldwide computer games market is estimated to be worth around USD 21bn annually, and is predicted to continue to grow rapidly. This paper reviews some of the recent developments in applying computational intelligence (CI) methods to games, points out some of the potential pitfalls, and suggests some fruitful directions for future research.


Games machine learning evolution temporal difference learning (TDL) neural networks n-tuple systems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    J. Schaeffer, N. Burch, Y. Björnsson, A. Kishimoto, M. Müller, R. Lake, P. Lu, S. Sutphen. Checkers Is Solved. Science, vol. 317, no. 5844, pp. 1518–1522, 2007.Google Scholar
  2. [2]
    K. Chellapilla, D. B. Fogel. Evolving an Expert Checkers Playing Program without Using Human Expertise. IEEE Transactions on Evolutionary Computation, vol. 5, no. 4, pp. 422–428, 2001.CrossRefGoogle Scholar
  3. [3]
    D. B. Fogel, T. J. Hays, S. L. Hahn, J. Quon. An Evolutionary Self-learning Chess Program. Proceedings of the IEEE, vol. 92, no. 12, pp. 1947–1954, 2004.CrossRefGoogle Scholar
  4. [4]
    S. Gelly, Y. Wang, R. Munos, O. Teytaud. Modification of UCT with Patterns in Monte-Carlo Go, Technical Report 6062, INRIA, France, 2006.Google Scholar
  5. [5]
    Y. Wang, S. Gelly. Modifications of UCT and Sequence-like Simulations for Monte-Carlo Go. In Proceedings of IEEE Symposium on Computational Intelligence and Games, IEEE Press, pp. 175–182, 2007.Google Scholar
  6. [6]
    E. Charniak. Statistical Language Learning, MIT Press, Cambrige, Massachusetts, USA, 1996.Google Scholar
  7. [7]
    S. Colton, P. Cowling, S. M. Lucas. An Industry/Academia Research Network on Artificial Intelligence and Games Technologies, Technical Report EP/F033834, EPSRC, Swindon, 2007.Google Scholar
  8. [8]
    M. Buro. ProbCut: An Effective Selective Extension of the Aalpha-Beta Algorithm. ICCA Journal, vol. 18, no. 2, pp. 71–76, 1995.MathSciNetGoogle Scholar
  9. [9]
    T. Gosling, N. Jin, E. Tsang. Games, Supply Chains and Automatic Strategy Discovery Using Evolutionary Computation. Handbook of Research on Nature-inspired Computing for Economics and Management, J. P. Rennard (ed.), vol. 2, pp. 572–588, 2007.Google Scholar
  10. [10]
    K. O. Stanley, B. D. Bryant, R. Miikkulainen. Real-time Neuroevolution in the NERO Video Game. IEEE Transactions on Evolutionary Computation, vol. 9, no. 6, pp. 653–668, 2005.CrossRefGoogle Scholar
  11. [11]
    R. M. Axelrod. The Evolution of Cooperation, Basic Books Inc., New York, USA, 1984.Google Scholar
  12. [12]
    N. Jin, E. Tsang. Co-adaptive Strategies for Sequential Bargaining Problems with Discount Factors and Outside Options. In Proceedings of Congress on Evolutionary Computation, Vancouver, BC, Canada, pp. 2149–2156, 2006.Google Scholar
  13. [13]
    D. Cliff. Minimal-intelligence Agents for Bargaining Behaviors in Market-based Environments, Technical Report HPL-97-91 970811, Hewlett Packard Laboratories, USA, 1997.Google Scholar
  14. [14]
    D. K. Gode, S. Sunder. Allocative Efficiency of Markets with Zero-intelligence Traders: Market as a Partial Substitute for Individual Rationality. The Journal of Political Economy, vol. 101, no. 1, pp. 119–137, 1993.CrossRefGoogle Scholar
  15. [15]
    D. Cliff. Explorations in Evolutionary Design of Online Auction Market Mechanisms. Journal of Electronic Commerce Research and Applications, vol. 2, no. 2, pp. 162–175, 2003.CrossRefMathSciNetGoogle Scholar
  16. [16]
    A. Moore. Efficient Memory-based Learning for Robot Control, Ph. D. dissertation, University of Cambridge, UK, 1990.Google Scholar
  17. [17]
    R. Sutton and A. Barto. Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, USA, 1998.Google Scholar
  18. [18]
    S. Whiteson, P. Stone. Evolutionary Function Approximation for Reinforcement Learning. Journal of Machine Learning Research, vol. 7, pp. 877–917, 2006.MathSciNetGoogle Scholar
  19. [19]
    G. Tesauro. Temporal Difference Learning and TD-gammon. Communications of the ACM, vol. 38, no. 3, pp. 58–68, 1995.CrossRefGoogle Scholar
  20. [20]
    J. B. Pollack, A. D. Blair. Co-evolution in the Successful Learning of Backgammon Strategy. Machine Learning, vol. 32, no. 3, pp. 225–240, 1998.zbMATHCrossRefGoogle Scholar
  21. [21]
    G. Tesauro. Comments on “Co-Evolution in the Successful Learning of Backgammon Strategy”. Machine Learning, vol. 32, no. 3, pp. 241–243, 1998.zbMATHCrossRefGoogle Scholar
  22. [22]
    P. J. Darwen. Why Co-evolution Beats Temporal Difference Learning at Backgammon for a Linear Architecture, but not a Non-linear Architecture. In Proceedings of Congress on Evolutionary Computation, IEEE Press, vol. 2, pp. 1003–1010, 2001.CrossRefGoogle Scholar
  23. [23]
    T. P. Runarsson, S. M. Lucas. Co-evolution versus Selfplay Temporal Difference Learning for Acquiring Position Evaluation in Small-board Go. IEEE Transactions on Evolutionary Computation, vol. 9, no. 6, pp. 628–640, 2005.CrossRefGoogle Scholar
  24. [24]
    S. M. Lucas, T. P. Runarsson. Temporal Difference Learning versus Co-evolution for Acquiring Othello Position Evaluation. In Proceedings of IEEE Symposium on Computational Intelligence and Games, Reno/Lake Tahoe, USA, pp. 53–59, 2006.Google Scholar
  25. [25]
    C. Kotnik, J. Kalita. The Significance of Temporaldifference Learning in Self-play Training: TD-rummy versus EVO-rummy. In Proceedings of the International Conference on Machine Learning, Washington D.C., USA, pp. 369–375. 2003.Google Scholar
  26. [26]
    M. E. Taylor, S. Whiteson, P. Stone. Comparing Evolutionary and Temporal Difference Methods in a Reinforcement Learning Domain. In Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, Seattle, Washington, USA, pp. 1321–1328, 2006.Google Scholar
  27. [27]
    F. Gomez, J. Schmidhuber, R. Miikkulainen. Efficient Nonlinear Control through Neuroevolution. In Proceedings of the European Conference on Machine Learning, Lecture Notes in Computer Science, vol. 4212, pp. 654–662, 2006.Google Scholar
  28. [28]
    S. M. Lucas, J. Togelius. Point-to-point Car Racing: An Initial Study of Evolution versus Temporal Difference Learning. In Proceedings of IEEE Symposium on Computational Intelligence and Games, Westin Harbour Castle Toronto, Ontario Canada, pp. 260–267, 2007.CrossRefGoogle Scholar
  29. [29]
    K.-F. Lee, S. Mahajan. A Pattern Classification Approach to Evaluation Function Learning. Artificial Intelligence, vol. 36, no. 1, pp. 1–25, 1988.CrossRefGoogle Scholar
  30. [30]
    K.-F. Lee, S. Mahajan. The Development of a World Class Othello Program. Artificial Intelligence, vol. 43,no. 1, pp. 21–36, 1990.CrossRefGoogle Scholar
  31. [31]
    M. Buro. LOGISTELLO-A Strong Learning Othello Program. NEC Research Institute, Princeton, NJ, 1997, [Online], Available: mburo/ps/ Scholar
  32. [32]
    S. Y. Chong, M. K. Tan, J. D. White. Observing the Evolution of Neural Networks Learning to Play the Game of Othello. IEEE Transactions on Evolutionary Computation, vol. 9, no. 3, pp. 240–251, 2005.CrossRefGoogle Scholar
  33. [33]
    W. W. Bledsoe, I. Browning. Pattern Recognition and Reading by Machine. In Proceedings of the Eastern Joint Computer Conference, pp. 225–232. 1959.Google Scholar
  34. [34]
    J. Ullman. Experiments with the n-tuple Method of Pattern Recognition. IEEE Transactions on Computers, vol. 18, no. 12, pp. 1135–1137, 1969.CrossRefMathSciNetGoogle Scholar
  35. [35]
    R. Rohwer, M. Morciniec. A Theoretical and Experimental Account of n-tuple Classifier Performance. Neural Computation, vol. 8, no. 3, pp. 629–642, 1996.CrossRefGoogle Scholar
  36. [36]
    B. Chaperot, C. Fyfe. Improving Artificial Intelligence in a Motocross Game. In Proceedings of IEEE Symposium on Computational Intelligence and Games, Reno/Lake Tahoe, USA, pp. 181–186, 2006.CrossRefGoogle Scholar
  37. [37]
    D. Floreano, T. Kato, D. Marocco, E. Sauser. Coevolution of Active Vision and Feature Selection. Biological Cybernetics, vol. 90, no. 3, pp. 218–228, 2004.zbMATHCrossRefGoogle Scholar
  38. [38]
    S. Thrun, M. Montemerlo, H. Dahlkamp, D. Stavens, A. Aron, J. Diebel, P. Fong, J. Gale, M. Halpenny, G. Hoffmann, K. Lau, C. Oakley, M. Palatucci, V. Pratt, P. Stang, S. Strohband, C. Dupont, L.-E. Jendrossek, C. Koelen, C. Markey, C. Rummel, J. van Niekerk, E. Jensen, P. Alessandrini, G. Bradski, B. Davies, S. Ettinger, A. Kaehler, A. Nefian, and P. Mahoney. The Robot that Won the DARPA Grand Challenge. Journal of Field Robotics, vol. 23, no. 9, pp. 661–692, 2006.CrossRefGoogle Scholar
  39. [39]
    I. Tanev, M. Joachimczak, H. Hemmi, K. Shimohara. Evolution of the Driving Styles of Anticipatory Agent Remotely Operating a Scaled Model of Racing Car. In Proceedings of IEEE Congress on Evolutionary Computation, vol. 2, pp. 1891–1898, 2005.CrossRefGoogle Scholar

Copyright information

© Institute of Automation, Chinese Academy of Sciences 2008

Authors and Affiliations

  1. 1.Centre for Computational Intelligence, Department of Computing and Electronic SystemsUniversity of EssexColchesterUK

Personalised recommendations