Cultural Evolution for Sequential Decision Tasks: Evolving Tic–Tac–Toe Players in Multi–agent Systems

  • Dara Curran
  • Colm O’Riordan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3102)


Sequential decision tasks represent a difficult class of problem where perfect solutions are often not available in advance. This paper presents a set of experiments involving populations of agents that evolve to play games of tic–tac–toe. The focus of the paper is to propose that cultural learning, i.e. the passing of information from one generation to the next by non–genetic means, is a better approach than population learning alone, i.e. the purely genetic evolution of agents. Population learning is implemented using genetic algorithms that evolve agents containing a neural network capable of playing games of tic–tac–toe. Cultural learning is introduced by allowing highly fit agents to teach the population, thus improving performance. We show via experimentation that agents employing cultural learning are better suited to solving a sequential decision task (in this case tic–tac–toe) than systems using population learning alone.


Neural Network Genetic Algorithm Cultural Evolution Simple Game Cultural Learning 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Moriarty, D., Miikkulainen, R.: Discovering complex othello strategies through evolutionary neural networks. Connection Science 7(3-4), 195–209 (1995)Google Scholar
  2. 2.
    Weaver, L., Bossomaier, T.: Evolution of neural networks to play the game of dots-and-boxes. In: Artificial Life V: Poster Presentations, pp. 43–50 (1996)Google Scholar
  3. 3.
    Chellapilla, K., Fogel, D.B.: Evolving neural networks to play checkers without relying on expert knowledge. In: IEEE Transactions on Neural Networks, vol. 10, pp. 1382–1391 (1999)Google Scholar
  4. 4.
    Richards, N., Moriarty, D., McQuesten, P., Miikkulainen, R.: Evolving neural networks to play Go. In: Proceedings of the 7th International Conference on Genetic Algorithms, East Lansing, MI (1997)Google Scholar
  5. 5.
    Baldwin, J.M.: A new factor in evolution. In: American Naturalist 30, pp. 441– 451 (1896)Google Scholar
  6. 6.
    Curran, D., O’Riordan, C.: On the design of an artificial life simulator. In: Palade, V., Howlett, R.J., Jain, L. (eds.) KES 2003. LNCS, vol. 2774, Springer, Heidelberg (2003)Google Scholar
  7. 7.
    Curran, D., O’Riordan, C.: Artificial life simulation using marker based encoding. In: Proceedings of the 2003 International Conference on Artificial Intelligence (IC-AI 2003), Las Vegas, Nevada, USA, vol. II, pp. 665–668 (2003)Google Scholar
  8. 8.
    Curran, D., O’Riordan, C.: Lifetime learning in multi-agent systems: Robustness in changing environments. In: Proceedings of the 14th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2003), Dublin, Ireland, pp. 46–50 (2003)Google Scholar
  9. 9.
    Hutchins, E., Hazlehurst, B.: How to invent a lexicon: The development of shared symbols in interaction. In: Gilbert, N., Conte, R. (eds.) Artificial Societies: The Computer Simulation of Social Life, pp. 157–189. UCL Press, London (1995)Google Scholar
  10. 10.
    Chomsky, N.: On the nature of language. In: Origins and evolution of language and speech, New York. Annals of the New York Academy of Science, vol. 280, pp. 46–57 (1976)Google Scholar
  11. 11.
    Yanco, H., Stein, L.: An adaptive communication protocol for cooperating mobile robots (1993)Google Scholar
  12. 12.
    Angelo, C., Parisi, D.: The emergence of a language in an evolving population of neural networks. Technical Report NSAL–96004, National Research Council, Rome (1996)Google Scholar
  13. 13.
    Spector, L., Luke, S.: Culture enhances the evolvability of cognition. In: Cognitive Science (CogSci) 1996 Conference Proceedings (1996)Google Scholar
  14. 14.
    Hutchins, E., Hazlehurst, B.: Learning in the cultural process. In: Langton, C., et al. (eds.) Artificial Life II, MIT Press, Cambridge (1991)Google Scholar
  15. 15.
    Cangelosi, A.: Evolution of communication using combination of grounded symbols in populations of neural networks. In: Proceedings of IJCNN 1999 International Joint Conference on Neural Networks, Washington, DC, vol. 6, pp. 4365–4368. IEEE Press, Los Alamitos (1999)CrossRefGoogle Scholar
  16. 16.
    MacLennan, B., Burghardt, G.: Synthetic ethology and the evolution of cooperative communication. In: Adaptive Behavior, vol. 2(2), pp. 161–188 (1993)Google Scholar
  17. 17.
    Billard, A., Hayes, G.: Learning to communicate through imitation in autonomous robots. In: 7th International Conference on Artificial Neural Networks, pp. 763–738 (1997)Google Scholar
  18. 18.
    Denaro, D., Parisi, D.: Cultural evolution in a population of neural networks. In: Marinaro, M., Tagliaferri, R. (eds.) Neural Nets Wirn 1996, pp. 100–111. Springer, New York (1996)Google Scholar
  19. 19.
    Curran, D., O’Riordan, C.: Learning in artificial life societies. In: Report number nuig-it-220202. Technical Report, Dept. of IT. NUI, Galway (2002)Google Scholar
  20. 20.
    Kitano, H.: Designing neural networks using genetic algorithm with graph generation system. In: Complex Systems, vol. 4, pp. 461–476 (1990)Google Scholar
  21. 21.
    Miller, G.F., Todd, P.M., Hedge, S.U.: Designing neural networks using genetic algorithms. In: Proceedings of the Third International Conference on Genetic Algorithms and Their Applications, pp. 379–384 (1989)Google Scholar
  22. 22.
    Angeline, P.J., Pollack, J.B.: Competitive environments evolve better solutions for complex tasks. In: Forrest, S. (ed.) Proceedings of the Fifth International Conference on Genetic Algorithms, San Francisco, CA, pp. 264–270. Morgan Kaufmann, San Francisco (1993)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Dara Curran
    • 1
  • Colm O’Riordan
    • 1
  1. 1.Dept. of Information TechnologyNational University of IrelandGalway

Personalised recommendations