Co-evolutionary Learning for Cognitive Computer Generated Entities

  • Xander Wilcke
  • Mark Hoogendoorn
  • Jan Joris Roessingh
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8482)


In this paper, an approach is advocated to use a hybrid approach towards learning behavior for computer generated entities (CGEs) in a serious gaming setting. Hereby, an agent equipped with cognitive model is used but this agent is enhanced with Machine Learning (ML) capabilities. This facilitates the agent to exhibit human like behavior but avoid an expert having to define all parameters explicitly. More in particular, the ML approach utilizes co-evolution as a learning paradigm. An evaluation in the domain of one-versus-one air combat shows promising results.


Situation Awareness Fitness Landscape Hill Climbing Baseline Algorithm Fighter Aircraft 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abt, C.A.: Serious games. Viking Press, New York (1970)Google Scholar
  2. 2.
    Anderson, J.R.: The architecture of cognition. Harvard University Press, Cambridge (1984)Google Scholar
  3. 3.
    Gini, M.L., Hoogendoorn, M., van Lambalgen, R.: Learning Belief Connections in a Model for Situation Awareness. In: Kinny, D., Hsu, J.Y.-j., Governatori, G., Ghose, A.K. (eds.) PRIMA 2011. LNCS (LNAI), vol. 7047, pp. 373–384. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  4. 4.
    Endsley, M.R.: Toward a theory of Situation Awareness in dynamic systems. Human Factors 37(1), 32–64 (1995)CrossRefGoogle Scholar
  5. 5.
    Floreano, D., Nolfi, S.: God Save the Red Queen! Competition in Co-Evolutionary Robotics. In: Koza, J.R., Deb, K., Dorigo, M., et al. (eds.) Proceedings of the Second Annual Conference Genetic Programming 1997, pp. 398–406. Morgan Kaufmann, San Francisco (1997)Google Scholar
  6. 6.
    Hillis, D.W.: Co-evolving parasites improve simulated evolution as an optimization procedure. In: Langton, C.G., Taylor, C., Farmer, J.D., et al. (eds.) Artificial Life II: SFI Studies in the Sciences of Complexity, vol. 10, pp. 313–324. Addison-Wesley, Redwood City (1991)Google Scholar
  7. 7.
    Hoogendoorn, M., Lambalgen, R.M., van, T.J.: Modeling Situation Awareness in Human-Like Agents using Mental Models. In: Walsh, T. (ed.) Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJCAI 2011, pp. 1697–1704 (2011)Google Scholar
  8. 8.
    Kassahun, Y., Sommer, G.: Efficient Reinforcement Learning Through Evolutionary Acquisition of Neural Topologies. In: Proceedings of the Thirteenth European Symposium on Artificial Neural Networks, Bruges, Belgium, pp. 259–266. D-Side Publications (2005)Google Scholar
  9. 9.
    Kieras, D.E., Meyer, D.E.: An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. In: Human-Computer Interaction, pp. 391-438 (1997)Google Scholar
  10. 10.
    Koopmanschap, R., Hoogendoorn, M., Roessingh, J.J.: Learning Parameters for a Cognitive Model on Situation Awareness. In: Ali, M., Bosse, T., Hindriks, K.V., Hoogendoorn, M., Jonker, C.M., Treur, J. (eds.) IEA/AIE 2013. LNCS, vol. 7906, pp. 22–32. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  11. 11.
    Laird, J., Rosenbloom, P., Newell, A.: Soar: An Architecture for General Intelligence. Artificial Intelligence 33, 1–64 (1987)CrossRefGoogle Scholar
  12. 12.
    Merk, R.J.: Making Enemies: Cognitive Modeling for Opponent Agents in Fighter Pilot Simulators. Ph.D. thesis, VU University Amsterdam, Amsterdam, the Netherlands (2013)Google Scholar
  13. 13.
    Michael, D., Chen, S.: Serious games: Games that educate, train and inform. Thomson Course Technology, Boston (2006)Google Scholar
  14. 14.
    Naveh, I., Sun, R.: A cognitively based simulation of academic science. Computational and Mathematical Organization Theory, 313–337 (2006)Google Scholar
  15. 15.
    Siebel, N., Bötel, B., Sommer, G.: Efficient Neural Network Pruning during Neuro-Evolution. Neural Networks, 2920–2929 (2009)Google Scholar
  16. 16.
    Smith, R.E., Dike, B.A., Mehra, R.K., et al.: Classier Systems in Combat: Two-sided Learning of Maneuvers for Advanced Fighter Aircraft. In: Computer Methods in Applied Mechanics and Engineering, vol. 186(2-4), pp. 421–437 (2000)Google Scholar
  17. 17.
    Stanley, K., Miikkulainen, R.: Evolving Neural Networks through Augmenting Topologies. Evolutionary Computation 10(2), 99–127 (2002)CrossRefGoogle Scholar
  18. 18.
    Yao, X.: Evolving Artificial Neural Networks. IEEE 87(9), 1423–1447 (1999)CrossRefGoogle Scholar
  19. 19.
    Yao, X., Liu, Y.: A New Evolutionary System for Evolving Artificial Neural Networks. IEEE Transactions on Neural Networks 8(3), 694–713 (1997)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Xander Wilcke
    • 1
    • 2
  • Mark Hoogendoorn
    • 2
  • Jan Joris Roessingh
    • 1
  1. 1.Department of Training, Simulations, and Operator PerformanceNational Aerospace LaboratoryAmsterdamThe Netherlands
  2. 2.Department of Computer ScienceVU University AmsterdamAmsterdamThe Netherlands

Personalised recommendations