Agent Learning Instead of Behavior Implementation for Simulations – A Case Study Using Classifier Systems

  • Franziska Klügl
  • Reinhard Hatko
  • Martin V. Butz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5244)


Although multi-agent simulations are an intuitive way of conceptualizing systems that consist of autonomous actors, a major problem is the actual design of the agent behavior. In this contribution, we examine the potential of using agent-based learning for implementing the agent behavior. We enhanced SeSAm, a platform for agent-based simulation, by replacing the usual rule-based agent architecture by XCS, a well-known learning classifier system (LCS). The resulting model is tested using a simple evacuation scenario. The results show that on the one hand side plausible agent behavior could be learned. On the other hand side, though, the results are quite brittle concerning the frame of environmental feedback, perception and action modeling.


Reinforcement Learning Multiagent System Agent Behavior Perception Category Feedback Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Holland, J.H., Reitman, J.S.: Cognitive systems based on adaptive algorithms. In: Waterman, D.A., Hayes-Roth, F. (eds.) Pattern directed inference systems, pp. 313–329. Academic Press, New York (1978)Google Scholar
  2. 2.
    Butz, M.V.: Combining gradient-basedwith evolutionary online learning: An introduction to learning classifier systems. In: 7th International Conference on Hybrid Intelligent Systems HIS 2007, pp. 12–17 (2007)Google Scholar
  3. 3.
    Klügl, F., Rindsfüser, G.: Large-scale agent-based pedestrian simulation. In: Petta, P., Müller, J.P., Klusch, M., Georgeff, M. (eds.) MATES 2007. LNCS (LNAI), vol. 4687, pp. 145–156. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Wilson, S.W.: Classifier fitness based on accuracy. Evolutionary Computation 3(2), 149–175 (1995)CrossRefGoogle Scholar
  5. 5.
    Weiss, G.: Adaptation and learning in multiagent systems: Some remarks and a bibliography. In: Weiss, G., Sen, S. (eds.) Adaption and learning in multi-agent systems. Springer, Heidelberg (1996)Google Scholar
  6. 6.
    Klügl, F., Tuys, K., Sen, S. (eds.): ALAMAS&ALAg - Adaptive and Learning Agents and Multiagent Systems (workshop at AAMAS 2008) (2008)Google Scholar
  7. 7.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning. MIT Press, Cambridge (1998)Google Scholar
  8. 8.
    Nowe, A., Verbeeck, K., Peeters, M.: Learning automata as a basis for multi agent reinforcement learning. In: Tuyls, K., ’t Hoen, P.J., Verbeeck, K., Sen, S. (eds.) LAMAS 2005. LNCS (LNAI), vol. 3898, pp. 71–85. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Oechslein, C., Hörnlein, A., Klügl, F.: Evolutionary optimization of societies in simulated multi-agent systems. In: Jonker, C., Letia, A., Lindemann, G., Uthmann, T. (eds.) MASHO Workshop at ECAI 2000, Humboldt-Universität, Berlin, vol. 149 (2000)Google Scholar
  10. 10.
    Smith, D.C., Cypher, A., Spohrer, J.: Kidsim: Programming agents without a programming language. Communications of the ACM 37(7), 54–67 (1994)CrossRefGoogle Scholar
  11. 11.
    Adami, C.: Introduction to Artificial Life. Springer, Heidelberg (1998)zbMATHGoogle Scholar
  12. 12.
    Grefenstette, J.J.: The Evolution of Strategies for Multi-agent Environments. Adaptive Behavior 1, 65–89 (1992)CrossRefGoogle Scholar
  13. 13.
    Collins, R.J., Jefferson, D.R.: Antfarm: Towards simulated evolution. In: Langton, C.G., Taylor, C., Farmer, J.D., Rasmussen, S. (eds.) Artificial Life II, pp. 579–601. Addison-Wesley, Redwood City (1992)Google Scholar
  14. 14.
    Denzinger, J., Fuchs, M.: Experiments in learning prototypical situations for variants of the pursuit game. In: Proc. of Int. Conf. on Multi-Agent Systems, 1996, Kyoto 1996, pp. 48–55 (1996)Google Scholar
  15. 15.
    Guessoum, Z., Rejeb, L., Durand, R.: Using adaptive multi-agent systems to simulate economic models. In: AAMAS 2004, pp. 68–75. IEEE Computer Society, Los Alamitos (2004)Google Scholar
  16. 16.
    Lanzi, P.L.: Learning classifier systems from a reinforcement learning perspective. Soft Computing: A Fusion of Foundations, Methodologies and Applications 6, 162–170 (2002)zbMATHCrossRefGoogle Scholar
  17. 17.
    Holland, J.H.: Adaptation in Natural and Artificial Systems, 2nd edn. (1992). University of Michigan Press (1975)Google Scholar
  18. 18.
    Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading (1989)zbMATHGoogle Scholar
  19. 19.
    Orriols-Puig, A., Bernadó-Mansilla, E.: Bounding XCS’s parameters for unbalanced datasets. In: GECCO 2006: Genetic and Evolutionary Computation Conference, pp. 1561–1568 (2006)Google Scholar
  20. 20.
    Butz, M.V.: Rule-Based Evolutionary Online Learning Systems: A Principled Approach to LCS Analysis and Design. Springer, Berlin (2006)zbMATHGoogle Scholar
  21. 21.
    Butz, M.V.: XCSJava 1.0: An implementation of the XCS classifier system in Java. IlliGAL report 2000027, Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Franziska Klügl
    • 1
  • Reinhard Hatko
    • 1
  • Martin V. Butz
    • 2
  1. 1.Dep. of Artificial Intelligence and Applied Computer ScienceUniversity of WürzburgWürzburgGermany
  2. 2.Dep. of Psychology, Cognitive Psychology IIIUniversity of WürzburgWürzburgGermany

Personalised recommendations