Ambulance Decision Support Using Evolutionary Reinforcement Learning in Robocup Rescue Simulation League

  • Ivette C. Martínez
  • David Ojeda
  • Ezequiel A. Zamora
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4434)


We present a complete design of agents for the RoboCup Rescue Simulation problem that uses an evolutionary reinforcement learning mechanism called XCS, a version of Holland’s Genetic Classifiers Systems, to decide the number of ambulances required to rescue a buried civilian. We also analyze the problems implied by the rescue simulation and present solutions for every identified sub-problem using multi-agent cooperation and coordination built over a subsumption architecture. Our agents’ classifier systems were trained in different disaster situations. Trained agents outperformed untrained agents and most participants of the 2004 RoboCup Rescue Simulation League competition. This system managed to extract general rules that could be applied on new disaster situations, with a computational cost of a reactive rule system.


Alive Agent Disaster Situation Evolutionary Reinforcement System Genetic Program Agent Team 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Brooks, R.: How to build complete creatures rather than isolated cognitive simulators. Architectures for Intelligence, 225–240 (1991)Google Scholar
  2. 2.
    Holland, J.H.: Escaping Brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems. In: Mitchell, Michalski, Carbonell (eds.) Machine Learning: an artificial intelligence approach, Morgan Kaufman, San Francisco (1986)Google Scholar
  3. 3.
    Littman, K.L.P., Moore, A.P.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 237–285 (1996)Google Scholar
  4. 4.
    Kleiner, A., Brenner, M., Bräuer, T., Dornhege, C., Göbelbecker, M., Luber, M., Prediger, J., Stückler, J.: ResQ Freiburg: Team Description Paper and Evaluation. RoboCupRescue simulation league (2004)Google Scholar
  5. 5.
    Moriarty, D.E., Schultz, A.C., Grefenstette, J.J.: Evolutionary Algorithms for Reinforcement Learning. Journal of Artificial Intelligence Research 11, 199–229 (1999)MathSciNetGoogle Scholar
  6. 6.
    Paquet, S., Bernier, N., Chaib-draa, B.: DAMAS-Rescue Description Paper. RoboCupRescue simulation league (2004)Google Scholar
  7. 7.
    RoboCup Rescue Commitee: Robocup 2004 Rescue Simulation League Official Home Page (2004),
  8. 8.
    Silva, P., Coelho, H.: The 5Rings Team Report. RoboCupRescue simulation league (2004)Google Scholar
  9. 9.
    Tadokoro, S., Kitano, H., Takahashi, T., Noda, I., Matsubara, H., Shinjoh, A., Koto, T., Takeuchi, I., Takahashi, H., Matsuno, F., Hatayama, M., Nobe, J., Shimada, S.: The RoboCup-Rescue Project: A Robotic Approach to the Disaster Mitigation Problem. ICRA (2000),
  10. 10.
    Wilson, S.: Classifier Fitness Based on Accuracy. Evolutionary Computation Journal, 149–175 (1995)Google Scholar
  11. 11.
    Wilson, S.: Generalization in the XCS Classifier System Genetic Programming 1998. In: Proceedings of the Third Annual Conference, pp. 665–674. Morgan Kaufmann, San Francisco (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Ivette C. Martínez
    • 1
  • David Ojeda
    • 1
  • Ezequiel A. Zamora
    • 1
  1. 1.Grupo de Inteligencia Artificial, Universidad Simón Bolívar, Caracas 1080-AVenezuela

Personalised recommendations