Layered Learning and Flexible Teamwork in RoboCup Simulation Agents

  • Peter Stone
  • Manuela Veloso
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1856)


RoboCup was introduced as a challenge area at IJCAI-97. We have been actively pursuing research in this area and have participated in the RoboCup competitions, winning the RoboCup-98 and RoboCup-99 simulator competitions. In this paper, we report on the main technical issues that we encountered and addressed in direct response to the learning and teamwork challenges stated in the IJCAI-97 challenge paper. We describe “layered learning” in which off-line and online, individual and collaborative, learned robotic soccer behaviors are combined hierarchically. We achieve effective teamwork through a team member agent architecture that encompasses a “flexible teamwork structure.” Agents are capable of decomposing the task space into flexible roles and can switch roles while acting. We report detailed empirical results verifying the effectiveness of the learned behaviors and the components of the team member agent architecture.


Multiagent System Pass Evaluation Task Space External Behavior Internal Behavior 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Minoru Asada and Hiroaki Kitano, editors. RoboCup-98: Robot Soccer World Cup II. Lecture Notes in Artificial Intelligence 1604. Springer Verlag, Berlin, 1999.Google Scholar
  2. 2.
    Minoru Asada, Shoichi Noda, Sukoya Tawaratumida, and Koh Hosoda. Purposive behavior acquisition for a real robot by vision-based reinforcement learning. Machine Learning, 23:279–303, 1996.Google Scholar
  3. 3.
    Hiroaki Kitano, editor. RoboCup-97: Robot Soccer World Cup I. Springer Verlag, Berlin, 1998.Google Scholar
  4. 4.
    Hiroaki Kitano, Milind Tambe, Peter Stone, Manuela Veloso, Silvia Coradeschi, Eiichi Osawa, Hitoshi Matsubara, Itsuki Noda, and Minoru Asada. The RoboCup synthetic agent challenge 97. In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, pages 24–29, San Francisco, CA, 1997. Morgan Kaufmann.Google Scholar
  5. 5.
    Itsuki Noda, Hitoshi Matsubara, Kazuo Hiraki, and Ian Frank. Soccer server: A tool for research on multiagent systems. Applied Artificial Intelligence, 12:233–250, 1998.CrossRefGoogle Scholar
  6. 6.
    J. Ross Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA, 1993.Google Scholar
  7. 7.
    Peter Stone. Layered Learning in Multi-Agent Systems. PhD thesis, Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, December 1998. Available as technical report CMU-CS-98-187.Google Scholar
  8. 8.
    Peter Stone and Manuela Veloso. A layered approach to learning client behaviors in the RoboCup soccer server. Applied Artificial Intelligence, 12:165–188, 1998.CrossRefGoogle Scholar
  9. 9.
    Peter Stone and Manuela Veloso. Task decomposition, dynamic role assignment, and low-bandwidth communication for real-time strategic teamwork. Artificial Intelligence, 110(2):241–273, June 1999.zbMATHCrossRefGoogle Scholar
  10. 10.
    Peter Stone and Manuela Veloso. Team partitioned, opaque transition reinforcement learning. In Proceedings of the Second International Conference on Autonomous Agents, pages 206–212. ACM Press, May 1999.Google Scholar
  11. 11.
    Milind Tambe, Jafar Adibi, Yaser Al-Onaizan, Ali Erdem and Gal A. Kaminka, Stacy C. Marsela, Ion Muslea, and Marcelo Tallis. Using an explicit model of teamwork in RoboCup-97. In Hiroaki Kitano, editor, RoboCup-97: Robot Soccer World Cup I, pages 123–131. Springer Verlag, Berlin, 1998.Google Scholar
  12. 12.
    Christopher J. C. H. Watkins. Learning from Delayed Rewards. PhD thesis, King’s College, Cambridge, UK, 1989.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Peter Stone
    • 1
  • Manuela Veloso
    • 2
  1. 1.AT&T Labs — ResearchFlorham Park
  2. 2.Computer Science DepartmentCarnegie Mellon UniversityPittsburgh

Personalised recommendations