Layered Learning and Flexible Teamwork in RoboCup Simulation Agents
RoboCup was introduced as a challenge area at IJCAI-97. We have been actively pursuing research in this area and have participated in the RoboCup competitions, winning the RoboCup-98 and RoboCup-99 simulator competitions. In this paper, we report on the main technical issues that we encountered and addressed in direct response to the learning and teamwork challenges stated in the IJCAI-97 challenge paper. We describe “layered learning” in which off-line and online, individual and collaborative, learned robotic soccer behaviors are combined hierarchically. We achieve effective teamwork through a team member agent architecture that encompasses a “flexible teamwork structure.” Agents are capable of decomposing the task space into flexible roles and can switch roles while acting. We report detailed empirical results verifying the effectiveness of the learned behaviors and the components of the team member agent architecture.
KeywordsMultiagent System Pass Evaluation Task Space External Behavior Internal Behavior
Unable to display preview. Download preview PDF.
- 1.Minoru Asada and Hiroaki Kitano, editors. RoboCup-98: Robot Soccer World Cup II. Lecture Notes in Artificial Intelligence 1604. Springer Verlag, Berlin, 1999.Google Scholar
- 2.Minoru Asada, Shoichi Noda, Sukoya Tawaratumida, and Koh Hosoda. Purposive behavior acquisition for a real robot by vision-based reinforcement learning. Machine Learning, 23:279–303, 1996.Google Scholar
- 3.Hiroaki Kitano, editor. RoboCup-97: Robot Soccer World Cup I. Springer Verlag, Berlin, 1998.Google Scholar
- 4.Hiroaki Kitano, Milind Tambe, Peter Stone, Manuela Veloso, Silvia Coradeschi, Eiichi Osawa, Hitoshi Matsubara, Itsuki Noda, and Minoru Asada. The RoboCup synthetic agent challenge 97. In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, pages 24–29, San Francisco, CA, 1997. Morgan Kaufmann.Google Scholar
- 6.J. Ross Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA, 1993.Google Scholar
- 7.Peter Stone. Layered Learning in Multi-Agent Systems. PhD thesis, Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, December 1998. Available as technical report CMU-CS-98-187.Google Scholar
- 10.Peter Stone and Manuela Veloso. Team partitioned, opaque transition reinforcement learning. In Proceedings of the Second International Conference on Autonomous Agents, pages 206–212. ACM Press, May 1999.Google Scholar
- 11.Milind Tambe, Jafar Adibi, Yaser Al-Onaizan, Ali Erdem and Gal A. Kaminka, Stacy C. Marsela, Ion Muslea, and Marcelo Tallis. Using an explicit model of teamwork in RoboCup-97. In Hiroaki Kitano, editor, RoboCup-97: Robot Soccer World Cup I, pages 123–131. Springer Verlag, Berlin, 1998.Google Scholar
- 12.Christopher J. C. H. Watkins. Learning from Delayed Rewards. PhD thesis, King’s College, Cambridge, UK, 1989.Google Scholar