Emergent Cooperation in RoboCup: A Review

  • Geoff Nitschke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4020)


This article presents a survey of prevalent research results pertaining to emergent cooperation in RoboCup soccer. Results reviewed maintain particular reference to research that uses biologically inspired design principles and concepts, such as emergence and evolution, as a means of attaining cooperative behavior. The core of this article argues that even though emergent cooperative behavior derived within RoboCup (and the larger field of multi-robot systems) is still in its infancy, it holds considerable future potential, as a problem solver in domains where systems comprising many interacting components must cooperatively solve a global task.


Genetic Programming Cooperative Behavior Artificial Evolution Genetic Programming Approach Game Scenario 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dugatkin, L.: N-Person games and the evolution of cooperation: A model based on predator inspection in fish. Journal of Theoretical Biology 142(1), 123–135 (1990)CrossRefMathSciNetGoogle Scholar
  2. 2.
    Akiyama, E., Kunihiko, K.: Evolution of Cooperation, Differentiation, Complexity, and Diversity in an Iterated Three-Person Game. Artificial Life 2(1), 293–304 (1995)CrossRefGoogle Scholar
  3. 3.
    Axelrod, R.: The Evolution of Cooperation. Basic Books, New York (1984)Google Scholar
  4. 4.
    Cao, Y., Fukunaga, A., Kahng, A.: Cooperative Mobile Robotics: Antecedents and Directions. Autonomous Robots 4, 7–27 (1997)CrossRefGoogle Scholar
  5. 5.
    Braitenberg, V.: Vehicles: Experiments in synthetic psychology. MIT Press, Cambridge, MA (1984)Google Scholar
  6. 6.
    Kube, C., Zhang, H.: Collective Robotics: From Social Insects to Robots. Adaptive Behavior 2(1), 189–218 (1993)CrossRefGoogle Scholar
  7. 7.
    Kube, C., Zhang, H.: Task Modeling in Collective Robotics. Autonomous Robots 4(1), 53–72 (1997)CrossRefGoogle Scholar
  8. 8.
    Bongard, J., Pfeifer, R.: Evolving Complete Agents Using Artificial Ontogeny. In: Hara, F., Pfeifer, R. (eds.) Morpho-functional Machines: The New Species (Designing Embodied Intelligence). Springer, Berlin (2003)Google Scholar
  9. 9.
    Harvey, I.: Artificial Evolution and Real Robots. In: Masanori, S. (ed.) Proceedings of International Symposium on Artificial Life and Robotics, pp. 18–20. Springer, Berlin (1997)Google Scholar
  10. 10.
    Mataric, M.: Designing emergent behaviors: From local interactions to collective intel-ligence. In: Meyer, J., Roitblat, H., Wilson, S. (eds.) From Animals to Animats 2: Second International Conference on the Simulation of Adaptive Behavior, pp. 432–441. MIT Press, Cambridge (1992)Google Scholar
  11. 11.
    Floreano, D., Nolfi, S.: Evolutionary Robotics. MIT Press, Cambridge (2000)Google Scholar
  12. 12.
    Di Caro, G., Dorigo, M.: AntNet: Distributed Stigmergic Control for Communications Networks. Journal of Artificial Intelligence Research 9, 317–365 (1998)MATHGoogle Scholar
  13. 13.
    Brooks, R., Flynn, A.: Fast, Cheap and Out of Control: A Robot Invasion of the Solar System. Journal of the British Interplanetary Society 1, 478–485 (1989)Google Scholar
  14. 14.
    Pfeifer, R., Scheier, C.: Understanding Intelligence. MIT Press, Cambridge (1999)Google Scholar
  15. 15.
    Veloso, M., Bowling, M., Stone, P.: The CMUnited-98 champion small-robot team. Advanced Robotics 13(1), 753–767 (2000)CrossRefGoogle Scholar
  16. 16.
    Kim, J.: Proceedings of the Micro-Robot World Cup Soccer Tournament. Elsevier Science, Taejon (1996)Google Scholar
  17. 17.
    Noda, I., Stone, P.: The RoboCup Soccer Server and CMUnited Clients: Imple-mented Infrastructure for MAS Research. Autonomous Agents and Multi-Agent Systems 7(1), 101–120 (2001)CrossRefGoogle Scholar
  18. 18.
    Jamzad, M., Foroughnassiraei, A., Chiniforooshan, E., Ghorbani, R., Kazemi, M., Chitsaz, H.R., Mobasser, F., Sadjad, S.B.S.: Middle Sized Soccer Robots: ARVAND. In: Veloso, M.M., Pagello, E., Kitano, H. (eds.) RoboCup 1999. LNCS (LNAI), vol. 1856, pp. 61–73. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  19. 19.
    Kitano, H., Fujita, M., Zrehen, S., Kageyama, K.: Sony Legged Robot for RoboCup Challenge. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 2605–2612. IEEE Press, Leuven (1998)Google Scholar
  20. 20.
    Kitano, H., Asada, M.: The RoboCup Humanoid Challenge as the Millennium Challenge for Advanced Robotics. Advanced Robotics 13(1), 723–736 (2000)CrossRefGoogle Scholar
  21. 21.
    Noda, I., Matsubara, H., Hiraki, K.: Learning of cooperative actions in multi-agent systems: a case study of pass and play in soccer. In: Sandip, S. (ed.) Adaptation, Co-evolution, and Learning in Multi-agent Systems: Papers from the 1996 AAAI Spring Symposium, pp. 63–67. AAAI Press, Boston (1996)Google Scholar
  22. 22.
    Stone, P., Veloso, M.: The CMUNITED 1997 Simulator Team. In: Kitano, H. (ed.) RoboCup 1997. LNCS, vol. 1395, pp. 389–397. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  23. 23.
    Stone, P., Veloso, M.: Task decomposition, dynamic role assignment, and low bandwidth communication for real time strategic teamwork. Artificial Intelligence 110(2), 241–273 (1999)MATHCrossRefGoogle Scholar
  24. 24.
    Stone, M., Veloso, M., Bowling, M.: Anticipation as a Key for Collaboration in a Team of Agents: A Case Study in Robotic Soccer. In: McKee, G., Schenker, P. (eds.) Proceedings of SPIE Sensor Fusion and Decentralized Control in Robotic Systems II, pp. 134–141. SPIE Press, Bellingham (1999)Google Scholar
  25. 25.
    Stone, P., Veloso, M.: Towards Collaborative and Adversarial Learning: A Case Study in Robotic Soccer. Evolution and learning in multi-agent systems 48(1), 83–104 (1998)Google Scholar
  26. 26.
    Noda, I., Matsubara, H., Hiraki, K., Frank, I.: Soccer Server: A Tool for Research on Multi-agent Systems. Applied Artificial Intelligence 12(1), 233–250 (1998)CrossRefGoogle Scholar
  27. 27.
    Whiteson, S., Kohl, N., Miikkulainen, R., Stone, P.: Evolving Keep away Soccer Players through Task Decomposition. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 356–368. Springer, Chicago (2003)Google Scholar
  28. 28.
    Di Pietro, A., While, L., Barone, L.: Learning In RoboCup Keep away Using Evolu-tionary Algorithms. In: Langdon, W., Cantupaz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., Honavar, V., Rudolph, G., Wegener, J., Bull, L., Potter, M., Schultz, A., Miller, J., Burke, E., Jonoska, N. (eds.) GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1065–1072. Morgan Kaufmann, New York (2002)Google Scholar
  29. 29.
    Hsu, W., Gustafson, S.: Layered Learning in Genetic Programming for a Cooperative Robot Soccer Problem. In: Miller, J., Tomassini, M., Lanzi, P., Ryan, C., Tetamanzi, A., Langdon, W. (eds.) Proceedings of the Fourth European Conference on Genetic Programming (pp, pp. 291–301. Springer, Como (2001)Google Scholar
  30. 30.
    Hsu, W., Gustafson, S.: Genetic Programming and Multi-Agent Layered Learning by Reinforcements. In: Langdon, W., Cantupaz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., Honavar, V., Rudolph, G., Wegener, J., Bull, L., Potter, M., Schultz, A., Miller, J., Burke, E., Jonoska, N. (eds.) GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 764–771. Morgan Kaufmann, New York (2002)Google Scholar
  31. 31.
    Noda, I., Matsubara, H., Hiraki, K.: Learning Cooperative Behavior in Multi-agent Environment: a case study of choice of play-plans in soccer. In: Foo, N., Goebel, R. (eds.) Proceedings of the Fourth Pacific Rim International Conference on Artificial Intelligence, pp. 570–579. Springer, Cairns (1996)Google Scholar
  32. 32.
    Stone, P., Veloso, M.: A Layered Approach to Learning Client Behaviors in the RoboCup Soccer Server. Applied Artificial Intelligence 12, 165–188 (1998)CrossRefGoogle Scholar
  33. 33.
    Luke, S., Hohn, C., Farris, J., Jackson, G., Hendler, J.: Co-evolving Soccer Softbot Team Coordination with Genetic Programming. In: Kitano, H. (ed.) RoboCup 1997. LNCS, vol. 1395, pp. 398–411. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  34. 34.
    Stone, P., Veloso, M.: Using Decision Tree Confidence Factors for Multi-agent Control. In: Kitano, H. (ed.) Proceedings of the Second International Conference on Autonomous Agents, pp. 110–116 (1998)Google Scholar
  35. 35.
    Veloso, M.M., Bowling, M., Achin, S., Han, K., Stone, P.: The cMUnited-98 champion small-robot team. In: Asada, M., Kitano, H. (eds.) RoboCup 1998. LNCS, vol. 1604, pp. 77–92. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  36. 36.
    Veloso, M.M., Uther, W.T.B.: The CMTrio-98 Sony Legged Robot Team. In: Asada, M., Kitano, H. (eds.) RoboCup 1998. LNCS, vol. 1604, pp. 491–497. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  37. 37.
    Kitano, H., Kuniyoshi, Y., Noda, I., Asada, M., Matsubara, H., Osawa, E.: RoboCup: A challenge problem for AI. AI Magazine 18(1), 73–85 (1997)Google Scholar
  38. 38.
    Rumelhart, D., Hinton, G., Williams, R.: Learning internal representations by error propagation. In: Rumelhart, D., McClelland, J. (eds.) Parallel Distributed Processing. Foundations, vol. 1, pp. 318–362. MIT Press, Cambridge (1986)Google Scholar
  39. 39.
    Mitchell, T.: Machine Learning. McGraw-Hill, Berkshire (1997)MATHGoogle Scholar
  40. 40.
    Bryant, B., Miikkulainen, R.: Neuro-evolution for Adaptive Teams. In: Proceedings of the 2003 Congress on Evolutionary Computation. IEEE Press, Piscataway (2003)Google Scholar
  41. 41.
    Koza, J.: Evolution of subsumption using genetic programming. In: Varela, F., Bourgine, P. (eds.) Proceedings of the First European Conference on Artificial Life, pp. 110–119. MIT Press, Cambridge, MA (1992)Google Scholar
  42. 42.
    Montana, D.: Strongly typed genetic programming. Evolutionary Computation 3(2), 199–230 (1995)CrossRefGoogle Scholar
  43. 43.
    Floreano, D., Nolfi, S.: God save the Red Queen! Competition in co-evolutionary robotics. In: Koza, J., Kalyanmoy, D., Dorigo, M., Fogel, D., Fogel, M., Garzon, M., Iba, H., Riolo, R. (eds.) Genetic Programming 1997: Proceedings of the Second Annual Conference. Morgan Kaufmann, San Francisco (1997)Google Scholar
  44. 44.
    Van Valen, L.: A new evolutionary law. Evolution Theory 1, 1–30 (1973)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Geoff Nitschke
    • 1
  1. 1.Computational Intelligence Group, Department of Computer Science, Faculty of SciencesAmsterdamThe Netherlands

Personalised recommendations