Skip to main content
SpringerLink
Log in

Evolving team behaviors with specialization

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

This article evaluates Collective Neuro-Evolution (CONE), a cooperative co-evolutionary method for solving collective behavior tasks and increasing task performance via facilitating behavioral specialization in agent teams. Specialization is used as a problem solving mechanism, and its emergence is guided and regulated by CONE. CONE is comparatively evaluated with related methods in a simulated evolutionary robotics pursuit-evasion task. This task required multiple pursuer robots to cooperatively capture evader robots. Results indicate that CONE is appropriate for evolving specialized behaviors. The interaction of specialized behaviors produces behavioral heterogeneity in teams and collective prey capture behaviors that yield significantly higher performances compared to related methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. Herein, referred to as burst mutation.

  2. Spider-fly is from related research of Nolfi and Floreano [81], where evolved predator behaviors exhibited spider like predatory behavior (when capturing a fly).

References

  1. H. Abdel-Rahman, When do cities specialize in production. Reg. Sci. Urban. Econ. 26(1), 1–22 (2001)

    Article  Google Scholar 

  2. A. Agogino, K. Tumer, Efficient evaluation functions for multi-rover systems, in Proceedings of the Genetic and Evolutionary Computation Conference (Springer, New York, 2004), pp. 1–12

  3. K. Baev, Biological Neural Networks. (Birkuser, Berlin, 1997)

    Google Scholar 

  4. T. Balch. Behavioral Diversity in Learning Robot Teams. PhD Thesis. College of Computing, Georgia Institute of Technology, Altanta, USA (1998)

  5. T. Balch, Taxonomies of multi-robot task and reward, in Robot teams: From diversity to polymorphism (A K Peters, Natick, USA, 2002), pp. 23–35

  6. G. Baldassarre, S. Nolfi, D. Parisi, Evolving mobile robots able to display collective behavior. Artif. Life 9(1), 255–267 (2003)

    Article  Google Scholar 

  7. M. Benda, V. Jagannathan, R. Dodhiawalla, On optimal cooperation of knowledge sources (Tech. Rep. BCS-G2010-28) (Boeing AI Center, Boeing Computer Services, Bellevue, USA, 1986)

  8. J. Blumenthal, G. Parker, Co-evolving team capture strategies for dissimilar robots, in Proceedings of the Artificial Multi-Agent Learning Symposium (AAAI Press, Arlinton, Virginia, 2004), pp. 15–23

  9. J. Blumenthal, G. Parker, Competing sample sizes for the co-evolution of heterogeneous agents, in Proceedings of the International Conference on Intelligent Robots and Systems (IEEE, Sendai, Japan, 2004), pp. 1438–1443

  10. J. Blumenthal, G. Parker, Punctuated anytime learning for evolving multi-agent capture strategies, in Proceedings of the Congress on Evolutionary Computation (IEEE Press, Portland, USA, 2004), pp. 1820–1827

  11. E. Bonabeau, M. Dorigo, G. Theraulaz, Swarm Intelligence: From Natural to Artificial Systems. (Oxford University Press, Oxford, 1998)

    Google Scholar 

  12. E. Bonabeau, A. Sobkowski, G. Theraulaz, J. Deneubourg, Adaptive task allocation inspired by a model of division of labour in social insects, in Bio-Computing and Emergent Computation (World Scientific, Singapore, 1997), pp. 36–45

  13. E. Bonabeau, G. Theraulaz, J. Deneubourg, Quantitative study of the fixed threshold model for the regulation of division of labour in insect societies. Proc. R. Soc. Lond. B 263(1), 1565–1569 (1996)

    Article  Google Scholar 

  14. B. Bryant, R. Miikkulainen, Neuro-evolution for adaptive teams, in Proceedings of the Congress on Evolutionary Computation (IEEE Press, Canberra, 2003), pp. 2194–2201

  15. L. Bui, J. Branke, H. Abbass, Diversity as a selection pressure in dynamic environments, in Proceedings of the Conference on Genetic and Evolutionary Computation (ACM, Washington D.C, 2005), pp. 1557–1558

  16. L. Bull, J. Holland, Evolutionary computing in multi-agent environments: Eusociality, in Proceedings of the Second Annual Conference on Genetic Programming (IEEE Press, San Francisco, 1997), pp. 347–352

  17. N. Calderone, R. Page, Genotypic variability in age polyethism and task specialization in the honey bee. Behav. Ecol. Sociobiol. 22(1), 17–25 (1988)

    Article  Google Scholar 

  18. K. Chellapilla, D. Fogel, Evolving neural networks to play checkers without expert knowledge. IEEE Trans. Neural Netw. 10(16), 1382–1391 (1999)

    Article  Google Scholar 

  19. C. Clarke, P. Sheppard, Interactions between major genes and polygenes in the determination of the mimetic patterns of papilio dardanus. Evolution 17, 404–413 (1963)

    Article  Google Scholar 

  20. A. Conradie, R. Miikkulainen, C. Aldrich, Adaptive control utilising neural swarming, in Proceedings of the Genetic and Evolutionary Computation Conference (2002)

  21. A. Conradie, R. Miikkulainen, C. Aldrich, Intelligent process control utilizing symbiotic memetic neuro-evolution, in Proceedings of the Congress on Evolutionary Computation. (IEEE Press, Honolulu, USA, 2002)

  22. E. De Jong, R. Watson, J. Pollack, Reducing bloat and promoting diversity using multi-objective methods, in Proceedings of the Congress on Genetic and Evolutionary Computation (ACM, San Francisco, 2001), pp. 11–18

  23. P. Domenici, J. Blagburn, J. Bacon, Animal escapology II: escape trajectory case studies. J. Exp. Biol. 214(15), 2474–2494 (2011)

    Article  Google Scholar 

  24. S. Doncieux, J. Mouret, Behavioral diversity measures for evolutionary robotics, in Proceedings of the IEEE Congress on Evolutionary Computation (Springer, Barcelona, Spain, 2010), pp. 1303–1310

  25. C. Dunnett, A multiple comparisons procedure for comparing several treatments with a control. J. Am. Stat. As. 50, 294–299 (1995)

    Google Scholar 

  26. A. Eiben, S. Smit, Parameter tuning for configuring and analyzing evolutionary algorithms. Swarm Evol. Comput. 1(1), 19–31 (2011)

    Article  Google Scholar 

  27. A. Eiben, J. Smith, Introduction to Evolutionary Computing. (Springer, Berlin, 2003)

    MATH  Google Scholar 

  28. J. Elman, Finding structure in time. Cogn. Sci. 14(1), 179–211 (1990)

    Google Scholar 

  29. J. Fewell, S. Bertram, Division of labor in a dynamic environment: response by honeybees (apis mellifera) to graded changes in colony pollen stores. Behav. Ecol. Sociobiol. 46, 171–179 (1999)

    Article  Google Scholar 

  30. B. Flannery, S. Teukolsky, W. Vetterling, Numerical Recipes. (Cambridge University Press, Cambridge, 1986)

    Google Scholar 

  31. D. Floreano, P. Dürr, C. Mattiussi, Neuroevolution: from architectures to learning. Evol. Intell. 1(1), 47–62 (2008)

    Article  Google Scholar 

  32. N. Garcia-Pedrajas, C. Hervas-Martnez, D. Ortiz-Boyer, Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Trans. Evol. Comput. 9(3), 271–302 (2005)

    Article  Google Scholar 

  33. J. Gautrais, G. Theraulaz, J. Deneubourg, C. Anderson, Emergent polyethism as a consequence of increased colony size in insect societies. J. Theor. Biol. 215(1), 363–373 (2002)

    Article  Google Scholar 

  34. H. Ghezelayagh, K. Lee, Intelligent predictive control of a power plant with evolutionary programming optimizer and neuro-fuzzy identifier, in Proceedings of the Congress on Evolutionary Computation (IEEE Press, Honolulu, USA, 2002), pp. 1308–1313

  35. D. Goldberg, Simple genetic algorithms and the minimal, deceptive problem, in Genetic algorithms and simulated annealing (Morgan Kaufman, Berlin, Germany, 1987), pp. 74–88

  36. F. Gomez, Robust Non-Linear Control Through Neuroevolution. PhD thesis. (Department of Computer Sciences, The University of Texas, Austin, 2003)

  37. F. Gomez, R. Miikkulainen, Incremental evolution of complex general behavior. Adapt. Behav. 5(1), 317–342 (1997)

    Article  Google Scholar 

  38. B. Grant, P. Grant, Mimicry and warning color at the boundary between races and species, in ed by D. Howard, S. Berlocher, Endless Forms: Species and Speciation. (Oxford University Press, Oxford, 1998), pp. 404–422

  39. W. Hamilton, The genetical evolution of social behavior. J. Theor. Biol. 7(1), 1–16 (1964)

    Article  Google Scholar 

  40. D. Hawthorne, Genetic linkage of ecological specialization and reproductive isolation in pea aphids. Nature 412(1), 904–907 (2001)

    Article  Google Scholar 

  41. S. Haykin, Neural Networks: A Comprehensive Foundation. (Prentice Hall, Ontario, 1995)

    Google Scholar 

  42. T. Haynes, S. Sen, Evolving cooperation strategies, in Proceedings of the First International Conference on Multi-Agent Systems (MIT Press, Cambridge, 1995), pp. 450–459

  43. T. Haynes, S. Sen, Co-adaptation in a team. Int. J. Comput. Intell. Org. 1(4), 1–20 (1996)

    Google Scholar 

  44. T. Haynes, S. Sen, Evolving behavioral strategies in predators and prey, in Adaptation and Learning in Multi-Agent Systems: Lecture Notes in Computer Science (Springer, Berlin, 1996), pp. 113–126

  45. T. Haynes, S. Sen, Crossover operators for evolving a team, in Proceedings of Genetic Programming: Second Annual Conference (Morgan Kaufmann, San Francisco, 1997), pp. 162–167

  46. A. Hedenström, M. Rosén, Predator versus prey: on aerial hunting and escape strategies in birds. Behav. Ecol. 12(2), 150–156 (2001)

    Article  Google Scholar 

  47. J. Hertz, A. Krogh, R. Palmer, Introduction to the Theory of Neural Computation. (Addison-Wesley, Redwood City, 1991)

    Google Scholar 

  48. N. Keerativuttitumrong, N. Chaiyaratana, V. Varavithya, Multi-objective co-operative co-evolutionary genetic algorithm, in Proceedings of Parallel Problem Solving from Nature (Springer, Granada, 2002), pp. 288–297

  49. S. Kernbach, L. Ricotti, J. Liedke, P. Corradi, M. Rothermel, Study of macroscopic morphological features of symbiotic robotic organisms, in Proceedings of the International Conference on Intelligent Robots and Systems (Workshop on Self-Reconfigurable Robots) (IEEE, Nice, 2008), pp. 18–25

  50. J. Koza, Evolution of subsumption using genetic programming, in Proceedings of the European Conference on Artificial Life (MIT Press, Cambridge, 1992), pp. 110–119

  51. J. Lehman, K. Stanley, Abandoning objectives: evolution through the search for novelty alone. Evol. Comput. 19(2), 189–223 (1994)

    Article  Google Scholar 

  52. L. Lehmann, L. Keller, The evolution of cooperation and altruism a general framework and a classification of models. J. Theor. Biol. 19(1), 1365–1376 (2006)

    Google Scholar 

  53. L. Li, A. Martinoli, Y. Mostafa, Emergent specialization in swarm systems, in Lecture notes in computer science: Vol. 2412. Intelligent data engineering and automated learning (Springer, Berlin, 2002), pp. 261–266

  54. L. Li, A. Martinoli, A. Yaser, Learning and measuring specialization in collaborative swarm systems. Adapt. Behav. 12(3), 199–212 (2004)

    Article  Google Scholar 

  55. S. Luke, Genetic programming produced competitive soccer softbot teams for robocup 97, in Proceedings of the Third Annual Genetic Programming Conference (Morgan Kaufmann, San Fransisco, 1998), pp. 204–222

  56. S. Luke, C. Hohn, J. Farris, G. Jackson, J. Hendler, Co-evolving soccer softbot team coordination with genetic programming, in RoboCup-97: Robot Soccer World Cup I (Springer, Berlin, 1998), pp. 398–411

  57. S. Luke, L. Spector, Evolving teamwork and coordination with genetic programming, in Proceedings of the International Conference on Genetic Programming (MIT Press, Stanford, USA, 1996), pp. 150–156

  58. S. Mahfoud, Handbook of Evolutionary Computation. (Taylor and Francis, Amsterdam, 1997)

    Google Scholar 

  59. S.W. Mahfoud, Niching Methods for Genetic Algorithms. Ph. D. Dissertation. (Department of Computer Science, University of Illinois, Urbana, 1995)

  60. J. Mallet, A species definition for the modern synthesis. Trends Ecol. Evol. 10, 294–299 (1995)

    Article  Google Scholar 

  61. J. Mallet, W. McMillan, C. Jiggins, Mimicry and warning color at the boundary between races and species, In: D. Howard, S. Berlocher (eds) Endless Forms: Species and Speciation, (Oxford University Press, New York, 1998) pp. 390–403.

    Google Scholar 

  62. M. Mataric, Reward functions for accelerated learning, in Proceedings of the Eleventh International Conference on Machine Learning (Morgan Kaufmann, San Francisco, USA, 2002), pp. 181–189

  63. E. Mayr, Animal species and evolution. (Harvard University Press, Cambridge , 1963)

    Google Scholar 

  64. D. Merkle, M. Middendorf, Dynamic polyethism and competition for tasks in threshold reinforcement models of social insects. Adapt. Behav. 12, 251–262 (2004)

    Article  Google Scholar 

  65. R. Miikkulainen, Neuroevolution, in Encyclopedia of Machine Learning (Springer, New York, 2010)

  66. F. Mondada, E. Franzi, P. Ienne, Mobile robot miniaturization: A tool for investigation in control algorithms, in Proceedings of Third International Symposium on Experimental Robotics (IEEE Press, Kyoto, 1993), pp. 501–513

  67. H. Moriguchi, S. Honiden, Sustaining behavioral diversity in neat, in Proceedings of the Conference on Genetic and Evolutionary Computation (ACM Press, Portland, 2010), pp. 611–618

  68. J. Mouret, Novelty-based multiobjectivization. In: S. Doncieux, N. Bredeche, J. Mouret (eds) New Horizons in Evolutionary Robotics: Extended Contributions from the 2009 EvoDeRob Workshop, (Springer, Berlin, 2011) pp. 139–154.

    Google Scholar 

  69. J. Mouret, S. Doncieux, Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity, in Proceedings of the IEEE Congress on Evolutionary Computation (ACM Press, Trondheim, 2009), pp. 1161–1168

  70. J. Mouret, S. Doncieux, Using behavioral exploration objectives to solve deceptive problems in neuro-evolution, in Proceedings of the Conference on Genetic and Evolutionary Computation (IEEE Press, Montreal, 2009), pp. 627–634

  71. J. Mouret, S. Doncieux, Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evolutionary Computation To appear (2011)

  72. A. Murciano, J. Millan, J. Zamora, Specialization in multi-agent systems through learning. Biol. Cybern. 76(1), 375 382 (1997)

    Google Scholar 

  73. Y. Ng, X. Yang, Specialization, information, and growth: A sequential equilibrium analysis. Rev. Dev. Econ. 1(1), 257–274 (1997)

    Article  Google Scholar 

  74. G. Nitschke, Designing emergent cooperation: a pursuit-evasion game case study. Artif. Life Robotics 9(4), 222–233 (2005)

    Article  Google Scholar 

  75. G. Nitschke, M. Schut, A. Eiben, Emergent specialization in biologically inspired collective behavior systems, in Intelligent Complex Adaptive Systems (IGI, New York, 2007), pp. 100–140

  76. G. Nitschke, M. Schut, A. Eiben, Collective neuro-evolution for evolving specialized sensor resolutions in a multi-rover task. Evol. Intell. 3(1), 13–29 (2010)

    Article  Google Scholar 

  77. G. Nitschke, M. Schut, A. Eiben, Evolving behavioral specialization in robot teams to solve a collective construction task. Swarm Evol. Comput. 2(1), 25–38 (2011)

    Google Scholar 

  78. C. Noirot, J. Pasteels, Ontogenetic development and the evolution of the worker caste in termites. Experientia 43(1), 851–860 (1987)

    Article  Google Scholar 

  79. S. Nolfi, Evorobot 1.1 User Manual. Technical Report. (Institute of Cognitive Sciences, National Research Council, Rome, 2000)

  80. S. Nolfi, G. Baldassarre, D. Parisi, Evolution of collective behaviour in a team of physically linked robots, in Applications of Evolutionary Computing (Springer, Berlin, 2003), pp. 581–592

  81. S. Nolfi, D. Floreano, Co-evolving predator and prey robots: Do arm races arise in artificial evolution. Artif. Life 4(4), 311–335 (1999)

    Article  Google Scholar 

  82. S. Nolfi, D. Floreano, Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines. (MIT Press, Cambridge, 2000)

    Google Scholar 

  83. R. Page, S. Mitchell, Self organization and adaptation in insect societies. Philos. Sci. Assoc. 2, 289–298 (1991)

    Google Scholar 

  84. G. Parker, Co-evolving model parameters for anytime learning in evolutionary robotics. Robot. Auton. Syst. 33(1), 13–30 (2000)

    Article  Google Scholar 

  85. A. Perez-Uribe, D. Floreano, L. Keller, Effects of group composition and level of selection in the evolution of cooperation in artificial ants, in Advances of Artificial Life: Proceedings of the Seventh European Conference on Artificial Life (Springer, Dortmund, 2003), pp. 128–137

  86. M. Potter, Design and Analysis of a Computational Model of Cooperative Coevolution. (Computer Science Department, George Mason University, Fairfax, 1997)

    Google Scholar 

  87. M. Potter, K. De Jong, Cooperative coevolution: An architecture for evolving coadapted subcomponents. Evol. Comput. 8(1), 1–29 (2000)

    Article  Google Scholar 

  88. M. Potter, L. Meeden, A. Schultz, Heterogeneity in the coevolved behaviors of mobile robots: The emergence of specialists, in Proceedings of the International Joint Conference on Artificial Intelligence (AAAI Press, Seattle, 2001), pp. 1337–1343

  89. M. Quinn, A comparison of approaches to the evolution of homogeneous multi-robot teams, in Proceedings of the Congress Evolutionary Computation (IEEE Press, Seoul, 2001), pp. 128–135

  90. M. Resnick, Turtles, Termites, and Traffic Jams: Explorations in Massively Parallel Microworlds. (MIT Press, Cambridge, 1997)

    Google Scholar 

  91. G. Robinson, Regulation of division of labor in insect societies. Annu. Rev. Entomol. 37(1), 637–665 (1992)

    Article  Google Scholar 

  92. B. Sareni, L. Krahenbuhl, Fitness sharing and niching methods revisited. IEEE Trans. Evol. Comput. 2(3), 97–106 (1998)

    Article  Google Scholar 

  93. C. Schultz, L. Parker, in Multi-robot Systems: From Swarms to Intelligent Automata (Kluwer Academic Publishers, Washington DC, 2002)

  94. H. Seligmann, Resource partition history and evolutionary specialization of subunits in complex systems. Biosystems 51(1), 31–39 (1999)

    Article  Google Scholar 

  95. F. Seredynski, Competitive coevolutionary multi-agent systems: The application to mapping and scheduling problems. J. Parallel Distrib. Comput. 47(1), 39–57 (1997)

    Article  MathSciNet  Google Scholar 

  96. K. Stanley, B. Bryant, R. Miikkulainen, Real-time neuro-evolution in the nero video game. IEEE Trans. Evol. Comput. 9(6), 653–668 (2005)

    Article  Google Scholar 

  97. P. Stone, Layered Learning in Multi-Agent Systems: A Winning Approach to Robotic Soccer. (MIT Press, Cambridge, 2000)

    Google Scholar 

  98. G. Theraulaz, E. Bonabeau, J. Deneubourg, Response threshold reinforcement and division of labour in insect societies. Proc. R. Soc. Lond. B 265(1), 327–332 (1998)

    Article  Google Scholar 

  99. A. Toffolo, E. Benini, Genetic diversity as an objective in multi-objective evolutionary algorithms. Evol. Comput. 11(2), 151–167 (2003)

    Article  Google Scholar 

  100. van den P. Houta, K. Mathotb, L. Maasc, T. Piersma, Predator escape tactics in birds: linking ecology and aerodynamics. Behav. Ecol. 21(1), 16–25 (2011)

    Google Scholar 

  101. M. Waibel, D. Floreano, L. Keller, Genetic team composition and level of selection in the evolution of cooperation. IEEE Trans. Evol. Comput. 13(3), 648–660 (2009)

    Article  Google Scholar 

  102. M. Waibel, D. Floreano, S. Magnenat, L. Keller, Division of labor and colony efficiency in social insects: effects of interactions between genetic architecture, colony kin structure and rate of perturbations. Proc. R. Soc. B 273(1), 1815–1823 (2006)

    Article  Google Scholar 

  103. T. Wenseleers, F. Ratnieks, J. Billen, Caste fate conflict in swarm-founding social hymenoptera: an inclusive fitness analysis. Evol. Biol. 16(1), 647–658 (2003)

    Google Scholar 

  104. S. Whiteson, N. Kohl, R. Miikkulainen, P. Stone, Evolving keep-away soccer players through task decomposition, in Proceeding of the Genetic and Evolutionary Computation Conference (AAAI Press, Chicago 2003), pp. 356–368

  105. R. Wiegand, An Analysis of Cooperative Coevolutionary Algorithms. PhD. Thesis. (George Mason University Press, George Mason University, Fairfax, 2004)

  106. A. Wieland, Evolving neural network controllers for unstable systems, in Proceedings of the International Joint Conference on Neural Networks (IEEE Press, Seattle, 1991), pp. 667–673

  107. M. Wineberg, F. Oppacher, The underlying similarity of diversity measures used in evolutionary computation, in Proceedings of the Genetic and Evolutionary Computation Conference (Springer, 2003), pp. 1493–1504

  108. X. Yao, Evolutionary artificial neural networks. J. Neural Syst. 4(3), 203–222 (1993)

    Article  Google Scholar 

  109. C. Yong, R. Miikkulainen, Co-evolution of role-based cooperation in multi-agent systems. IEEE Trans. Auton. Mental Dev. 1(3), 170–186 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ikegami Lab, Department of General Systems Studies, University of Tokyo, Tokyo, Japan

    G. S. Nitschke

  2. Computational Intelligence Group, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

    A. E. Eiben & M. C. Schut

Authors
  1. G. S. Nitschke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. E. Eiben
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. C. Schut
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. S. Nitschke.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nitschke, G.S., Eiben, A.E. & Schut, M.C. Evolving team behaviors with specialization. Genet Program Evolvable Mach 13, 493–536 (2012). https://doi.org/10.1007/s10710-012-9166-5

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-012-9166-5

Keywords

Search

Navigation