Swarm Intelligence

, Volume 7, Issue 2–3, pp 115–144 | Cite as

Evolution of swarm robotics systems with novelty search

  • Jorge Gomes
  • Paulo Urbano
  • Anders Lyhne Christensen
Article

Abstract

Novelty search is a recent artificial evolution technique that challenges traditional evolutionary approaches. In novelty search, solutions are rewarded based on their novelty, rather than their quality with respect to a predefined objective. The lack of a predefined objective precludes premature convergence caused by a deceptive fitness function. In this paper, we apply novelty search combined with NEAT to the evolution of neural controllers for homogeneous swarms of robots. Our empirical study is conducted in simulation, and we use a common swarm robotics task—aggregation, and a more challenging task—sharing of an energy recharging station. Our results show that novelty search is unaffected by deception, is notably effective in bootstrapping evolution, can find solutions with lower complexity than fitness-based evolution, and can find a broad diversity of solutions for the same task. Even in non-deceptive setups, novelty search achieves solution qualities similar to those obtained in traditional fitness-based evolution. Our study also encompasses variants of novelty search that work in concert with fitness-based evolution to combine the exploratory character of novelty search with the exploitatory character of objective-based evolution. We show that these variants can further improve the performance of novelty search. Overall, our study shows that novelty search is a promising alternative for the evolution of controllers for robotic swarms.

Keywords

Evolutionary robotics Neuroevolution Swarm robotics Novelty search NEAT Behavioural diversity Deception 

Supplementary material

(MP4 12.1 MB)

(MP4 11.0 MB)

(MP4 15.9 MB)

References

  1. Ampatzis, C., Tuci, E., Trianni, V., & Dorigo, M. (2008). Evolution of signaling in a multi-robot system: categorization and communication. Adaptive Behavior, 16(1), 5–26. CrossRefGoogle Scholar
  2. Bahgeçi, E., & Şahin, E. (2005). Evolving aggregation behaviors for swarm robotic systems: a systematic case study. In Swarm intelligence symposium (pp. 333–340). New York: IEEE Press. Google Scholar
  3. Baldassarre, G., Nolfi, S., & Parisi, D. (2003). Evolving mobile robots able to display collective behaviors. Artificial Life, 9(3), 255–268. CrossRefGoogle Scholar
  4. Baldassarre, G., Trianni, V., Bonani, M., Mondada, F., Dorigo, M., & Nolfi, S. (2007). Self-organized coordinated motion in groups of physically connected robots. IEEE Transactions on Systems, Man and Cybernetics, 37(1), 224–239. CrossRefGoogle Scholar
  5. Bayindir, L., & Şahin, E. (2007). A review of studies in swarm robotics. Turkish Journal of Electrical Engineering and Computer Sciences, 15(2), 115–147. Google Scholar
  6. Bonabeau, E., Dorigo, M., & Theraulaz, G. (1999). Swarm intelligence: from natural to artificial systems. New York: Oxford University Press. MATHGoogle Scholar
  7. Brambilla, M., Ferrante, E., Birattari, M., & Dorigo, M. (2013). Swarm robotics: a review from the swarm engineering perspective. Swarm Intelligence, 7(1), 1–41. CrossRefGoogle Scholar
  8. Cao, Y. U., Fukunaga, A. S., & Kahng, A. B. (1997). Cooperative mobile robotics: antecedents and directions. Autonomous Robots, 4(1), 7–27. CrossRefGoogle Scholar
  9. Castelli, M., Manzoni, L., & Vanneschi, L. (2011). A method to reuse old populations in genetic algorithms. In LNCS: Vol. 7026. Portuguese conference on artificial intelligence (EPIA) (pp. 138–152). Berlin: Springer. CrossRefGoogle Scholar
  10. Chellapilla, K., & Fogel, D. B. (1999). Evolving neural networks to play checkers without relying on expert knowledge. IEEE Transactions on Neural Networks, 10(6), 1382–1391. CrossRefGoogle Scholar
  11. Correll, N., & Martinoli, A. (2007). Modeling self-organized aggregation in a swarm of miniature robots. In IEEE international conference on robotics and automation (ICRA) (pp. 379–384). New York: IEEE Press. Google Scholar
  12. Cuccu, G., & Gomez, F. J. (2011). When novelty is not enough. In LNCS: Vol. 6624. European conference on the applications of evolutionary computation (EvoApplications) (pp. 234–243). Berlin: Springer. CrossRefGoogle Scholar
  13. Cuccu, G., Gomez, F. J., & Glasmachers, T. (2011). Novelty-based restarts for evolution strategies. In IEEE congress on evolutionary computation (IEEE CEC) (pp. 158–163). New York: IEEE Press. Google Scholar
  14. Deb, K. (2001). Multi-objective optimization using evolutionary algorithms. Hoboken: Wiley. MATHGoogle Scholar
  15. Doncieux, S., & Mouret, J.-B. (2010). Behavioral diversity measures for evolutionary robotics. In IEEE congress on evolutionary computation (IEEE CEC) (pp. 1–8). New York: IEEE Press. CrossRefGoogle Scholar
  16. Doncieux, S., Mouret, J.-B., Bredeche, N., & Padois, V. (2011). Evolutionary robotics: exploring new horizons. In Studies in computational intelligence: Vol. 341. New horizons in evolutionary robotics (pp. 3–25). Berlin: Springer. CrossRefGoogle Scholar
  17. Goldberg, D. E., & Richardson, J. (1987). Genetic algorithms with sharing for multimodal function optimization. In Genetic algorithms and their applications: second international conference on genetic algorithms (pp. 41–49). Mahwah: Erlbaum. Google Scholar
  18. Gomes, J., Urbano, P., & Christensen, A. L. (2012). Progressive minimal criteria novelty search. In LNAI: Vol. 7637. Ibero-American conference on artificial intelligence (IBERAMIA) (pp. 281–290). Berlin: Springer. Google Scholar
  19. Gomez, F., & Mikkulainen, R. (1997). Incremental evolution of complex general behavior. Adaptative Behaviour, 5(3–4), 317–342. CrossRefGoogle Scholar
  20. Gross, R., & Dorigo, M. (2008). Evolution of solitary and group transport behaviors for autonomous robots capable of self-assembling. Adaptive Behavior, 16(5), 285–305. CrossRefGoogle Scholar
  21. Gutiérrez, A., Campo, A., Dorigo, M., Amor, D., Magdalena, L., & Monasterio-Huelin, F. (2008). An open localization and local communication embodied sensor. Sensors, 8(11), 7545–7563. CrossRefGoogle Scholar
  22. Harvey, I., Husbands, P., & Cliff, D. (1993). Issues in evolutionary robotics. In International conference on simulation of adaptive behavior (SAB) (pp. 364–373). Cambridge: MIT Press. Google Scholar
  23. Hauert, S., Zufferey, J.-C., & Floreano, D. (2009). Evolved swarming without positioning information: an application in aerial communication relay. Autonomous Robots, 26(1), 21–32. CrossRefGoogle Scholar
  24. Hornby, G. (2006). ALPS: the age-layered population structure for reducing the problem of premature convergence. In Genetic and evolutionary computation conference (GECCO) (pp. 815–822). New York: ACM. Google Scholar
  25. Hu, J., Goodman, E. D., Seo, K., Fan, Z., & Rosenberg, R. (2005). The hierarchical fair competition (HFC) framework for sustainable evolutionary algorithms. Evolutionary Computation, 13(2), 241–277. CrossRefGoogle Scholar
  26. Hugues, L., & Bredeche, N. (2006). Simbad: an autonomous robot simulation package for education and research. In LNCS: Vol. 4095. International conference on simulation of adaptive behavior (SAB) (pp. 831–842). Berlin: Springer. Google Scholar
  27. Hutter, M., & Legg, S. (2006). Fitness uniform optimization. IEEE Transactions on Evolutionary Computation, 10(5), 568–589. CrossRefGoogle Scholar
  28. Jeanson, R., Rivault, C., Deneubourg, J.-L., Blanco, S., Fournier, R., Jost, C., & Theraulaz, G. (2005). Self-organized aggregation in cockroaches. Animal Behaviour, 69(1), 169–180. CrossRefGoogle Scholar
  29. Jones, T., & Forrest, S. (1995). Fitness distance correlation as a measure of problem difficulty for genetic algorithms. In International conference on genetic algorithms (ICGA) (pp. 184–192). San Mateo: Morgan Kaufmann. Google Scholar
  30. Kernbach, S., & Kernbach, O. (2011). Collective energy homeostasis in a large-scale microrobotic swarm. Robotics and Autonomous Systems, 59(12), 1090–1101. CrossRefGoogle Scholar
  31. Kirkpatrick, D. A. (2012) Novelty search in competitive coevolution using normalized compression distance. Master thesis, College of Engineering, Florida Institute of Technology. Google Scholar
  32. Kistemaker, S., & Whiteson, S. (2011). Critical factors in the performance of novelty search. In Genetic and evolutionary computation conference (GECCO) (pp. 965–972). New York: ACM. Google Scholar
  33. Knowles, J., Watson, R., & Corne, D. (2001). Reducing local optima in single-objective problems by multi-objectivization. In LNCS: Vol. 1993. Evolutionary multi-criterion optimization (pp. 269–283). Berlin: Springer. CrossRefGoogle Scholar
  34. Kohonen, T. (1990). The self-organizing map. Proceedings of the IEEE, 78(9), 1464–1480. CrossRefGoogle Scholar
  35. Krcah, P. (2010). Solving deceptive tasks in robot body-brain co-evolution by searching for behavioral novelty. In International conference on intelligent systems design and applications (ISDA) (pp. 284–289). New York: IEEE Press. Google Scholar
  36. Lehman, J., & Stanley, K. O. (2008). Exploiting open-endedness to solve problems through the search for novelty. In International conference on the synthesis and simulation of living systems (ALIFE) (pp. 329–336). Cambridge: MIT Press. Google Scholar
  37. Lehman, J., & Stanley, K. O. (2010a). Revising the evolutionary computation abstraction: minimal criteria novelty search. In Genetic and evolutionary computation conference (GECCO) (pp. 103–110). New York: ACM. Google Scholar
  38. Lehman, J., & Stanley, K. O. (2010b). Efficiently evolving programs through the search for novelty. In Genetic and evolutionary computation conference (GECCO) (pp. 837–844). New York: ACM. Google Scholar
  39. Lehman, J., & Stanley, K. O. (2011a). Abandoning objectives: evolution through the search for novelty alone. Evolutionary Computation, 19(2), 189–223. CrossRefGoogle Scholar
  40. Lehman, J., & Stanley, K. O. (2011b). Evolving a diversity of virtual creatures through novelty search and local competition. In Genetic and evolutionary computation conference (GECCO) (pp. 211–218). New York: ACM. Google Scholar
  41. Liu, W., Winfield, A. F. T., & Sa, J. (2007). Modelling swarm robotic systems: a case study in collective foraging. In Towards autonomous robotic systems (TAROS) (pp. 25–32). Google Scholar
  42. Michaud, F., & Robichaud, E. (2002). Sharing charging stations for long-term activity of autonomous robots. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (Vol. 3, pp. 2746–2751). New York: IEEE Press. CrossRefGoogle Scholar
  43. Mondada, F., Bonani, M., Raemy, X., Pugh, J., Cianci, C., Klaptocz, A., Magnenat, S., Zufferey, J.-C., Floreano, D., & Martinoli, A. (2009). The e-puck, a robot designed for education in engineering. In 9th conference on autonomous robot systems and competitions (ROBOTICA) (pp. 59–65). Castelo Branco: IPCB. Google Scholar
  44. Mouret, J.-B. (2011). Novelty-based multiobjectivization. In Studies in computational intelligence: Vol. 341. New horizons in evolutionary robotics (pp. 139–154). Berlin: Springer. CrossRefGoogle Scholar
  45. Mouret, J.-B., & Doncieux, S. (2009). Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity. In IEEE congress on evolutionary computation (IEEE CEC) (pp. 1161–1168). New York: IEEE Press. Google Scholar
  46. Mouret, J.-B., & Doncieux, S. (2012). Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evolutionary Computation, 20(1), 91–133. CrossRefGoogle Scholar
  47. Muñoz Meléndez, A., Sempé, F., & Drogoul, A. (2002). Sharing a charging station without explicit communication in collective robotics. In International conference on simulation of adaptive behavior (SAB) (pp. 383–384). Cambridge: MIT Press. Google Scholar
  48. Nelson, A. L., Barlow, G. J., & Doitsidis, L. (2009). Fitness functions in evolutionary robotics: a survey and analysis. Robotics and Autonomous Systems, 57(4), 345–370. CrossRefGoogle Scholar
  49. Pini, G., & Tuci, E. (2008). On the design of neuro-controllers for individual and social learning behaviour in autonomous robots: an evolutionary approach. Connection Science, 20(2–3), 211–230. CrossRefGoogle Scholar
  50. Şahin, E. (2005). Swarm robotics: from sources of inspiration to domains of application. In LNCS: Vol. 3342. International workshop on swarm robotics (pp. 10–20). Berlin: Springer. CrossRefGoogle Scholar
  51. Schmidt, M., & Lipson, H. (2011). Age-fitness Pareto optimization. In Genetic and evolutionary computation: Vol. 8. Genetic programming theory and practice VIII (pp. 129–146). Berlin: Springer. CrossRefGoogle Scholar
  52. Soltoggio, A., & Jones, B. (2009). Novelty of behaviour as a basis for the neuro-evolution of operant reward learning. In Genetic and evolutionary computation conference (GECCO) (pp. 169–176). New York: ACM. Google Scholar
  53. Soysal, O., Bahgeçi, E., & Şahin, E. (2007). Aggregation in swarm robotic systems: evolution and probabilistic control. Turkish Journal of Electrical Engineering and Computer Sciences, 15(2), 199–225. Google Scholar
  54. Sperati, V., Trianni, V., & Nolfi, S. (2008). Evolving coordinated group behaviours through maximisation of mean mutual information. Swarm Intelligence, 2(2–4), 73–95. CrossRefGoogle Scholar
  55. Sprong, C. (2011). Common tasks in evolutionary robotics, an overview. Technical report, Faculty of Sciences, University of Amsterdam, Netherlands. URL http://www.few.vu.nl/nl/Images/werkstuk-sprong_tcm38-217791.pdf.
  56. Stanley, K. O., & Miikkulainen, R. (2002). Evolving neural network through augmenting topologies. Evolutionary Computation, 10(2), 99–127. CrossRefGoogle Scholar
  57. Trianni, V. (2008). Studies in computational intelligence: Vol. 108. Evolutionary swarm robotics: evolving self-organising behaviours in groups of autonomous robots. Berlin: Springer. CrossRefGoogle Scholar
  58. Trianni, V., Gross, R., Labella, T. H., Şahin, E., & Dorigo, M. (2003). Evolving aggregation behaviors in a swarm of robots. In LNCS: Vol. 2801. European conference on artificial life (ECAL) (pp. 865–874). Berlin: Springer. CrossRefGoogle Scholar
  59. Trianni, V., Nolfi, S., & Dorigo, M. (2006). Cooperative hole avoidance in a swarm-bot. Robotics and Autonomous Systems, 54(2), 97–103. CrossRefGoogle Scholar
  60. Uchibe, E., Yanase, M., & Asada, M. (2002). Behavior generation for a mobile robot based on the adaptive fitness function. Robotics and Autonomous Systems, 40(2–3), 69–77. CrossRefGoogle Scholar
  61. Watson, R. A., & Pollack, J. B. (2001). Coevolutionary dynamics in a minimal substrate. In Genetic and evolutionary computation conference (GECCO) (pp. 702–709). San Mateo: Morgan Kaufmann. Google Scholar
  62. Whitley, L. D. (1991). Fundamental principles of deception in genetic search. In Foundations of genetic algorithms (pp. 221–241). San Mateo: Morgan Kaufmann. Google Scholar
  63. Zaera, N., Cliff, D., & Bruten, J. (1996). (Not) Evolving collective behaviours in synthetic fish. In International conference on simulation of adaptive behavior (SAB) (pp. 635–644). Cambridge: MIT Press. Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Jorge Gomes
    • 1
  • Paulo Urbano
    • 2
  • Anders Lyhne Christensen
    • 3
  1. 1.LabMAgFCUL & Instituto de TelecomunicaçõesLisbonPortugal
  2. 2.LabMAgFCULLisbonPortugal
  3. 3.Instituto Universitário de Lisboa (ISCTE-IUL) & Instituto de TelecomunicaçõesLisbonPortugal

Personalised recommendations