Novelty Search in Competitive Coevolution

  • Jorge Gomes
  • Pedro Mariano
  • Anders Lyhne Christensen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8672)


One of the main motivations for the use of competitive coevolution systems is their ability to capitalise on arms races between competing species to evolve increasingly sophisticated solutions. Such arms races can, however, be hard to sustain, and it has been shown that the competing species often converge prematurely to certain classes of behaviours. In this paper, we investigate if and how novelty search, an evolutionary technique driven by behavioural novelty, can overcome convergence in coevolution. We propose three methods for applying novelty search to coevolutionary systems with two species: (i) score both populations according to behavioural novelty; (ii) score one population according to novelty, and the other according to fitness; and (iii) score both populations with a combination of novelty and fitness. We evaluate the methods in a predator-prey pursuit task. Our results show that novelty-based approaches can evolve a significantly more diverse set of solutions, when compared to traditional fitness-based coevolution.


Competitive coevolution behavioural diversity novelty search convergence evolutionary robotics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ashlock, D., Willson, S., Leahy, N.: Coevolution and tartarus. In: Congress on Evolutionary Computation, CEC, vol. 2, pp. 1618–1624. IEEE Press (2004)Google Scholar
  2. 2.
    Avery, P., Louis, S.: Coevolving team tactics for a real-time strategy game. In: Congress on Evolutionary Computation, CEC, pp. 1–8. IEEE Press (2010)Google Scholar
  3. 3.
    Chong, S.Y., Tino, P., Yao, X.: Relationship between generalization and diversity in coevolutionary learning. IEEE Transactions on Computational Intelligence and AI in Games 1(3), 214–232 (2009)CrossRefGoogle Scholar
  4. 4.
    Cliff, D., Miller, G.F.: Tracking the red queen: Measurements of adaptive progress in co-evolutionary simulations. In: Morán, F., Merelo, J.J., Moreno, A., Chacon, P. (eds.) ECAL 1995. LNCS, vol. 929, pp. 200–218. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  5. 5.
    Dziuk, A., Miikkulainen, R.: Creating intelligent agents through shaping of coevolution. In: Congress on Evolutionary Computation, CEC, pp. 1077–1083. IEEE Press (2011)Google Scholar
  6. 6.
    Ebner, M., Watson, R.A., Alexander, J.: Coevolutionary dynamics of interacting species. In: Di Chio, C., et al. (eds.) EvoApplicatons 2010, Part I. LNCS, vol. 6024, pp. 1–10. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Ficici, S.G., Pollack, J.B.: Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states. In: Artificial Life, pp. 238–247. MIT Press (1998)Google Scholar
  8. 8.
    Gomes, J., Mariano, P., Christensen, A.L.: Avoiding convergence in cooperative coevolution with novelty search. In: International Conference on Autonomous Agents and Multi-agent Systems, AAMAS, pp. 1149–1156. IFAAMAS (2014)Google Scholar
  9. 9.
    Gomes, J., Urbano, P., Christensen, A.L.: Progressive minimal criteria novelty search. In: Pavón, J., Duque-Méndez, N.D., Fuentes-Fernández, R. (eds.) IBERAMIA 2012. LNCS, vol. 7637, pp. 281–290. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Gomes, J., Urbano, P., Christensen, A.: Evolution of swarm robotics systems with novelty search. Swarm Intelligence 7(2-3), 115–144 (2013)CrossRefGoogle Scholar
  11. 11.
    Lehman, J., Stanley, K.O.: Abandoning objectives: Evolution through the search for novelty alone. Evolutionary Computation 19(2), 189–223 (2011)CrossRefGoogle Scholar
  12. 12.
    Mouret, J.B., Doncieux, S.: Encouraging behavioral diversity in evolutionary robotics: An empirical study. Evolutionary Computation 20(1), 91–133 (2012)CrossRefGoogle Scholar
  13. 13.
    Nolfi, S.: Co-evolving predator and prey robots. Adaptive Behavior 20(1), 10–15 (2012)CrossRefGoogle Scholar
  14. 14.
    Nolfi, S., Floreano, D.: Coevolving predator and prey robots: Do arms races arise in artificial evolution? Artificial Life 4(4), 311–335 (1998)CrossRefGoogle Scholar
  15. 15.
    Popovici, E., Bucci, A., Wiegand, R.P., De Jong, E.D.: Coevolutionary principles. In: Handbook of Natural Computing, pp. 987–1033. Springer (2012)Google Scholar
  16. 16.
    Reisinger, J., Bahçeci, E., Karpov, I., Miikkulainen, R.: Coevolving strategies for general game playing. In: Computational Intelligence and Games, pp. 320–327. IEEE Press (2007)Google Scholar
  17. 17.
    Rosin, C.D., Belew, R.K.: New methods for competitive coevolution. Evolutionary Computation 5(1), 1–29 (1997)CrossRefGoogle Scholar
  18. 18.
    Watson, R.A., Pollack, J.B.: Coevolutionary dynamics in a minimal substrate. In: Genetic and Evolutionary Computation Conference, GECCO, pp. 702–709. Morgan Kaufmann (2001)Google Scholar
  19. 19.
    Yannakakis, G.N., Hallam, J.: Modeling and augmenting game entertainment through challenge and curiosity. International Journal on Artificial Intelligence Tools 16(6), 981–999 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Jorge Gomes
    • 1
    • 2
  • Pedro Mariano
    • 2
  • Anders Lyhne Christensen
    • 1
    • 3
  1. 1.Instituto de TelecomunicaçõesLisbonPortugal
  2. 2.LabMAg – Faculdade de Ciências da Universidade de LisboaPortugal
  3. 3.Instituto Universitário de Lisboa (ISCTE-IUL)LisbonPortugal

Personalised recommendations