Novelty Search in Competitive Coevolution
One of the main motivations for the use of competitive coevolution systems is their ability to capitalise on arms races between competing species to evolve increasingly sophisticated solutions. Such arms races can, however, be hard to sustain, and it has been shown that the competing species often converge prematurely to certain classes of behaviours. In this paper, we investigate if and how novelty search, an evolutionary technique driven by behavioural novelty, can overcome convergence in coevolution. We propose three methods for applying novelty search to coevolutionary systems with two species: (i) score both populations according to behavioural novelty; (ii) score one population according to novelty, and the other according to fitness; and (iii) score both populations with a combination of novelty and fitness. We evaluate the methods in a predator-prey pursuit task. Our results show that novelty-based approaches can evolve a significantly more diverse set of solutions, when compared to traditional fitness-based coevolution.
KeywordsCompetitive coevolution behavioural diversity novelty search convergence evolutionary robotics
Unable to display preview. Download preview PDF.
- 1.Ashlock, D., Willson, S., Leahy, N.: Coevolution and tartarus. In: Congress on Evolutionary Computation, CEC, vol. 2, pp. 1618–1624. IEEE Press (2004)Google Scholar
- 2.Avery, P., Louis, S.: Coevolving team tactics for a real-time strategy game. In: Congress on Evolutionary Computation, CEC, pp. 1–8. IEEE Press (2010)Google Scholar
- 5.Dziuk, A., Miikkulainen, R.: Creating intelligent agents through shaping of coevolution. In: Congress on Evolutionary Computation, CEC, pp. 1077–1083. IEEE Press (2011)Google Scholar
- 7.Ficici, S.G., Pollack, J.B.: Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states. In: Artificial Life, pp. 238–247. MIT Press (1998)Google Scholar
- 8.Gomes, J., Mariano, P., Christensen, A.L.: Avoiding convergence in cooperative coevolution with novelty search. In: International Conference on Autonomous Agents and Multi-agent Systems, AAMAS, pp. 1149–1156. IFAAMAS (2014)Google Scholar
- 15.Popovici, E., Bucci, A., Wiegand, R.P., De Jong, E.D.: Coevolutionary principles. In: Handbook of Natural Computing, pp. 987–1033. Springer (2012)Google Scholar
- 16.Reisinger, J., Bahçeci, E., Karpov, I., Miikkulainen, R.: Coevolving strategies for general game playing. In: Computational Intelligence and Games, pp. 320–327. IEEE Press (2007)Google Scholar
- 18.Watson, R.A., Pollack, J.B.: Coevolutionary dynamics in a minimal substrate. In: Genetic and Evolutionary Computation Conference, GECCO, pp. 702–709. Morgan Kaufmann (2001)Google Scholar