Abstract
A network-based modelling technique, search trajectory networks (STNs), has recently helped to understand the dynamics of neuroevolution algorithms such as NEAT. Modelling and visualising variants of NEAT made it possible to analyse the dynamics of search operators. Thus far, this analysis was applied directly to the NEAT genotype space composed of neural network topologies and weights. Here, we extend this work, by illuminating instead the behavioural space, which is available when the evolved neural networks control the behaviour of agents. Recent interest in behaviour characterisation highlights the need for divergent search strategies. Quality-diversity and Novelty search are examples of divergent search, but their dynamics are not yet well understood. In this article, we examine the idiosyncrasies of three neuroevolution variants: novelty, random and objective search operating as usual on the genotypic search space, but analysed in the behavioural space. Results show that novelty is a successful divergent search strategy. However, its abilities to produce diverse solutions are not always consistent. Our visual analysis highlights interesting relationships between topological complexity and behavioural diversity which may pave the way for new characterisations and search strategies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chatzilygeroudis, K., Cully, A., Vassiliades, V., Mouret, J.-B.: Quality-diversity optimization: a novel branch of stochastic optimization. In: Pardalos, P.M., Rasskazova, V., Vrahatis, M.N. (eds.) Black Box Optimization, Machine Learning, and No-Free Lunch Theorems. SOIA, vol. 170, pp. 109–135. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-66515-9_4
Costa, V., Lourenço, N., Machado, P.: Coevolution of generative adversarial networks. In: International Conference on the Applications of Evolutionary Computation (Part of EvoStar). pp. 473–487. Springer (2019)
Csardi, G., Nepusz, T.: The iGraph software package for complex network research. Int. J. Complex Syst. 1695 (2006)
Cully, A., Demiris, Y.: Quality and diversity optimization: a unifying modular framework. IEEE Trans. Evol. Comput. 22(2), 245–259 (2018)
Doncieux, S., Laflaquière, A., Coninx, A.: Novelty search: a theoretical perspective. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 99–106 (2019)
Kamada, T., Kawai, S.: An algorithm for drawing general undirected graphs. Inf. Process. Lett. 31, 7–15 (1989)
Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. In: ALIFE Xi, pp. 329–336 (2008)
Lehman, J., Stanley, K.O.: Abandoning objectives: evolution through the search for novelty alone. Evol. Comput. 19(2), 189–222 (2011). https://doi.org/10.1162/EVCO_a_00025
Lehman, J., Stanley, K.O.: Evolving a diversity of virtual creatures through novelty search and local comp. In: GECCO, pp. 211–218 (2011)
Lehman, J., Stanley, K.O., et al.: Exploiting open-endedness to solve problems through the search for novelty (2008)
McIntyre, A., Kallada, M., Miguel, C.G., da Silva, C.F.: NEAT-Python. https://github.com/CodeReclaimers/neat-python
Meyerson, E., Lehman, J., Miikkulainen, R.: Learning behavior characterizations for novelty search, pp. 149–156 (2016). https://doi.org/10.1145/2908812.2908929
Mouret, J.B., Doncieux, S.: Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evol. Comput. 20(1), 91–133 (2012)
Narvaez-Teran, V., Ochoa, G., Rodriguez-Tello, E.: Search trajectory networks applied to the cyclic bandwidth sum problem. IEEE Access 9, 1–1 (2021). https://doi.org/10.1109/access.2021.3126015
Newman, M.E.J.: Networks: An Introduction. Oxford University Press, Oxford (2010)
Ochoa, G., Malan, K.M., Blum, C.: Search trajectory networks: a tool for analysing and visualising the behaviour of metaheuristics. Appl. Soft Comput. 109, 107492 (2021). https://doi.org/10.1016/j.asoc.2021.107492
Ochoa, G., Malan, K.M., Blum, C.: Search trajectory networks of population-based algorithms in continuous spaces. In: Castillo, P.A., Jiménez Laredo, J.L., Fernández de Vega, F. (eds.) EvoApplications 2020. LNCS, vol. 12104, pp. 70–85. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43722-0_5
Ochoa, G., Veerapen, N., Daolio, F., Tomassini, M.: Understanding phase transitions with local optima networks: number partitioning as a case study. In: Hu, B., López-Ibáñez, M. (eds.) EvoCOP 2017. LNCS, vol. 10197, pp. 233–248. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-55453-2_16
Omelianenko, I.: Hands-On Neuroevolution with Python. Packt Publishing Limited, Birmingham (2019)
Pugh, J.K., Soros, L.B., Stanley, K.O.: Quality diversity: a new frontier for evolutionary computation. Front. Robot. AI 3(JUL), 1–17 (2016). https://doi.org/10.3389/frobt.2016.00040
Real, E., et al.: Large-scale evolution of image classifiers. In: International Conference on Machine Learning, pp. 2902–2911. PMLR (2017)
Sarti, S., Ochoa, G.: A NEAT visualisation of neuroevolution trajectories. In: Castillo, P.A., Jiménez Laredo, J.L. (eds.) EvoApplications 2021. LNCS, vol. 12694, pp. 714–728. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72699-7_45
Siebel, N.T., Sommer, G.: Evolutionary reinforcement learning of artificial neural networks. Int. J. Hybrid Intell. Syst. 4(3), 171–183 (2007)
Silva, F., Correia, L., Christensen, A.L.: Evolutionary online behaviour learning and adaptation in real robots. Roy. Soc. Open Sci. 4(7) (2017). https://doi.org/10.1098/rsos.160938
Stanley, K.O., Clune, J., Lehman, J., Miikkulainen, R.: Designing neural networks through neuroevolution. Nat. Mach. Intell. 2, 24–35 (2019)
Stanley, K.O., Miikkulainen, R.: Competitive coevolution through evolutionary complexification. J. Artif. Intell. Res. 21, 63–100 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Sarti, S., Adair, J., Ochoa, G. (2022). Neuroevolution Trajectory Networks of the Behaviour Space. In: Jiménez Laredo, J.L., Hidalgo, J.I., Babaagba, K.O. (eds) Applications of Evolutionary Computation. EvoApplications 2022. Lecture Notes in Computer Science, vol 13224. Springer, Cham. https://doi.org/10.1007/978-3-031-02462-7_43
Download citation
DOI: https://doi.org/10.1007/978-3-031-02462-7_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-02461-0
Online ISBN: 978-3-031-02462-7
eBook Packages: Computer ScienceComputer Science (R0)