Interactive Evolution of Complex Behaviours Through Skill Encapsulation

  • Pablo González de Prado SalasEmail author
  • Sebastian Risi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10199)


Human-based computation (HBC) is an emerging research area in which humans and machines collaborate to solve tasks that neither one can solve in isolation. In evolutionary computation, HBC is often realized through interactive evolutionary computation (IEC), in which a user guides evolution by iteratively selecting the parents for the next generation. IEC has shown promise in a variety of different domains, but evolving more complex or hierarchically composed behaviours remains challenging with the traditional IEC approach. To overcome this challenge, this paper combines the recently introduced ESP (encapsulation, syllabus and pandemonium) algorithm with IEC to allow users to intuitively break complex challenges into smaller pieces and preserve, reuse and combine interactively evolved sub-skills. The combination of ESP principles with IEC provides a new way in which human insights can be leveraged in evolutionary computation and, as the results in this paper show, IEC-ESP is able to solve complex control problems that are challenging for a traditional fitness-based approach.


Evolutionary computation Interactive evolutionary computation Modular networks Neuroevolution 



We thank Fundación Ramón Areces for funding as part of their postdoc fellowship program.


  1. 1.
    Floreano, D., Dürr, P., Mattiussi, C.: Neuroevolution: from architectures to learning. Evol. Intel. 1(1), 47–62 (2008)CrossRefGoogle Scholar
  2. 2.
    Yao, X.: Evolving artificial neural networks. Proc. IEEE 87(9), 1423–1447 (1999)CrossRefGoogle Scholar
  3. 3.
    Risi, S., Togelius, J.: Neuroevolution in games: state of the art and open challenges. IEEE Trans. Comput. Intell. AI Games PP(99), 1–1 (2015)Google Scholar
  4. 4.
    Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. In: Proceedings of the Eleventh International Conference on Artificial Life. Alife XI, MIT Press (2008)Google Scholar
  5. 5.
    Goldberg, D.E.: Simple genetic algorithms and the minimal, deceptive problem. Genet. Algorithms Simul. Annealing 74, 88 (1987)Google Scholar
  6. 6.
    Takagi, H.: Interactive evolutionary computation: fusion of the capacities of EC optimization and human evaluation. Proc. IEEE 89, 1275–1296 (2001)CrossRefGoogle Scholar
  7. 7.
    Lehman, J., Stanley, K.O.: Abandoning objectives: evolution through the search for novelty alone. Evol. Comput. 19(2), 189–223 (2011)CrossRefGoogle Scholar
  8. 8.
    Woolley, B.G., Stanley, K.O.: A novel human-computer collaboration: combining novelty search with interactive evolution. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, GECCO 2014, pp. 233–240. ACM, New York (2014)Google Scholar
  9. 9.
    Löwe, M., Risi, S.: Accelerating the evolution of cognitive behaviors through human-computer collaboration. In: Proceedings of the Genetic and Evolutionary Computation Conference 2016, GECCO 2016, pp. 133–140. ACM, New York (2016)Google Scholar
  10. 10.
    Lessin, D., Fussell, D., Miikkulainen, R.: Open-ended behavioral complexity for evolved virtual creatures. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, GECCO 2013, pp. 335–342. ACM, New York (2013)Google Scholar
  11. 11.
    Lessin, D., Fussell, D., Miikkulainen, R., Risi, S.: Increasing behavioral complexity for evolved virtual creatures with the ESP method. arXiv preprint arXiv:1510.07957 (2015)
  12. 12.
    Michelucci, P., Dickinson, J.L.: The power of crowds. Science 351(6268), 32–33 (2016)CrossRefGoogle Scholar
  13. 13.
    Khatib, F., Dimaio, F., Cooper, S., Kazmierczyk, M., Gilski, M., Krzywda, S., Zabranska, H., Pichova, I., Thompson, J., Popović, Z., Jaskolski, M., Baker, D.: Crystal structure of a monomeric retroviral protease solved by protein folding game players. Nat. Struct. Mol. Biol. 18(10), 1175–1177 (2010)CrossRefGoogle Scholar
  14. 14.
    Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)CrossRefGoogle Scholar
  15. 15.
    Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2(1), 14–23 (1986)CrossRefGoogle Scholar
  16. 16.
    Doucette, J.A., Lichodzijewski, P., Heywood, M.I.: Hierarchical task decomposition through symbiosis in reinforcement learning. In: Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation, pp. 97–104. ACM (2012)Google Scholar
  17. 17.
    Whiteson, S., Kohl, N., Miikkulainen, R., Stone, P.: Evolving keepaway soccer players through task decomposition. In: Cantú-Paz, E., et al. (eds.) GECCO 2003. LNCS, vol. 2723, pp. 356–368. Springer, Heidelberg (2003). doi: 10.1007/3-540-45105-6_41CrossRefGoogle Scholar
  18. 18.
    Lee, W.P., Hallam, J., Lund, H.H.: Applying genetic programming to evolve behavior primitives and arbitrators for mobile robots. In: IEEE International Conference on Evolutionary Computation, pp. 501–506, April 1997Google Scholar
  19. 19.
    Secretan, J., Beato, N., D’Ambrosio, D.B., Rodriguez, A., Campbell, A., Folsom-Kovarik, J.T., Stanley, K.O.: Picbreeder: a case study in collaborative evolutionary exploration of design space. Evol. Comput. 19(3), 373–403 (2011)CrossRefGoogle Scholar
  20. 20.
    Clune, J., Lipson, H.: Evolving 3D objects with a generative encoding inspired by developmental biology. SIGEVOlution 5(4), 2–12 (2011)CrossRefGoogle Scholar
  21. 21.
    Hoover, A.K., Szerlip, P.A., Norton, M.E., Brindle, T.A., Merritt, Z., Stanley, K.O.: Generating a complete multipart musical composition from a single monophonic melody with functional scaffolding. In: Proceedings of the Third International Conference on Computational Creativity, Dublin, Ireland, pp. 111–118, May 2012Google Scholar
  22. 22.
    Bernatskiy, A., Hornby, G., Bongard, J.: Improving robot behavior optimization by combining user preferences. In: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems, ALIFE 2014 (2014)Google Scholar
  23. 23.
    Wagy, M.D., Bongard, J.C.: Combining computational and social effort for collaborative problem solving. PLoS ONE 10(11), e0142524 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Pablo González de Prado Salas
    • 1
    Email author
  • Sebastian Risi
    • 1
  1. 1.IT University of CopenhagenCopenhagenDenmark

Personalised recommendations