Interactive Evolution of Complex Behaviours Through Skill Encapsulation
Human-based computation (HBC) is an emerging research area in which humans and machines collaborate to solve tasks that neither one can solve in isolation. In evolutionary computation, HBC is often realized through interactive evolutionary computation (IEC), in which a user guides evolution by iteratively selecting the parents for the next generation. IEC has shown promise in a variety of different domains, but evolving more complex or hierarchically composed behaviours remains challenging with the traditional IEC approach. To overcome this challenge, this paper combines the recently introduced ESP (encapsulation, syllabus and pandemonium) algorithm with IEC to allow users to intuitively break complex challenges into smaller pieces and preserve, reuse and combine interactively evolved sub-skills. The combination of ESP principles with IEC provides a new way in which human insights can be leveraged in evolutionary computation and, as the results in this paper show, IEC-ESP is able to solve complex control problems that are challenging for a traditional fitness-based approach.
KeywordsEvolutionary computation Interactive evolutionary computation Modular networks Neuroevolution
We thank Fundación Ramón Areces for funding as part of their postdoc fellowship program.
- 3.Risi, S., Togelius, J.: Neuroevolution in games: state of the art and open challenges. IEEE Trans. Comput. Intell. AI Games PP(99), 1–1 (2015)Google Scholar
- 4.Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. In: Proceedings of the Eleventh International Conference on Artificial Life. Alife XI, MIT Press (2008)Google Scholar
- 5.Goldberg, D.E.: Simple genetic algorithms and the minimal, deceptive problem. Genet. Algorithms Simul. Annealing 74, 88 (1987)Google Scholar
- 8.Woolley, B.G., Stanley, K.O.: A novel human-computer collaboration: combining novelty search with interactive evolution. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, GECCO 2014, pp. 233–240. ACM, New York (2014)Google Scholar
- 9.Löwe, M., Risi, S.: Accelerating the evolution of cognitive behaviors through human-computer collaboration. In: Proceedings of the Genetic and Evolutionary Computation Conference 2016, GECCO 2016, pp. 133–140. ACM, New York (2016)Google Scholar
- 10.Lessin, D., Fussell, D., Miikkulainen, R.: Open-ended behavioral complexity for evolved virtual creatures. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, GECCO 2013, pp. 335–342. ACM, New York (2013)Google Scholar
- 11.Lessin, D., Fussell, D., Miikkulainen, R., Risi, S.: Increasing behavioral complexity for evolved virtual creatures with the ESP method. arXiv preprint arXiv:1510.07957 (2015)
- 13.Khatib, F., Dimaio, F., Cooper, S., Kazmierczyk, M., Gilski, M., Krzywda, S., Zabranska, H., Pichova, I., Thompson, J., Popović, Z., Jaskolski, M., Baker, D.: Crystal structure of a monomeric retroviral protease solved by protein folding game players. Nat. Struct. Mol. Biol. 18(10), 1175–1177 (2010)CrossRefGoogle Scholar
- 16.Doucette, J.A., Lichodzijewski, P., Heywood, M.I.: Hierarchical task decomposition through symbiosis in reinforcement learning. In: Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation, pp. 97–104. ACM (2012)Google Scholar
- 18.Lee, W.P., Hallam, J., Lund, H.H.: Applying genetic programming to evolve behavior primitives and arbitrators for mobile robots. In: IEEE International Conference on Evolutionary Computation, pp. 501–506, April 1997Google Scholar
- 21.Hoover, A.K., Szerlip, P.A., Norton, M.E., Brindle, T.A., Merritt, Z., Stanley, K.O.: Generating a complete multipart musical composition from a single monophonic melody with functional scaffolding. In: Proceedings of the Third International Conference on Computational Creativity, Dublin, Ireland, pp. 111–118, May 2012Google Scholar
- 22.Bernatskiy, A., Hornby, G., Bongard, J.: Improving robot behavior optimization by combining user preferences. In: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems, ALIFE 2014 (2014)Google Scholar