An Interactively Constrained Neuro-Evolution Approach for Behavior Control of Complex Robots

  • Christian Rempis
  • Frank Pasemann


Behavior control of complex technical systems, such as robots, is a challenging problem. In this context, embodied neuro-control is a bio-inspired method for handling this type of problems, and evolutionary robotics has taken up some essential research topics in this field. However, for systems with many multi-modal sensor inputs and actuating outputs, new evolutionary methods have to be applied because the search spaces are high-dimensional and comprise many local optima. This becomes even harder when functional recurrent network structures cannot be given in advance and have to be evolved together with other parameters like synaptic weights and bias terms. This chapter describes a new evolutionary method, called Interactively Constrained Neuro − Evolution (ICONE), which restricts large search spaces by utilizing not only domain knowledge and user experience but also by applying constraints to the networks. The interactive use of this tool enables the experimenter to bias the solution space towards desired control approaches. The application of the ICONE method is demonstrated by evolving a walking behavior for a physical humanoid robot, for which a whole library of behaviors has been developed.


Search Space Recurrent Neural Network Humanoid Robot Behavior Control Model Rotation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Nolfi, S., Floreano, D.: Evolutionary Robotics. MIT Press, Cambridge (2004); ISBN-13: 978-0-262-14070-6Google Scholar
  2. 2.
    Floreano, D., Dürr, P., Mattiussi, C.: Neuroevolution: from architectures to learning. Evolutionary Intelligence 1(1), 47–62 (2008)CrossRefGoogle Scholar
  3. 3.
    Trianni, V.: Evolutionary Swarm Robotics: Evolving Self-Organising Behaviours in Groups of Autonomous Robots. SCI. Springer Publishing Company, Heidelberg (2008)Google Scholar
  4. 4.
    From animals to animats 10, Proceedings of 10th International Conference on Simulation of Adaptive Behavior, SAB 2008, Osaka, Japan, July 7-12 (2008)Google Scholar
  5. 5.
    Steels, L.: Language games for autonomous robots. IEEE Intelligent Systems 16(5), 16–22 (2001)Google Scholar
  6. 6.
    Hild, M., Meissner, M., Spranger, M.: Humanoid Team Humboldt Team Description 2007 for RoboCup 2007, Atlanta, USA (2007)Google Scholar
  7. 7.
    Yao, X.: Evolving artificial neural networks. Proceedings of the IEEE 87(9), 1423–1447 (1999)CrossRefGoogle Scholar
  8. 8.
    Harvey, I., Husbands, P., Cliff, D., Thompson, A., Jakobi, N.: Evolutionary robotics: the sussex approach. Robotics and Autonomous Systems (1997)Google Scholar
  9. 9.
    Harvey, I., Di Paolo, E., Wood, R., Quinn, M., Tuci, E.: Evolutionary robotics: A new scientific tool for studying cognition. Artificial Life 11(1-2), 79–98 (2005)CrossRefGoogle Scholar
  10. 10.
    Lungarella, M., Mettay, G., Pfeifer, R., Sandiniy, G.: Developmental robotics: a survey. Connection Science 15(4), 151–190 (2003)CrossRefGoogle Scholar
  11. 11.
    Pfeifer, R.: On the role of embodiment in the emergence of cognition: Grey walter’s turtles and beyond. In: Proc. of the Workshop The Legacy of Grey Walter (2002)Google Scholar
  12. 12.
    Hülse, M., Wischmann, S., Pasemann, F.: The Role of Non-linearity for Evolved Multifunctional Robot Behavior. In: Moreno, J.M., Madrenas, J., Cosp, J. (eds.) ICES 2005. LNCS, vol. 3637, pp. 108–118. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Braitenberg, V.: Vehicles: Experiments in Synthetic Psychology. MIT Press, Cambridge (1984)Google Scholar
  14. 14.
    Yamauchi, B.M., Beer, R.D.: Sequential behavior and learning in evolved dynamical neural networks. Adaptive Behaviour 2(3), 219–246 (1994)CrossRefGoogle Scholar
  15. 15.
    Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. Journal of Machine Learning Research 9, 937–965 (2008)MathSciNetGoogle Scholar
  16. 16.
    Gomez, F. J.: Robust Non-Linear Control through Neuroevolution. PhD thesis, August 1, Tue, 6 Jan 104 19:10:41 GMT (2003)Google Scholar
  17. 17.
    Stanley, K.O.: Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines 8(2), 131–162 (2007)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artificial Life 15(2), 185–212 (2009)CrossRefGoogle Scholar
  19. 19.
    Koza, J.R., Rice, J.P.: Genetic generation of both the weights and architecture for a neural network. In: International Joint Conference on Neural Networks (1991)Google Scholar
  20. 20.
    Pasemann, F., Steinmetz, U., Hülse, M., Lara, B.: Robot control and the evolution of modular neurodynamics. Theory in Biosciences 120(3-4), 311–326 (2001)Google Scholar
  21. 21.
    Angeline, P.J., Saunders, G.M., Pollack, J.P.: An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks 5(1), 54–65 (1994)CrossRefGoogle Scholar
  22. 22.
    Gruau, F.: Neural Network Synthesis using Cellular Encoding and the Genetic Algorithm. Laboratoire de l’Informatique du Parallilisme, Ecole Normale Supirieure de Lyon (1994)Google Scholar
  23. 23.
    Hornby, G.S., Pollack, J.B.: Body-brain co-evolution using L-systems as a generative encoding. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W.B., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), July 7-11, pp. 868–875. Morgan Kaufmann, San Francisco (2001)Google Scholar
  24. 24.
    Cangelosi, A., Parisi, D., Nolfi, S.: Cell division and migration in a ’genotype’ for neural networks. Network: Computation in Neural Systems 5(4), 497–515 (1994)zbMATHCrossRefGoogle Scholar
  25. 25.
    Belew, R.K.: Interposing an ontogenetic model between genetic algorithms and neural networks. In: Advances in Neural Information Processing Systems 5, NIPS Conference, p. 106. Morgan Kaufmann, San Francisco (1992)Google Scholar
  26. 26.
    Moriaty, D.E.: Symbiotic Evolution of Neural Networks in Sequential Decision Tasks. PhD thesis, The University of Texas at Austin, 1 (1997)Google Scholar
  27. 27.
    Clune, J., Beckmann, B.E., Pennock, R.T., Ofria, C.: HybrID: A hybridization of indirect and direct encodings for evolutionary computation. In: Kampis, G., Karsai, I., Szathmáry, E. (eds.) ECAL 2009, Part II. LNCS, vol. 5778, pp. 134–141. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  28. 28.
    Inden, B.: Stepwise Transition from Direct Encoding to Artificial Ontogeny in Neuroevolution. In: Almeida e Costa, F., Rocha, L.M., Costa, E., Harvey, I., Coutinho, A. (eds.) ECAL 2007. LNCS (LNAI), vol. 4648, pp. 1182–1191. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  29. 29.
    Doncieux, S., Meyer, J.-A.: Evolving modular neural networks to solve challenging control problems. In: Proceedings of The Fourth International ICSC Symposium on Engineering of Intelligent Systems (EIS 2004), Acta Press (2004)Google Scholar
  30. 30.
    Meyer, J.-A., Doncieux, S., David, Guillot, A.: Evolutionary approaches to neural control of rolling, walking, swimming and flying animats or robots. Biologically Inspired Robot Behavior Engineering, 1–43 (2003)Google Scholar
  31. 31.
    Jeffrey, L., Elman, J.L.: Learning and development in neural networks: The importance of starting small. Cognition 48, 71–99 (1993)CrossRefGoogle Scholar
  32. 32.
    Hülse, M., Wischmann, S., Pasemann, F.: Structure and function of evolved neuro-controllers for autonomous robots. Connection Science 16(4), 249–266 (2004)CrossRefGoogle Scholar
  33. 33.
    Stanley, K.O., Miikkulainen, R.P.: Efficient evolution of neural networks through complexification. PhD thesis, The University of Texas at Austin (2004)Google Scholar
  34. 34.
    Lee Giles, C., Omlin, C.W.: Pruning recurrent neural networks for improved generalization performance. IEEE Transactions on Neural Networks/A Publication of The IEEE Neural Networks Council 5(5), 848 (1994)Google Scholar
  35. 35.
    Bongard, J.C., Pfeifer, R.: Evolving complete agents using artificial ontogeny. Morpho-Functional Machines: The New Species Designing Embodied Intelligence, 237–258 (2003)Google Scholar
  36. 36.
    Nolfi, S., Parisi, D.: Evolving Artificial Neural Networks that Develop in Time. In: Proceedings of the Third European Conference on Advances in Artificial Life, p. 367. Springer, Heidelberg (1995)Google Scholar
  37. 37.
    Gruau, F.: Automatic definition of modular neural networks. Adaptive Behaviour 3(2), 151–183 (1995)CrossRefGoogle Scholar
  38. 38.
    Nolfi, N., Parisi, D.: Growing neural networks. Technical Report PCIA-91-15, Institute of Psychology (December 1991)Google Scholar
  39. 39.
    Pasemann, F.: Neuromodules: A dynamical systems approach to brain modelling. In: Herrmann, H.J., Wolf, D.E., Poppel, E. (eds.) Workshop on Supercomputing in Brain Research: From Tomography to Neural Networks, November 21-23. World Scientific Publishing Co., Germany (1995)Google Scholar
  40. 40.
    Horton, J.C., Adams, D.L.: The cortical column: a structure without a function. Philosophical Transactions of the Royal Society B: Biological Sciences 360(1456), 837 (2005)CrossRefGoogle Scholar
  41. 41.
    Reisinger, J., Stanley, K.O., Miikkulainen, R.: Evolving Reusable Neural Modules. In: Deb, K., Poli, R., Banzhaf, W., Beyer, H.-G., Burke, E., Darwen, P., Dasgupta, D., Floreano, D., Foster, J., Harman, M., Holland, O., Lanzi, P.L., Spector, L., Tettamanzi, A., Thierens, D., Tyrrell, A., et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 69–81. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  42. 42.
    Valsalam, V.K., Miikkulainen, R.: Evolving symmetric and modular neural networks for distributed control. In: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 731–738. ACM, New York (2009)CrossRefGoogle Scholar
  43. 43.
    Gomez, F., Miikkulainen, R.: Incremental evolution of complex general behavior. Technical Report AI96-248, The University of Texas at Austin, Department of Computer Sciences, June 1, November 7, 106 21:26:08 GMT (1997)Google Scholar
  44. 44.
    Gauci, J., Stanley, K.O.: Generating large-scale neural networks through discovering geometric regularities. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, p. 1004. ACM, New York (2007)Google Scholar
  45. 45.
    David, B., D’Ambrosio, D.B., Stanley, K.O.: A novel generative encoding for exploiting neural network sensor and output geometry. In: Genetic and Evolutionary Computation Conference: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation. Association for Computing Machinery, Inc., New York (2007)Google Scholar
  46. 46.
    Dieckmann, U.: Coevolution as an autonomous learning strategy for neuromodules. In: Herrmann, H.J., Wolf, D.E., Poppel, E. (eds.) Workshop On Supercomputing In Brain Research: From Tomography To Neural Networks, November 21-23. World Scientific Publishing Co., Germany (1995)Google Scholar
  47. 47.
    Pasemann, F., Steinmetz, U., Dieckman, U.: Evolving structure and function of neurocontrollers. In: Angeline, P.J., Michalewicz, Z., Schoenauer, M., Yao, X., Zalzala, A. (eds.) Proceedings of the Congress on Evolutionary Computation, July 6-9, vol. 3, pp. 1973–1978. IEEE Press, USA (1999)Google Scholar
  48. 48.
    Moriarty, D.E., Miikkulainen, R.: Efficient reinforcement learning through symbiotic evolution. Technical Report AI94-224, The University of Texas at Austin, Department of Computer Sciences (September 1, 1994)Google Scholar
  49. 49.
    Christian, W., Rempis, C.W., Pasemann, F.: Search space restriction of neuro-evolution through constrained modularization of neural networks. In: Mandai, K. (ed.) Proceedings of the 6th International Workshop on Artificial Neural Networks and Intelligent Information Processing (ANNIIP), in Conjunction with ICINCO 2010, pp. 13–22. SciTePress, Portugal (2010)Google Scholar
  50. 50.
    Mahfoud, S.W.: Niching methods for genetic algorithms. Department of Computer Science, University of Illinois at Urbana-Champaign (1995)Google Scholar
  51. 51.
    Hancock, P.J.B.: An empirical comparison of selection methods in evolutionary algorithms. To appear in the Proceedings of the AISB Workshop on Evolutionary Computation, vol. 1 (1994)Google Scholar
  52. 52.
    Reil, T., Husbands, P.: Evolution of central pattern generators for bipedal walking in a real-time physics environment. IEEE Transactions on Evolutionary Computation 6(2), 159–168 (2002)CrossRefGoogle Scholar
  53. 53.
    Hein, D., Hild, M., Berger, R.: Evolution of biped walking using neural oscillators and physical simulation. In: Visser, U., Ribeiro, F., Ohashi, T., Dellaert, F. (eds.) RoboCup 2007: Robot Soccer World Cup XI. LNCS (LNAI), vol. 5001, pp. 433–440. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  54. 54.
    Geng, T., Porr, B., Wörgötter, F.: A reflexive neural network for dynamic biped walking control. Neural computation 18(5), 1156–1196 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  55. 55.
    Manoonpong, P., Geng, T., Kulvicius, T., Porr, B., Wörgötter, F.: Adaptive, Fast Walking in a Biped Robot under Neuronal Control and Learning. PLoS Computational Biology 3(7) (2007)Google Scholar
  56. 56.
    McHale, G., Husbands, P.: GasNets and other evolvable neural networks applied to bipedal locomotion. In: From Animals to Animats 8: Proceedings of the Seventh [ie Eighth] International Conference on Simulation of Adaptive Behavior, p. 163. The MIT Press, Cambridge (1994)Google Scholar
  57. 57.
    Ishiguro, A., Fujii, A., Hotz, P.E.: Neuromodulated control of bipedal locomotion using a polymorphic cpg circuit. Adaptive Behavior 11(1), 7 (2003)CrossRefGoogle Scholar
  58. 58.
    Hase, K., Yamazaki, N.: Computational evolution of human bipedal walking by a neuro-musculo-skeletal model. Artificial Life and Robotics 3(3), 133–138 (1999)CrossRefGoogle Scholar
  59. 59.
    Josh, C.: Making evolution an offer it can’t refuse: Morphology and the extradimensional bypass. Advances in Artificial Life, 401–412 (2001)Google Scholar
  60. 60.
    Cliff, D., Harvey, I., Husbands, P.: Incremental evolution of neural network architectures for adaptive behaviour. In: Proceedings of the First European Symposium on Artificial Neural Networks, ESANN 039, vol. 93, pp. 39–44. D facto Publishing (1992)Google Scholar
  61. 61.
    Pasemann, F.: Complex dynamics and the structure of small neural networks. Network: Computation in Neural Systems 13(2), 195–216 (2002)zbMATHGoogle Scholar
  62. 62.
    Rempis, C., Thomas, V., Bachmann, F., Pasemann, F.: NERD Neurodynamics and Evolutionary Robotics Development Kit. In: Ando, N., Balakirsky, S., Hemker, T., Reggiani, M., von Stryk, O. (eds.) SIMPAR 2010. LNCS, vol. 6472, pp. 121–132. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  63. 63.
    Pasemann, F.: Characterization of periodic attractors in neural ring networks. Neural Networks 8(3), 421–429 (1995)CrossRefGoogle Scholar
  64. 64.
    Pasemann, F., Hild, M., Zahedi, K.: So(2)-networks as neural oscillators. In: Computational Methods in Neural Modeling, vol. 2686, pp. 144–151 (2003)Google Scholar
  65. 65.
    Miller, B.L., Goldberg, D.E.: Genetic algorithms, tournament selection, and the effects of noise. Urbana 51, 61801 (1995)Google Scholar
  66. 66.
    Jakobi, N.: Evolutionary robotics and the radical envelope-of-noise hypothesis. Adaptive behavior 6(2), 325 (1997)CrossRefGoogle Scholar
  67. 67.
    Pollack, J.B., Lipson, H., Ficici, S., Funes, P., Hornby, G., Watson, R.A.: Evolutionary techniques in physical robotics. Evolvable Systems: from biology to hardware, 175–186 (2000)Google Scholar
  68. 68.
    Bongard, J.C., Lipson, H.: Automating Genetic Network Inference with Minimal Physical Experimentation Using Coevolution. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3102, pp. 333–345. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  69. 69.
    von Twickel, A., Pasemann, F.: Reflex-oscillations in evolved single leg neurocontrollers for walking machines. Natural Computing 6(3), 311–337 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  70. 70.
    Rempis, C.W.: Short-term memory structures in additive recurrent neural networks. Master’s thesis, University of Applied Sciences Bonn-Rhein-Sieg, Germany (2007)Google Scholar
  71. 71.
    Wischmann, S., Pasemann, F.: The emergence of communication by evolving dynamical systems. From Animals to Animats 9, 777–788 (2006)CrossRefGoogle Scholar
  72. 72.
    Sidel, T., Hild, M., Weidner, M.: Concept and Design of the Modular Actuator System for the Humanoid Robot MYON. In: International Conference on Intelligent and Applications, ICIRA 2011 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Christian Rempis
    • 1
  • Frank Pasemann
    • 1
  1. 1.Neurocybernetics Group, Institute of Cognitive ScienceUniversity of OsnabrückOsnabrückGermany

Personalised recommendations