Skip to main content
SpringerLink
Log in

An Interactively Constrained Neuro-Evolution Approach for Behavior Control of Complex Robots

  • Chapter
Book cover Variants of Evolutionary Algorithms for Real-World Applications

Abstract

Behavior control of complex technical systems, such as robots, is a challenging problem. In this context, embodied neuro-control is a bio-inspired method for handling this type of problems, and evolutionary robotics has taken up some essential research topics in this field. However, for systems with many multi-modal sensor inputs and actuating outputs, new evolutionary methods have to be applied because the search spaces are high-dimensional and comprise many local optima. This becomes even harder when functional recurrent network structures cannot be given in advance and have to be evolved together with other parameters like synaptic weights and bias terms. This chapter describes a new evolutionary method, called Interactively Constrained Neuro − Evolution (ICONE), which restricts large search spaces by utilizing not only domain knowledge and user experience but also by applying constraints to the networks. The interactive use of this tool enables the experimenter to bias the solution space towards desired control approaches. The application of the ICONE method is demonstrated by evolving a walking behavior for a physical humanoid robot, for which a whole library of behaviors has been developed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Nolfi, S., Floreano, D.: Evolutionary Robotics. MIT Press, Cambridge (2004); ISBN-13: 978-0-262-14070-6

    Google Scholar 

  2. Floreano, D., Dürr, P., Mattiussi, C.: Neuroevolution: from architectures to learning. Evolutionary Intelligence 1(1), 47–62 (2008)

    Article  Google Scholar 

  3. Trianni, V.: Evolutionary Swarm Robotics: Evolving Self-Organising Behaviours in Groups of Autonomous Robots. SCI. Springer Publishing Company, Heidelberg (2008)

    Google Scholar 

  4. From animals to animats 10, Proceedings of 10th International Conference on Simulation of Adaptive Behavior, SAB 2008, Osaka, Japan, July 7-12 (2008)

    Google Scholar 

  5. Steels, L.: Language games for autonomous robots. IEEE Intelligent Systems 16(5), 16–22 (2001)

    Google Scholar 

  6. Hild, M., Meissner, M., Spranger, M.: Humanoid Team Humboldt Team Description 2007 for RoboCup 2007, Atlanta, USA (2007)

    Google Scholar 

  7. Yao, X.: Evolving artificial neural networks. Proceedings of the IEEE 87(9), 1423–1447 (1999)

    Article  Google Scholar 

  8. Harvey, I., Husbands, P., Cliff, D., Thompson, A., Jakobi, N.: Evolutionary robotics: the sussex approach. Robotics and Autonomous Systems (1997)

    Google Scholar 

  9. Harvey, I., Di Paolo, E., Wood, R., Quinn, M., Tuci, E.: Evolutionary robotics: A new scientific tool for studying cognition. Artificial Life 11(1-2), 79–98 (2005)

    Article  Google Scholar 

  10. Lungarella, M., Mettay, G., Pfeifer, R., Sandiniy, G.: Developmental robotics: a survey. Connection Science 15(4), 151–190 (2003)

    Article  Google Scholar 

  11. Pfeifer, R.: On the role of embodiment in the emergence of cognition: Grey walter’s turtles and beyond. In: Proc. of the Workshop The Legacy of Grey Walter (2002)

    Google Scholar 

  12. Hülse, M., Wischmann, S., Pasemann, F.: The Role of Non-linearity for Evolved Multifunctional Robot Behavior. In: Moreno, J.M., Madrenas, J., Cosp, J. (eds.) ICES 2005. LNCS, vol. 3637, pp. 108–118. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  13. Braitenberg, V.: Vehicles: Experiments in Synthetic Psychology. MIT Press, Cambridge (1984)

    Google Scholar 

  14. Yamauchi, B.M., Beer, R.D.: Sequential behavior and learning in evolved dynamical neural networks. Adaptive Behaviour 2(3), 219–246 (1994)

    Article  Google Scholar 

  15. Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. Journal of Machine Learning Research 9, 937–965 (2008)

    MathSciNet  Google Scholar 

  16. Gomez, F. J.: Robust Non-Linear Control through Neuroevolution. PhD thesis, August 1, Tue, 6 Jan 104 19:10:41 GMT (2003)

    Google Scholar 

  17. Stanley, K.O.: Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines 8(2), 131–162 (2007)

    Article  MathSciNet  Google Scholar 

  18. Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artificial Life 15(2), 185–212 (2009)

    Article  Google Scholar 

  19. Koza, J.R., Rice, J.P.: Genetic generation of both the weights and architecture for a neural network. In: International Joint Conference on Neural Networks (1991)

    Google Scholar 

  20. Pasemann, F., Steinmetz, U., Hülse, M., Lara, B.: Robot control and the evolution of modular neurodynamics. Theory in Biosciences 120(3-4), 311–326 (2001)

    Google Scholar 

  21. Angeline, P.J., Saunders, G.M., Pollack, J.P.: An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks 5(1), 54–65 (1994)

    Article  Google Scholar 

  22. Gruau, F.: Neural Network Synthesis using Cellular Encoding and the Genetic Algorithm. Laboratoire de l’Informatique du Parallilisme, Ecole Normale Supirieure de Lyon (1994)

    Google Scholar 

  23. Hornby, G.S., Pollack, J.B.: Body-brain co-evolution using L-systems as a generative encoding. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W.B., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), July 7-11, pp. 868–875. Morgan Kaufmann, San Francisco (2001)

    Google Scholar 

  24. Cangelosi, A., Parisi, D., Nolfi, S.: Cell division and migration in a ’genotype’ for neural networks. Network: Computation in Neural Systems 5(4), 497–515 (1994)

    Article  MATH  Google Scholar 

  25. Belew, R.K.: Interposing an ontogenetic model between genetic algorithms and neural networks. In: Advances in Neural Information Processing Systems 5, NIPS Conference, p. 106. Morgan Kaufmann, San Francisco (1992)

    Google Scholar 

  26. Moriaty, D.E.: Symbiotic Evolution of Neural Networks in Sequential Decision Tasks. PhD thesis, The University of Texas at Austin, 1 (1997)

    Google Scholar 

  27. Clune, J., Beckmann, B.E., Pennock, R.T., Ofria, C.: HybrID: A hybridization of indirect and direct encodings for evolutionary computation. In: Kampis, G., Karsai, I., Szathmáry, E. (eds.) ECAL 2009, Part II. LNCS, vol. 5778, pp. 134–141. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  28. Inden, B.: Stepwise Transition from Direct Encoding to Artificial Ontogeny in Neuroevolution. In: Almeida e Costa, F., Rocha, L.M., Costa, E., Harvey, I., Coutinho, A. (eds.) ECAL 2007. LNCS (LNAI), vol. 4648, pp. 1182–1191. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  29. Doncieux, S., Meyer, J.-A.: Evolving modular neural networks to solve challenging control problems. In: Proceedings of The Fourth International ICSC Symposium on Engineering of Intelligent Systems (EIS 2004), Acta Press (2004)

    Google Scholar 

  30. Meyer, J.-A., Doncieux, S., David, Guillot, A.: Evolutionary approaches to neural control of rolling, walking, swimming and flying animats or robots. Biologically Inspired Robot Behavior Engineering, 1–43 (2003)

    Google Scholar 

  31. Jeffrey, L., Elman, J.L.: Learning and development in neural networks: The importance of starting small. Cognition 48, 71–99 (1993)

    Article  Google Scholar 

  32. Hülse, M., Wischmann, S., Pasemann, F.: Structure and function of evolved neuro-controllers for autonomous robots. Connection Science 16(4), 249–266 (2004)

    Article  Google Scholar 

  33. Stanley, K.O., Miikkulainen, R.P.: Efficient evolution of neural networks through complexification. PhD thesis, The University of Texas at Austin (2004)

    Google Scholar 

  34. Lee Giles, C., Omlin, C.W.: Pruning recurrent neural networks for improved generalization performance. IEEE Transactions on Neural Networks/A Publication of The IEEE Neural Networks Council 5(5), 848 (1994)

    Google Scholar 

  35. Bongard, J.C., Pfeifer, R.: Evolving complete agents using artificial ontogeny. Morpho-Functional Machines: The New Species Designing Embodied Intelligence, 237–258 (2003)

    Google Scholar 

  36. Nolfi, S., Parisi, D.: Evolving Artificial Neural Networks that Develop in Time. In: Proceedings of the Third European Conference on Advances in Artificial Life, p. 367. Springer, Heidelberg (1995)

    Google Scholar 

  37. Gruau, F.: Automatic definition of modular neural networks. Adaptive Behaviour 3(2), 151–183 (1995)

    Article  Google Scholar 

  38. Nolfi, N., Parisi, D.: Growing neural networks. Technical Report PCIA-91-15, Institute of Psychology (December 1991)

    Google Scholar 

  39. Pasemann, F.: Neuromodules: A dynamical systems approach to brain modelling. In: Herrmann, H.J., Wolf, D.E., Poppel, E. (eds.) Workshop on Supercomputing in Brain Research: From Tomography to Neural Networks, November 21-23. World Scientific Publishing Co., Germany (1995)

    Google Scholar 

  40. Horton, J.C., Adams, D.L.: The cortical column: a structure without a function. Philosophical Transactions of the Royal Society B: Biological Sciences 360(1456), 837 (2005)

    Article  Google Scholar 

  41. Reisinger, J., Stanley, K.O., Miikkulainen, R.: Evolving Reusable Neural Modules. In: Deb, K., Poli, R., Banzhaf, W., Beyer, H.-G., Burke, E., Darwen, P., Dasgupta, D., Floreano, D., Foster, J., Harman, M., Holland, O., Lanzi, P.L., Spector, L., Tettamanzi, A., Thierens, D., Tyrrell, A., et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 69–81. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  42. Valsalam, V.K., Miikkulainen, R.: Evolving symmetric and modular neural networks for distributed control. In: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 731–738. ACM, New York (2009)

    Chapter  Google Scholar 

  43. Gomez, F., Miikkulainen, R.: Incremental evolution of complex general behavior. Technical Report AI96-248, The University of Texas at Austin, Department of Computer Sciences, June 1, November 7, 106 21:26:08 GMT (1997)

    Google Scholar 

  44. Gauci, J., Stanley, K.O.: Generating large-scale neural networks through discovering geometric regularities. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, p. 1004. ACM, New York (2007)

    Google Scholar 

  45. David, B., D’Ambrosio, D.B., Stanley, K.O.: A novel generative encoding for exploiting neural network sensor and output geometry. In: Genetic and Evolutionary Computation Conference: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation. Association for Computing Machinery, Inc., New York (2007)

    Google Scholar 

  46. Dieckmann, U.: Coevolution as an autonomous learning strategy for neuromodules. In: Herrmann, H.J., Wolf, D.E., Poppel, E. (eds.) Workshop On Supercomputing In Brain Research: From Tomography To Neural Networks, November 21-23. World Scientific Publishing Co., Germany (1995)

    Google Scholar 

  47. Pasemann, F., Steinmetz, U., Dieckman, U.: Evolving structure and function of neurocontrollers. In: Angeline, P.J., Michalewicz, Z., Schoenauer, M., Yao, X., Zalzala, A. (eds.) Proceedings of the Congress on Evolutionary Computation, July 6-9, vol. 3, pp. 1973–1978. IEEE Press, USA (1999)

    Google Scholar 

  48. Moriarty, D.E., Miikkulainen, R.: Efficient reinforcement learning through symbiotic evolution. Technical Report AI94-224, The University of Texas at Austin, Department of Computer Sciences (September 1, 1994)

    Google Scholar 

  49. Christian, W., Rempis, C.W., Pasemann, F.: Search space restriction of neuro-evolution through constrained modularization of neural networks. In: Mandai, K. (ed.) Proceedings of the 6th International Workshop on Artificial Neural Networks and Intelligent Information Processing (ANNIIP), in Conjunction with ICINCO 2010, pp. 13–22. SciTePress, Portugal (2010)

    Google Scholar 

  50. Mahfoud, S.W.: Niching methods for genetic algorithms. Department of Computer Science, University of Illinois at Urbana-Champaign (1995)

    Google Scholar 

  51. Hancock, P.J.B.: An empirical comparison of selection methods in evolutionary algorithms. To appear in the Proceedings of the AISB Workshop on Evolutionary Computation, vol. 1 (1994)

    Google Scholar 

  52. Reil, T., Husbands, P.: Evolution of central pattern generators for bipedal walking in a real-time physics environment. IEEE Transactions on Evolutionary Computation 6(2), 159–168 (2002)

    Article  Google Scholar 

  53. Hein, D., Hild, M., Berger, R.: Evolution of biped walking using neural oscillators and physical simulation. In: Visser, U., Ribeiro, F., Ohashi, T., Dellaert, F. (eds.) RoboCup 2007: Robot Soccer World Cup XI. LNCS (LNAI), vol. 5001, pp. 433–440. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  54. Geng, T., Porr, B., Wörgötter, F.: A reflexive neural network for dynamic biped walking control. Neural computation 18(5), 1156–1196 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  55. Manoonpong, P., Geng, T., Kulvicius, T., Porr, B., Wörgötter, F.: Adaptive, Fast Walking in a Biped Robot under Neuronal Control and Learning. PLoS Computational Biology 3(7) (2007)

    Google Scholar 

  56. McHale, G., Husbands, P.: GasNets and other evolvable neural networks applied to bipedal locomotion. In: From Animals to Animats 8: Proceedings of the Seventh [ie Eighth] International Conference on Simulation of Adaptive Behavior, p. 163. The MIT Press, Cambridge (1994)

    Google Scholar 

  57. Ishiguro, A., Fujii, A., Hotz, P.E.: Neuromodulated control of bipedal locomotion using a polymorphic cpg circuit. Adaptive Behavior 11(1), 7 (2003)

    Article  Google Scholar 

  58. Hase, K., Yamazaki, N.: Computational evolution of human bipedal walking by a neuro-musculo-skeletal model. Artificial Life and Robotics 3(3), 133–138 (1999)

    Article  Google Scholar 

  59. Josh, C.: Making evolution an offer it can’t refuse: Morphology and the extradimensional bypass. Advances in Artificial Life, 401–412 (2001)

    Google Scholar 

  60. Cliff, D., Harvey, I., Husbands, P.: Incremental evolution of neural network architectures for adaptive behaviour. In: Proceedings of the First European Symposium on Artificial Neural Networks, ESANN 039, vol. 93, pp. 39–44. D facto Publishing (1992)

    Google Scholar 

  61. Pasemann, F.: Complex dynamics and the structure of small neural networks. Network: Computation in Neural Systems 13(2), 195–216 (2002)

    MATH  Google Scholar 

  62. Rempis, C., Thomas, V., Bachmann, F., Pasemann, F.: NERD Neurodynamics and Evolutionary Robotics Development Kit. In: Ando, N., Balakirsky, S., Hemker, T., Reggiani, M., von Stryk, O. (eds.) SIMPAR 2010. LNCS, vol. 6472, pp. 121–132. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  63. Pasemann, F.: Characterization of periodic attractors in neural ring networks. Neural Networks 8(3), 421–429 (1995)

    Article  Google Scholar 

  64. Pasemann, F., Hild, M., Zahedi, K.: So(2)-networks as neural oscillators. In: Computational Methods in Neural Modeling, vol. 2686, pp. 144–151 (2003)

    Google Scholar 

  65. Miller, B.L., Goldberg, D.E.: Genetic algorithms, tournament selection, and the effects of noise. Urbana 51, 61801 (1995)

    Google Scholar 

  66. Jakobi, N.: Evolutionary robotics and the radical envelope-of-noise hypothesis. Adaptive behavior 6(2), 325 (1997)

    Article  Google Scholar 

  67. Pollack, J.B., Lipson, H., Ficici, S., Funes, P., Hornby, G., Watson, R.A.: Evolutionary techniques in physical robotics. Evolvable Systems: from biology to hardware, 175–186 (2000)

    Google Scholar 

  68. Bongard, J.C., Lipson, H.: Automating Genetic Network Inference with Minimal Physical Experimentation Using Coevolution. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3102, pp. 333–345. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  69. von Twickel, A., Pasemann, F.: Reflex-oscillations in evolved single leg neurocontrollers for walking machines. Natural Computing 6(3), 311–337 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  70. Rempis, C.W.: Short-term memory structures in additive recurrent neural networks. Master’s thesis, University of Applied Sciences Bonn-Rhein-Sieg, Germany (2007)

    Google Scholar 

  71. Wischmann, S., Pasemann, F.: The emergence of communication by evolving dynamical systems. From Animals to Animats 9, 777–788 (2006)

    Article  Google Scholar 

  72. Sidel, T., Hild, M., Weidner, M.: Concept and Design of the Modular Actuator System for the Humanoid Robot MYON. In: International Conference on Intelligent and Applications, ICIRA 2011 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Neurocybernetics Group, Institute of Cognitive Science, University of Osnabrück, 49069, Osnabrück, Germany

    Christian Rempis & Frank Pasemann

Authors
  1. Christian Rempis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frank Pasemann
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of ICT, Swinburne University of Technology, 3122, Melbourne, VIC, Australia

    Raymond Chiong

  2. Nature Inspired Computation and Applications Laboratory School of Computer Science and Technology, University of Science and Technology of China (USTC), 230027, Hefei, Anhui, China

    Thomas Weise

  3. School of Computer Science, University of Adelaide, 5005, Adelaide, SA, Australia

    Zbigniew Michalewicz

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Rempis, C., Pasemann, F. (2012). An Interactively Constrained Neuro-Evolution Approach for Behavior Control of Complex Robots. In: Chiong, R., Weise, T., Michalewicz, Z. (eds) Variants of Evolutionary Algorithms for Real-World Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23424-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-23424-8_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-23423-1

  • Online ISBN: 978-3-642-23424-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation