Evolving a Single Scalable Controller for an Octopus Arm with a Variable Number of Segments

  • Brian G. Woolley
  • Kenneth O. Stanley
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6239)


While traditional approaches to machine learning are sensitive to high-dimensional state and action spaces, this paper demonstrates how an indirectly encoded neurocontroller for a simulated octopus arm leverages regularities and domain geometry to capture underlying motion principles and sidestep the superficial trap of dimensionality. In particular, controllers are evolved for arms with 8, 10, 12, 14, and 16 segments in equivalent time. Furthermore, when transferred without further training, solutions evolved on smaller arms retain the fundamental motion model because they simply extend the general kinematic concepts discovered at the original size. Thus this work demonstrates that dimensionality can be a false measure of domain complexity and that indirect encoding makes it possible to shift the focus to the underlying conceptual challenge.


Action Space Contractive Pattern Ventral Muscle Domain Geometry Reinforcement Learning Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bongard, J.C., Pfeifer, R.: Evolving complete agents using artificial ontogeny. In: Morpho-functional Machines: The New Species (Designing Embodied Intelligence), pp. 237–258. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  2. 2.
    Clune, J., Beckmann, B.E., Ofria, C., Pennock, R.T.: Evolving coordinated quadruped gaits with the hyperneat generative encoding. In: Proceedings of the IEEE Congress on Evolutionary Computing Special Section on Evolutionary Robotics. IEEE Press, Los Alamitos (May 2009)Google Scholar
  3. 3.
    D’Ambrosio, D.B., Stanley, K.O.: A novel generative encoding for exploiting neural network sensor and output geometry. In: Proceedings of the 9th annual conference on Genetic and Evolutionary Computation, pp. 974–981. ACM, New York (2007)CrossRefGoogle Scholar
  4. 4.
    Engel, Y., Mannor, S., Meir, R.: Bayes meets Bellman: The Gaussian process approach to temporal difference learning. In: Proceedings of the 20th International Conference on Machine Learning, vol. 20, pp. 154–161. AAAI Press, Menlo Park (2003)Google Scholar
  5. 5.
    Engel, Y., Szabo, P., Volkinshtein, D.: Learning to control an octopus arm with gaussian process temporal difference methods. Advances in Neural Information Processing Systems 18, 347–354 (2006)Google Scholar
  6. 6.
    Floreano, D., Dürr, P., Mattiussi, C.: Neuroevolution: from architectures to learning. Evolutionary Intelligence 1(1), 47–62 (2008)CrossRefGoogle Scholar
  7. 7.
    Gauci, J., Stanley, K.O.: Generating large-scale neural networks through discovering geometric regularities. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, pp. 997–1004. ACM, New York (2007)CrossRefGoogle Scholar
  8. 8.
    Gauci, J., Stanley, K.O.: Autonomous evolution of topographic regularities in artificial neural networks. Neural Computation, 38 (2010) (to appear)Google Scholar
  9. 9.
    Hornby, G.S., Pollack, J.B.: Creating high-level components with a generative representation for body-brain evolution. Artificial Life 8(3), 223–246 (2002)CrossRefGoogle Scholar
  10. 10.
    James, D., Tucker, P.: ANJI homepage (2004),
  11. 11.
    Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4(1), 102–138 (1996)Google Scholar
  12. 12.
    Kier, W.M., Smith, K.K.: Tongues, tentacles and trunks: the biomechanics of movement in muscular-hydrostats. Zoological Journal of the Linnean Society 83, 307–324 (1985)CrossRefGoogle Scholar
  13. 13.
    Stanley, K.O.: Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines 8(2), 131–162 (2007)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based indirect encoding for evolving large-scale neural networks. Artificial Life 15(2) (2009)Google Scholar
  15. 15.
    Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 10(2), 99–127 (2002)CrossRefGoogle Scholar
  16. 16.
    Stanley, K.O., Miikkulainen, R.: A taxonomy for artificial embryogeny. Artificial Life 9(2), 93–130 (2003)CrossRefGoogle Scholar
  17. 17.
    Stanley, K.O., Miikkulainen, R.: Competitive coevolution through evolutionary complexification. Journal of Artificial Intelligence Research 21(1), 63–100 (2004)Google Scholar
  18. 18.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: an introduction. MIT Press, Cambridge (1998)Google Scholar
  19. 19.
    Yao, X.: Evolving artificial neural networks. Proc. of the IEEE 87(9), 1423–1447 (1999)CrossRefGoogle Scholar
  20. 20.
    Yekutieli, Y., Sagiv-Zohar, R., Aharonov, R., Engel, Y., Hochner, B., Flash, T.: Dynamic model of the octopus arm. I. Biomechanics of the octopus reaching movement. Journal of Neurophysiology 94(2), 1443–1506 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Brian G. Woolley
    • 1
  • Kenneth O. Stanley
    • 1
  1. 1.School of Electrical Engineering and Computer ScienceUniversity of Central FloridaOrlando

Personalised recommendations