Abstract
Deriving a successful neural control of behavior of autonomous and embodied systems poses a great challenge. The difficulty lies in finding suitable learning mechanisms, and in specifying under what conditions learning becomes necessary. Here, we provide a solution to the second issue in the form of an additional feedback loop that augments the sensorimotor loop in which autonomous systems live. The second feedback loop provides proprioceptive signals, allowing the assessment of behavior through self-monitoring, and accordingly, the control of learning. We show how the behaviors can be defined with the aid of this framework, and we show that, in combination with simple stochastic plasticity mechanisms, behaviors are successfully learned.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dean, J.: Animats and what they can tell us. Trends Cogn. Sci. 2(2), 60–67 (1998)
Meyer, J.A.: The animat approach to cognitive science. In: Roitblat, H. Meyer, J.A. (eds.) Comparative Approaches to Cognitive Science, pp. 27–44. The MIT Press/Bradford Books (1995)
Meyer, J.A., Guillot, A.: Biologically inspired robots. In: Siciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, pp. 1395–1422. Springer (2008)
Pfeifer, R., Bongard, J.: How the body shapes the way we think: a new view of intelligence. MIT press (2007)
Ross Ashby, W.: Design for a brain: the origin of adaptive behavior (2nd edn). Chapman and Hall, London UK (1960)
Di Paolo, E.A.: Organismically-inspired robotics: homeostatic adaptation and teleology beyond the closed sensorimotor loop. In: Murase, K., Asakura, T. (eds.) Dynamical Systems Approach to Embodiment and Sociality, pp. 19–42. Advanced Knowledge International, Adelaide, Australia (2003)
Ziemke, Tom: The embodied self: theories, hunches and robot models. J. Conscious. Stud. 14(7), 167–179 (2007)
Ikegami, T., Suzuki, K.: From a homeostatic to a homeodynamic self. BioSystems 91(2), 388–400 (2008)
Der, R.: Artificial life from the principle of homeokinesis. In: Proceedings of the German Workshop on Artificial Life (2008)
Hebb. D.O.: The Organization of Behavior. Wiley, New York (1949)
Cooper, L.N., Intrator, N., Blais, B.S., Shouval, H.Z.: Theory of Cortical Plasticity. World Scientific (2004)
Turrigiano, G.G., Nelson, S.B.: Homeostatic plasticity in the developing nervous system. Nat. Rev. Neurosci. 5(2), 97–107 (2004)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(9), 533–536 (1986)
Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79(8), 2554–2558 (1982)
Jennings, H.S.: Contributions to the study of the behavior of lower organisms. Number 16, Carnegie institution of Washington (1904)
Smith, T., Husbands, P., Philippides, A., O’Shea, M.: Neuronal plasticity and temporal adaptivity: Gasnet robot control networks. Adapt. Behav. 10(3–4), 161–183 (2002)
Timmis, J., Neal, M., Thorniley, J.: An adaptive neuro-endocrine system for robotic systems. In: Proceedings of the IEEE Workshop on Robotic Intelligence in Informationally Structured Space, RIISS’09, pp. 129–136 (2009)
Moioli, R.C., Vargas, P.A., Husbands, P.: A multiple hormone approach to the homeostatic control of conflicting behaviours in an autonomous mobile robot. In: Proceedings of IEEE Congress on Evolutionary Computation, CEC’09, pp. 47–54 (2009)
Rempis, C., Thomas, V., Bachmann, F., Pasemann, F.: NERD—Neurodynamics and Evolutionary Robotics Development Kit. In: Simulation, Modeling, and Programming for Autonomous Robots, pp. 121–132. Springer (2010)
Acknowledgments
This research was partially funded by the German Research Foundation (DFG) priority program 1527. The contribution of Christian Rempis to this project is gratefully acknowledged. The authors thank Josef Behr, Andrea Suckro, and Florian Ziegler for testing and refining the simulation models in the NERD Toolkit, and particularly the latter for his role in the current study. Thanks to Kevin Koschmieder for implementing the modulated Gaussian walk.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Toutounji, H., Pasemann, F. (2016). Autonomous Learning Needs a Second Environmental Feedback Loop. In: Madani, K., Dourado, A., Rosa, A., Filipe, J., Kacprzyk, J. (eds) Computational Intelligence. IJCCI 2013. Studies in Computational Intelligence, vol 613. Springer, Cham. https://doi.org/10.1007/978-3-319-23392-5_25
Download citation
DOI: https://doi.org/10.1007/978-3-319-23392-5_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23391-8
Online ISBN: 978-3-319-23392-5
eBook Packages: EngineeringEngineering (R0)