Abstract
In this paper, the processes of exploration and of incremental learning in the robot navigation task are studied using the dynamical systems approach. A neural network model which performs the forward modeling, planning, consolidation learning and novelty rewarding is used for the robot experiments. Our experiments showed that the robot repeated a few variation of travel patterns in the beginning of the exploration, and later the robot explored more diversely in the workspace by combining and mutating the previously experienced patterns. Our analysis indicates that internal confusion due to immature learning plays the role of a catalyst in generating diverse action sequences. It is found that these diverse exploratory travels enable the robot to acquire the rational modeling of the environment in the end.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R.D. Beer. A dynamical systems perspective on agent-environment interaction. Artificial Intelligence, Vol. 72, No. 1, pp. 173–215, 1995.
J.H. Holland and J.S. Reitman. Cognitive systems based on adaptive algorithms. In D.A. Watermann and F. Hayes-Roth, editors, Pattern Directed Inference Systems. New York: Academic Press, 1978.
M.I. Jordan and D.E. Rumelhart. Forward models: supervised learning with a distal teacher. Cognitive Science, Vol. 16, pp. 307–354, 1992.
J.B. Pollack. The induction of dynamical recognizers. Machine Learning, Vol. 7, pp. 227–252, 1991.
D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations by error propagation. In D.E. Rumelhart and J.L. Mclelland, editors, Parallel Distributed Processing. Cambridge, MA: MIT Press, 1986.
J. Schmidhuber. A possiblity for implementing curiosity and boredom in model-building neural controllers. In J.A. Meyer and S.W. Wilson, editors, From Animals to Animats: Proc. of the First, International Conference on Simulation of Adaptive Behavior, pp. 222–227. Cambridge, MA: MIT press, 1991.
L.R. Squire, N.J. Cohen, and L. Nadel. The medial temporal region and memory consolidation: A new hypothesis. In H. Weingartner and E. Parker, editors, Memory consolidation, pp. 185–210. Erlbaum, Hillsdale, N.J., 1984.
J. Tani. Model-Based Learning for Mobile Robot Navigation from the Dynamical Systems Perspective. IEEE Trans. on SMC (B), Vol. 26, No. 3, pp. 421–436, 1996.
J. Tani. An interpretation of the “self” from the dynamical systems perspective: a constructivist approach. Journal of Consciousness Studies, Vol. 5, No. 5–6, pp. 516–42, 1998.
S.B. Thrun and Knut Moller. Active exploration in dynamic environments. In in Proc. of NIPS 4, pp. 531–538. 1990.
P. Werbos. A menu of designs for reinforcement learning over time. In W.T. Miller, R.S. Sutton, and P.J. Werbos, editors, Neural Networks for Control, pp. 67–95. MIT Press, Boston, MA, 1990.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tani, J., Sugita, Y. (1999). On the Dynamics of Robot Exploration Learning. In: Floreano, D., Nicoud, JD., Mondada, F. (eds) Advances in Artificial Life. ECAL 1999. Lecture Notes in Computer Science(), vol 1674. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48304-7_36
Download citation
DOI: https://doi.org/10.1007/3-540-48304-7_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-66452-9
Online ISBN: 978-3-540-48304-5
eBook Packages: Springer Book Archive