Motivational Engine with Sub-goal Identification in Neuroevolution Based Cognitive Robotics
A first approach towards a new motivational system for an autonomous robot that can learn chains of sub-goals leading to a final reward is proposed in this paper. The motivational system provides the motivation that guides the robot operation according to its knowledge of its sensorial space so that rewards are maximized during its lifetime. In order to do this, a motivational engine progressively and interactively creates an internal model of expected future reward (value function) for areas of the robot’s state space, through a neuroevolutionary process, over samples obtained in the sensorial (state space) traces followed by the robot whenever it obtained a reward. To improve this modelling process, a strategy is proposed to decompose the global value function leading to the reward or goal into several more local ones, thus discovering sub-goals that simplify the whole learning process and that can be reused in the future. The motivational engine is tested in a simulated experiment with very promising results.
KeywordsNeuroevolution Cognitive systems Motivation Autonomous robots
This work was partially funded by the EU’s H2020 research and innovation programme under grant agreement No 640891 (DREAM project) and by the Xunta de Galicia and European Regional Development Funds under grants GRC 2013-050 and redTEIC network (R2014/037).
- 2.Scott, P.D., Markovitch, S.: Learning novel domains through curiosity and conjecture. In: Proceedings IJCAI 1989, pp. 669–674 (1989)Google Scholar
- 3.Shmidhuber, J.: Adaptive confidence and adaptive curiosity. Technical Report. Institut fur Informatik, Technische Universität Munchen (1991)Google Scholar
- 4.Lenat, D.B.: AM: an artificial intelligence approach to discovery in mathematics as heuristic search, Doctoral Dissertation No. STAN-CS-76-570, Department of Computer Science, Stanford University (1976)Google Scholar
- 5.Sutton, R.S.: Reinforcement learning architectures for animals. In: From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, pp. 288–296 (1991)Google Scholar
- 6.Oudeyer, P.Y., Kaplan, F.: What is intrinsic motivation? a typology of computational approaches. Front. Neurorobot. 1(A.6), 1–14 (2007)Google Scholar
- 7.McGovern, A., Barto, A.: Automatic discovery of subgoals in reinforcement learning using diverse density. Technical report of the faculty publication series, Computer Science Department, University of Massachusetts, Amherst (2001)Google Scholar
- 9.Simsek, O., Barto, A.: Betweenness centrality as a basis for forming skills. Technical report, Department of Computer Science, University of Massachusetts (2007)Google Scholar
- 10.Mannor, S., Menache, I., Hoze, A., Klein, U.: Dynamic abstraction in reinforcement learning via clustering. In: Proceedings of the Twenty-First International Conference on Machine Learning, pp. 560–567. ACM (2004)Google Scholar
- 11.Konidaris, G., Barto, A.: Sensorimotor abstraction selection for efficient, autonomous robot skill acquisition. In: 7th IEEE International Conference on Development and Learning, ICDL 2008, pp. 151–156, (2008)Google Scholar