Abstract
A first approach towards a new motivational system for an autonomous robot that can learn chains of sub-goals leading to a final reward is proposed in this paper. The motivational system provides the motivation that guides the robot operation according to its knowledge of its sensorial space so that rewards are maximized during its lifetime. In order to do this, a motivational engine progressively and interactively creates an internal model of expected future reward (value function) for areas of the robot’s state space, through a neuroevolutionary process, over samples obtained in the sensorial (state space) traces followed by the robot whenever it obtained a reward. To improve this modelling process, a strategy is proposed to decompose the global value function leading to the reward or goal into several more local ones, thus discovering sub-goals that simplify the whole learning process and that can be reused in the future. The motivational engine is tested in a simulated experiment with very promising results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ryan, R.M., Deci, E.L.: Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp. Educ. Psychol. 25, 54–67 (2000)
Scott, P.D., Markovitch, S.: Learning novel domains through curiosity and conjecture. In: Proceedings IJCAI 1989, pp. 669–674 (1989)
Shmidhuber, J.: Adaptive confidence and adaptive curiosity. Technical Report. Institut fur Informatik, Technische Universität Munchen (1991)
Lenat, D.B.: AM: an artificial intelligence approach to discovery in mathematics as heuristic search, Doctoral Dissertation No. STAN-CS-76-570, Department of Computer Science, Stanford University (1976)
Sutton, R.S.: Reinforcement learning architectures for animals. In: From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, pp. 288–296 (1991)
Oudeyer, P.Y., Kaplan, F.: What is intrinsic motivation? a typology of computational approaches. Front. Neurorobot. 1(A.6), 1–14 (2007)
McGovern, A., Barto, A.: Automatic discovery of subgoals in reinforcement learning using diverse density. Technical report of the faculty publication series, Computer Science Department, University of Massachusetts, Amherst (2001)
Menache, I., Mannor, S., Shimkin, N.: Q-cut - dynamic discovery of sub-goals in reinforcement learning. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 295–306. Springer, Heidelberg (2002)
Simsek, O., Barto, A.: Betweenness centrality as a basis for forming skills. Technical report, Department of Computer Science, University of Massachusetts (2007)
Mannor, S., Menache, I., Hoze, A., Klein, U.: Dynamic abstraction in reinforcement learning via clustering. In: Proceedings of the Twenty-First International Conference on Machine Learning, pp. 560–567. ACM (2004)
Konidaris, G., Barto, A.: Sensorimotor abstraction selection for efficient, autonomous robot skill acquisition. In: 7th IEEE International Conference on Development and Learning, ICDL 2008, pp. 151–156, (2008)
Salge, C., Glackin, C., Polani, D.: Approximation of empowerment in the continuous domain. Adv. Complex Syst. 16(1–2), 1250079 (2013)
Bellas, F., Duro, R.J., Faina, A., Souto, D.: Multilevel Darwinist Brain (MDB): artificial evolution in a cognitive architecture for real robots. IEEE Trans. Auton. Ment. Dev. 2(4), 340–354 (2010)
Acknowledgements
This work was partially funded by the EU’s H2020 research and innovation programme under grant agreement No 640891 (DREAM project) and by the Xunta de Galicia and European Regional Development Funds under grants GRC 2013-050 and redTEIC network (R2014/037).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Salgado, R., Prieto, A., Caamaño, P., Bellas, F., Duro, R.J. (2016). Motivational Engine with Sub-goal Identification in Neuroevolution Based Cognitive Robotics. In: Martínez-Álvarez, F., Troncoso, A., Quintián, H., Corchado, E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2016. Lecture Notes in Computer Science(), vol 9648. Springer, Cham. https://doi.org/10.1007/978-3-319-32034-2_55
Download citation
DOI: https://doi.org/10.1007/978-3-319-32034-2_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-32033-5
Online ISBN: 978-3-319-32034-2
eBook Packages: Computer ScienceComputer Science (R0)