Incremental Skill Acquisition for Self-motivated Learning Animats
A central role in the development process of children is played by self-exploratory activities. Through a playful interaction with the surrounding environment, they test their own capabilities, explore novel situations, and understand how their actions affect the world. During this kind of exploration, interesting situations may be discovered. By learning to reach these situations, a child incrementally develops more and more complex skills. Inspired by studies from psychology, neuroscience, and machine learning, we designed SMILe (Self-Motivated Incremental Learning), a learning framework that allows artificial agents to autonomously identify and learn a set of abilities useful to face several different tasks, through an iterated three phase process: by means of a random exploration of the environment (babbling phase), the agent identifies interesting situations and generates an intrinsic motivation (motivating phase) aimed at learning how to get into these situations (skill acquisition phase). This process incrementally increases the skills of the agent, so that new interesting configurations can be experienced. We present results on two gridworld environments to show how SMILe makes it possible to learn skills that enable the agent to perform well and robustly in many different tasks.
KeywordsIntrinsic Motivation Reinforcement Learn Motivate Learning Skill Acquisition Reward Function
Unable to display preview. Download preview PDF.
- 2.Barto, A.G., Singh, S., Chentanez, N.: Intrinsically motivated learning of hierarchical collections of skills. In: Proceedings of ICDL (2004)Google Scholar
- 4.Bonarini, A., Lazaric, A., Restelli, M.: Smile: Self-motivated incremental learning. Technical report, Politecnico di Milano (2006), www.airlab.elet.polimi.it/papers/bonarini06smile.pdf
- 8.Marshall, J., Blank, D., Meeden, L.: An emergent framework for self-motivation in developmental robotics. In: Proceedings of ICDL (2004)Google Scholar
- 9.McGovern, A., Barto, A.G.: Automatic discovery of subgoals in reinforcement learning using diverse density. In: Proceedings of ICML (2001)Google Scholar
- 12.Oudeyer, P.-Y., Kaplan, F., Hafner, V.: The playground experiment: Task-independent development of a curious robot. In: AAAI Spring Symposium Workshop on Developmental Robotics (2005)Google Scholar
- 14.Ratitch, B., Precup, D.: Using mdp characteristics to guide exploration in reinforcement learning. In: European Conference on Reinforcement Learning (2003)Google Scholar
- 15.Schmidhuber, J.: Self-motivated development through rewards for predictor errors / improvements. In: AAAI Spring Symposium on Developmental Robotics (2005)Google Scholar
- 16.Stout, A., Konidaris, G., Barto, A.: Intrinsically motivated reinforcement learning: A promising framework for developmental robot learning. In: AAAI Spring Symposium on Developmental Robotics (2005)Google Scholar
- 17.Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
- 19.Uchibe, E., Doya, K.: Reinforcement learning with multiple heterogeneous modules: A framework for developmental robot learning. In: Proceedings of ICDL (2005)Google Scholar
- 21.Weng, J., Zhang, Y.: Novelty and reinforcement learning in the value system of developmental robots. In: International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems (2002)Google Scholar