Abstract
Children seem intrinsically motivated to manipulate, to explore, to test, to learn and they look for activities and situations that provide such learning opportunities. Inspired by research in developmental psychology and neuroscience, some researchers have started to address the problem of designing intrinsic motivation systems. A robot controlled by such systems is able to autonomously explore its environment not to fulfil predefined tasks but driven by an incentive to search for situations where learning happens efficiently. In this paper, we present the origins of these intrinsically motivated machines, our own research in this novel field and we argue that intrinsic motivation might be a crucial step towards machines capable of life-long learning and open-ended development.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barto, A., Singh, S., Chentanez, N.: Intrinsically motivated learning of hierarchical collections of skills. In: ICDL 2004. Proceedings of the 3rd International Conference on Development and Learning, Salk Institute, San Diego (2004)
Berlyne, D.: Conflict, Arousal and Curiosity. McGraw-Hill, New York (1960)
Cohn, D., Ghahramani, Z., Jordan, M.: Active learning with statistical models. Journal of artificial intelligence research 4, 129–145 (1996)
Csikszenthmihalyi, M.: Flow-the psychology of optimal experience. Harper Perennial (1991)
Dayan, P., Belleine, W.: Reward, motivation and reinforcement learning. Neuron 36, 285–298 (2002)
De Charms, R.: Personal causation: the internal affective determinants of behavior. Academic Press, New York (1968)
Deci, E., Ryan, R.: Intrinsic Motivation and Self-Determination in Human Behavior. Plenum Press, New York (1985)
Dember, W.N., Earl, R.W.: Analysis of exploratory, manipulatory and curiosity behaviors. Psychological Review 64, 91–96 (1957)
Drescher, G.L.: Made-up minds. MIT Press, Cambridge, MA (1991)
Fedorov, V.: Theory of Optimal Experiment. Academic Press, New York (1972)
Festinger, L.: A theory of cognitive dissonance. Evanston, Row, Peterson (1957)
Harlow, H.: Learning and satiation of response in intrinsically motivated complex puzzle performances by monkeys. Journal of Comparative and Physiological Psychology 43, 289–294 (1950)
Hebb, D.O.: Drives and the c.n.s (conceptual nervous system). Psychological review 62, 243–254 (1955)
Horvitz, J.-C.: Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96(4), 651–656 (2000)
Huang, X., Weng, J.: Novelty and reinforcement learning in the value system of developmental robots. In: Prince, C., Demiris, Y., Marom, Y., Kozima, H., Balkenius, C. (eds.) Proceedings of the 2nd international workshop on Epigenetic Robotics: Modeling cognitive development in robotic systems, Lund University Cognitive Studies 94, pp. 47–55 (2002)
Hull, C.L.: Principles of behavior: an introduction to behavior theory. Appleton-Century-Croft, New York (1943)
Hunt, J.M.: Intrinsic motivation and its role in psychological development. Nebraska symposium on motivation 13, 189–282 (1965)
Kagan, J.: Motives and development. Journal of Personality and Social Psychology 22, 51–66 (1972)
Kakade, S., Dayan, P.: Dopamine: Generalization and bonuses. Neural Networks 15, 549–559 (2002)
Kaplan, F., Oudeyer, P.-Y.: The progress-drive hypothesis: an interpretation of early imitation. In: Nehaniv, C., Dautenhahn, K. (eds.) Models and mechanisms of imitation and social learning: Behavioural, social and communication dimensions, pp. 361–377. Cambridge University Press, Cambridge (2007a)
Kaplan, F., Oudeyer, P.-Y.: Un robot motivé pour apprendre: le role des motivations intrinseques dans le development sensorimoteur. Enfance 59(1), 46–58 (2007b)
Marshall, J., Blank, D., Meeden, L.: An emergent framework for self-motivation in developmental robotics. In: ICDL 2004. Proceedings of the 3rd International Conference on Development and Learning, Salk Institute, San Diego (2004)
Minsky, M.: A framework for representing knowledge. In: Wiston, P. (ed.) The psychology of computer vision, pp. 211–277. Mc Graw Hill, New York (1975)
Montgomery, K.: The role of exploratory drive in learning. Journal of Comparative and Physiological Psychology 47, 60–64 (1954)
Oudeyer, P.-Y., Kaplan, F.: Discovering communication. Connection Science 18(2), 189–206 (2006)
Oudeyer, P.-Y., Kaplan, F., Hafner, V.: Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation 11(1), 265–286 (2007)
Panksepp, J.: Affective neuroscience: the foundations of human and animal emotions. Oxford University Press, Oxford (1998)
Piaget, J.: The origins of intelligence in children. Norton, New York (1952)
Schank, R., Abelson, R.: Scripts, plans, goals and understanding: An inquiry into human knowledge structures. Lawrence Erlbaum Associates, Hillsdale, NJ (1977)
Schmidhuber, J.: Curious model-building control systems. In: Proceeding International Joint Conference on Neural Networks, vol. 2, pp. 1458–1463. IEEE, Singapore (1991)
Steels, L.: The autotelic principle. In: Iida, F., Pfeifer, R., Steels, L., Kuniyoshi, Y. (eds.) Embodied Artificial Intelligence. LNCS (LNAI), vol. 3139, pp. 231–242. Springer, Heidelberg (2004)
Sutton, R., Barto, A.: Reinforcement learning: an introduction. MIT Press, Cambridge, MA (1998)
Sutton, R., Precup, D., Singh, S.: Between mdpss and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112, 181–211 (1999)
Thrun, S., Pratt, L.: Learning to Learn. Kluwer Academic Publishers, Dordrecht (1998)
Turing, A.: Computing machinery and intelligence. Mind 59, 433–460 (1950)
White, R.: Motivation reconsidered: The concept of competence. Psychological review 66, 297–333 (1959)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Kaplan, F., Oudeyer, PY. (2007). Intrinsically Motivated Machines. In: Lungarella, M., Iida, F., Bongard, J., Pfeifer, R. (eds) 50 Years of Artificial Intelligence. Lecture Notes in Computer Science(), vol 4850. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77296-5_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-77296-5_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77295-8
Online ISBN: 978-3-540-77296-5
eBook Packages: Computer ScienceComputer Science (R0)