Skip to main content
SpringerLink
Log in

Intrinsically Motivated Learning of Real-World Sensorimotor Skills with Developmental Constraints

  • Chapter
  • First Online:
Intrinsically Motivated Learning in Natural and Artificial Systems

Abstract

Open-ended exploration and learning in the real world is a major challenge of developmental robotics. Three properties of real-world sensorimotor spaces provide important conceptual and technical challenges: unlearnability, high dimensionality, and unboundedness. In this chapter, we argue that exploration in such spaces needs to be constrained and guided by several combined developmental mechanisms. While intrinsic motivation, that is, curiosity-driven learning, is a key mechanism to address this challenge, it has to be complemented and integrated with other developmental constraints, in particular: sensorimotor primitives and embodiment, task space representations, maturational processes (i.e., adaptive changes of the embodied sensorimotor apparatus), and social guidance. We illustrate and discuss the potential of such an integration of developmental mechanisms in several robot learning experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Part of the material presented in this section is adapted from Baranes and Oudeyer (2010a, 2012).

  2. 2.

    Part of the material presented in this section is adapted from Baranes and Oudeyer (2011).

References

  • Angluin, D.: Queries and concept learning. Mach. Learn. 2, 319–342 (1988)

    MathSciNet  Google Scholar 

  • Asada, M., Hosoda, K., Kuniyoshi, Y., Ishiguro, H., Inui, T., Yoshikawa, Y., Ogino, M., Yoshida, C.: Cognitive developmental robotics: A survey. IEEE Trans. Auton. Mental Dev. 1(1), 12–34 (2009)

    Google Scholar 

  • Bakker, B., Schmidhuber, J.: Hierarchical reinforcement learning based on subgoal discovery and subpolicy specialization. In: Proceedings of the 8th Conference on Intelligent Autonomous Systems (IAS-8) (2004)

    Google Scholar 

  • Baldassarre, G., Mirolli, M.: Temporal-difference competence-based intrinsic motivation (TD-CB-IM): A mechanism that uses the td-error as an intrinsic reinforcement for deciding which skill to learn when. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 255–276. Springer, Berlin (2012)

    Google Scholar 

  • Ball, P.: The Self-made Tapestry-Pattern formation in nature. Oxford University Press, New York (1999)

    MATH  Google Scholar 

  • Baranes, A., Oudeyer, P.-Y.: Riac: Robust intrinsically motivated exploration and active learning. IEEE Trans. Auton. Mental Dev. 1(3), 155–169 (2009)

    Google Scholar 

  • Baranes, A., Oudeyer, P.-Y.: Intrinsically motivated goal exploration for active motor learning in robots: A case study. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2010) (2010a)

    Google Scholar 

  • Baranes, A., Oudeyer, P.-Y.: Maturationally constrained competence-based intrinsically motivated learning. In: Proceedings of IEEE International Conference on Development and Learning (ICDL 2010) (2010b)

    Google Scholar 

  • Baranes, A., Oudeyer, P.-Y.: The interaction of maturational constraints and intrinsic motivations in active motor development. In: Proceedings of IEEE ICDL-Epirob 2011 (2011)

    Google Scholar 

  • Baranes, A., Oudeyer, P-Y.: Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots, Robotics and Autonomous Systems, http://dx.doi.org/10.1016/j.robot.2012.05.008, (2012)

  • Barto, A., Singh, S., Chenatez, N.: Intrinsically motivated learning of hierarchical collections of skills. In: Proceedings of the 3rd International Conference Development Learning, San Diego, pp. 112–119 (2004)

    Google Scholar 

  • Barto, A.G.: Intrinsic motivation and reinforcement learning. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 17–47. Springer, Berlin (2012)

    Google Scholar 

  • Berk, L.: Child Development. Allyn and Bacon, Boston (2008)

    Google Scholar 

  • Berlyne, D.: Conflict, Arousal and Curiosity. McGraw-Hill, New York (1960)

    Google Scholar 

  • Berthier, N.E., Clifton, R., McCall, D., Robin, D.: Proximodistal structure of early reaching in human infants. Exp. Brain Res. 127(3), 259–269 (1999)

    Google Scholar 

  • Billard, A., Calinon, S., Dillmann, R., Schaal, S.: Robot programming by demonstration. In: Handbook of Robotics. Springer, Berlin (2008)

    Google Scholar 

  • Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)

    MATH  Google Scholar 

  • Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, Berlin (1st ed., 2006/corr. 2nd printing edition, 2007)

    Google Scholar 

  • Bjorklund, D.: The role of immaturity in human development. Psychol. Bull. 122(2), 153–169 (1997)

    Google Scholar 

  • Blank, D., Kumar, D., Meeden, L., Marshall, J.: Bringing up robot: Fundamental mechanisms for creating a self-motivated, self-organizing architecture. Cybern. Syst. 36(2), 125–150 (2002)

    MATH  Google Scholar 

  • Brafman, R., Tennenholtz, M.: R-max: A general polynomial time algorithm for near-optimal reinforcement learning. In: Proceedings of IJCAI’01 (2001)

    Google Scholar 

  • Bremner, J., Slater, A. (eds.): Theories of Infant Development. Blackwell, Cambridge (2003)

    Google Scholar 

  • Bronson, G.: The postnatal growth of visual capacity. Child. Dev. 45(4), 873–890 (1974)

    Google Scholar 

  • Calinon, S., Guenter, F., Billard, A.: On learning, representing and generalizing a task in a humanoid robot. IEEE Trans. Syst. Man Cybern. B, 37(2), 286–298 (2007)

    Google Scholar 

  • Castro, R., Novak, R.: Minimax bounds for active learning. IEEE Trans. Inform. Theory 54, 151–156 (2008)

    MathSciNet  Google Scholar 

  • Cazalets, J., Borde, M., Clarac, F.: Localization and organization of the central pattern generator for hindlimb locomotion in newborn rat. J. Neurosci. 15, 4943–4951 (1995)

    Google Scholar 

  • Cederborg, T., Ming, L., Baranes, A., Oudeyer, P.-Y.: Incremental local online gaussian mixture regression for imitation learning of multiple tasks. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2010) (2010)

    Google Scholar 

  • Chaloner, K., Verdinelli, I.: Bayesian experimental design: A review. J. Stat. Sci. 10, 273–304 (1995)

    MathSciNet  MATH  Google Scholar 

  • Chung, W., Fu, L.-C., Hsu, S.-H.: Motion control. In: Handbook of Robotics, pp. 133–159. Springer, Berlin (2008)

    Google Scholar 

  • Cohn, D., Atlas, L., Ladner, R.: Improving generalization with active learning. Mach. Learn. 15(2), 201–221 (1994)

    Google Scholar 

  • Cohn, D., Ghahramani, Z., Jordan, M.: Active learning with statistical models. J. Artif. Intell. Res. 4, 129–145 (1996)

    MATH  Google Scholar 

  • Csikszentmihalyi, M.: Creativity-Flow and the Psychology of Discovery and Invention. Harper Perennial, New York (1996)

    Google Scholar 

  • d’Avella, A., Portone, A., Fernandez, L., Lacquaniti, F.: Control of fast-reaching movement by muscle synergies combinations. J. Neurosci. 26(30), 7791–7810 (2006)

    Google Scholar 

  • d’Avella, A., Saltiel, P., Bizzi, E.: Combinations of muscle synergies in the construction of a natural motor behavior. Nat. Neurosci. 6, 300–308 (2003)

    Google Scholar 

  • Dayan, P.: Exploration from generalisation mediated by multiple controllers. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 73–91. Springer, Berlin (2012)

    Google Scholar 

  • Dayan, P., Belleine, W.: Reward, motivation and reinforcement learning. Neuron 36, 285–298 (2002)

    Google Scholar 

  • De Charms, R.: Personal Causation: The Internal Affective Determinants of Behavior. Academic, New York (1968)

    Google Scholar 

  • Deci, E., Ryan, M.: Intrinsic Motivation and Self-determination in Human Behavior. Plenum, New York (1985)

    Google Scholar 

  • Dick, T., Oku, Y., Romaniuk, J., Cherniack, N.: Interaction between cpgs for breathing and swallowing in the cat. J. Physiol. 465, 715–730 (1993)

    Google Scholar 

  • Doya, K.: Metalearning and neuromodulation. Neural Netw. 15(4–5), 495–506 (2002)

    Google Scholar 

  • Eyre, J.: Development and Plasticity of the Corticospinal System in Man. Neural Plast.; 10(1–2), 93–106 (2003)

    Google Scholar 

  • Faller, D., Klingmüller, U., Timmer, J.: Simulation methods for optimal experimental design in systems biology. Simulation 79, 717–725 (2003)

    Google Scholar 

  • Fedorov, V.: Theory of Optimal Experiment. Academic, New York (1972)

    Google Scholar 

  • Festinger, L.: A Theory of Cognitive Dissonance. Row & Peterson, Evanston (1957)

    Google Scholar 

  • Fisher, K., Silvern, L.: Stages and individual differences in cognitive development. Annu. Rev. Psychol. 36, 613–648 (1985)

    Google Scholar 

  • Franceschini, N., Pichon, J., Blanes, C.: From insect vision to robot vision. Phil. Trans. R. Soc. Lond. B 337, 283–294 (1992)

    Google Scholar 

  • Ghahramani, Z.: Solving inverse problems using an em approach to density estimation. In: Mozer, M., Smolensky, P., Toureztky, D., Elman, J., Weigend, A. (eds.) Proceedings of the 1993 Connectionist Models Summer School (1993)

    Google Scholar 

  • Gibson, J.: The Ecological Approach to Visual Perception. Lawrence Erlbaum Associates, Hillsdale (1986)

    Google Scholar 

  • Grollman, D.H., Jenkins, O.C.: Incremental learning of subtasks from unsegmented demonstration. In: International Conference on Intelligent Robots and Systems, Taipei (2010)

    Google Scholar 

  • Hart, S., Grupen, R.: Intrinsically motivated hierarchical manipulation. In: Proceedings of the 2008 IEEE Conference on Robots and Automation (ICRA) (2008)

    Google Scholar 

  • Hart, S., Grupen, R.: Intrinsically motivated affordance discovery and modeling. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 279–300. Springer, Berlin (2012)

    Google Scholar 

  • Huang, X., Weng, J.: Novelty and reinforcement learning in the value system of developmental robots. In: Prince, C., Demiris, Y., Marom, Y., Kozima, H., Balkenius, C. (eds.) Proceedings of the 2nd International Workshop on Epigenetic Robotics : Modeling Cognitive Development in Robotic Systems, pp. 47–55. Lund University Cognitive Studies 94, Lund (2002)

    Google Scholar 

  • Hull, C.L.: Principles of Behavior: An Introduction to Behavior Theory. Appleton-Century-Croft, New York (1943)

    Google Scholar 

  • Hunt, J.M.: Intrinsic motivation and its role in psychological development. Nebraska Symp. Motiv. 13, 189–282 (1965)

    Google Scholar 

  • Iida, F., Pfeifer, R.: Cheap and rapid locomotion of a quadruped robot: Self-stabilization of bounding gait. In: Proceedings of the 8th International Conference on Intelligent Autonomous Systems (IAS-8), Amsterdam, Netherlands, Groen, F. et al. (Eds.) (2004)

    Google Scholar 

  • James, W.: The Principles of Psychology. Harvard University Press, Cambridge (1890)

    Google Scholar 

  • Johnson, M.: Functional brain development in humans. Nat. Rev. Neurosci. 2(7), 475–483 (2001)

    Google Scholar 

  • Kagan, J.: Motives and development. J. Pers. Soc. Psychol. 22, 51–66 (1972)

    Google Scholar 

  • Kakade, S., Dayan, P.: Dopamine: Generalization and bonuses. Neural Netw. 15, 549–559 (2002)

    Google Scholar 

  • Kaplan, F., Oudeyer, P.-Y.: The progress-drive hypothesis: An interpretation of early imitation. In: Nehaniv, C., Dautenhahn, K. (eds.) Models and Mechanisms of Imitation and Social Learning: Behavioural, Social and Communication Dimensions, pp. 361–377. Cambridge University Press, Cambridge (2007)

    Google Scholar 

  • Kemp, C., Edsinger, A.: What can i control?: The development of visual categories for a robots body and the world that it influences. In: In 5th IEEE International Conference on Development and Learning (ICDL-06), Special Session on Autonomous Mental Development (2006)

    Google Scholar 

  • Khatib, O.: A unified approach for motion and force control of robot manipulators: The operational space formulation. IEEE J. Robot. Autom. 3(1), 43–53 (1987)

    Google Scholar 

  • Konczak, J., Borutta, M., Dichgans, J.: The development of goal-directed reaching in infants. Learning to produce task-adequate patterns of joint torque. Exp. Brain Res. 106(1), 156–168 (1997)

    Google Scholar 

  • Kumar, S., Narasimhan, K., Patwardhan, S., Prasad, V.: Experiment design, identification and control in large-scale chemical processes. In: The 2010 International Conference on Modelling, Identification and Control (ICMIC), pp. 155–160 (2010)

    Google Scholar 

  • Lee, M., Meng, Q., Chao, F.: Staged competence learning in developmental robotics. Adap. Behav. 15(3), 241–255 (2007)

    Google Scholar 

  • Lee, W.: Neuromotor synergies as a basis for coordinated intentional action. J. Mot. Behav. 16, 135–170 (1984)

    Google Scholar 

  • Lopes, M., Melo, F., Montesano, L.: Active learning for reward estimation in inverse reinforcement learning. In: Proceedings of European Conference on Machine Learning (ECML/PKDD) (2009)

    Google Scholar 

  • Lopes, M., Oudeyer, P.-Y.: Active learning and intrinsically motivated exploration in robots: Advances and challenges (guest editorial): IEEE Trans. Auton. Mental Dev. 2(2), 65–69 (2010)

    Google Scholar 

  • Lungarella, M., Metta, G., Pfeifer, R., Sandini, G.: Developmental robotics: A survey. Connect. Sci. 15(4), 151–190 (2003)

    Google Scholar 

  • Ly, O., Lapeyre, M., Oudeyer, P.-Y.: Bio-inspired vertebral column, compliance and semi-passive dynamics in a lightweight robot. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2011) (2011)

    Google Scholar 

  • Ly, O., Oudeyer, P.-Y.: Acroban the humanoid: Playful and compliant physical child-robot interaction. In: ACM Siggraph Emerging Technologies, pp. 1–1 (2010)

    Google Scholar 

  • MacNeilage, P.: The Origin of Speech. Oxford University Press, Oxford (2008)

    Google Scholar 

  • Meltzoff, A., Moore, M.: Imitation of facial and manual gestures by human neonates. Science 198(4312), 75–8 (1977)

    Google Scholar 

  • Merrick, K.E.: Novelty and beyond: Towards combined motivation models and integrated learning architectures. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 209–233. Springer, Berlin (2012)

    Google Scholar 

  • Meyer, J.A., Wilson, S.W. (eds.): A Possibility for Implementing Curiosity and Boredom in Model-Building Neural Controllers. MIT/Bradford Books, Cambridge (1991)

    Google Scholar 

  • Mirolli, M., Baldassarre, G.: Functions and mechanisms of intrinsic motivations: The knowledge versus competence distinction. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 47–72. Springer, Berlin (2012)

    Google Scholar 

  • Modayil, J., Pilarski, P., White, A., Degris, T., Sutton, R.: Off-policy knowledge maintenance for robots. In: Proceedings of Robotics Science and Systems Workshop (Towards Closing the Loop: Active Learning for Robotics) (2010)

    Google Scholar 

  • Montgomery, K.: The role of exploratory drive in learning. J. Comp. Physiol. Psychol. 47, 60–64 (1954)

    Google Scholar 

  • Moore, A.: Fast, robust adaptive control by learning only forward models. In: Advances in Neural Information Processing Systems, vol. 4 (1992)

    Google Scholar 

  • Muja, M., Lowe, D.: Fast approximate nearest neighbors with automatic algorithm. In: International Conference on Computer Vision Theory and Applications (VISAPP’09) (2009)

    Google Scholar 

  • Ng, A.Y., Russell, S.: Algorithms for inverse reinforcement learning. In: Proceedings of the 17th International Conference on Machine Learning, pp. 663–670. Morgan Kaufmann, San Francisco (2000)

    Google Scholar 

  • Nguyen, M., Baranes, A., Oudeyer, P.-Y.: Bootstrapping intrinsically motivated learning with human demonstrations. In: Proceedings of IEEE ICDL-Epirob 2011 (2011)

    Google Scholar 

  • Nguyen-Tuong, D., Peters, J.: Model learning in robotics: A survey. Cogn. Process. 12(4), 319–340 (2011)

    Google Scholar 

  • Oudeyer, P.-Y.: On the impact of robotics in behavioral and cognitive sciences: From insect navigation to human cognitive development. IEEE Trans. Auton. Mental Dev. 2(1), 2–16 (2010)

    Google Scholar 

  • Oudeyer, P.-Y. Kaplan, F.: The discovery of communication. Connect. Sci. 18(2), 189–206 (2006)

    Google Scholar 

  • Oudeyer, P.-Y. Kaplan, F.: What is intrinsic motivation? A typology of computational approaches. Front. Neurorobot. 1, 6 (2007)

    Google Scholar 

  • Oudeyer, P.-Y. Kaplan, F.: How can we define intrinsic motivations ? In: Proceedings of the 8th Conference on Epigenetic Robotics (2008)

    Google Scholar 

  • Oudeyer, P.-Y., Kaplan, F., Hafner, V.: Intrinsic motivation systems for autonomous mental development. IEEE Trans. Evol. Comput. 11(2), 265–286 (2007)

    Google Scholar 

  • Oudeyer, P.-Y., Ly, O., Rouanet, P.: Exploring robust, intuitive and emergent physical human–robot interaction with the humanoid acroban. In: Proceedings of IEEE-RAS International Conference on Humanoid Robots (2011)

    Google Scholar 

  • Paul, C.: Morphology and computation. In: Proceedings of the International Conference on the Simulation of Adaptive Behaviour (2004)

    Google Scholar 

  • Peters, J., Schaal, S.: Natural actor critic. Neurocomputing 71, 1180–1190 (2008)

    Google Scholar 

  • Pfeifer, R., Bongard, J.C.: How the Body Shapes the Way We Think: A New View of Intelligence. MIT/Bradford Books, Cambridge (2006)

    Google Scholar 

  • Pfeifer, R., Lungarella, M., Iida, F.: Self-organization, embodiment, and biologically inspired robotics. Science 318, 1088–1093 (2007)

    Google Scholar 

  • Pfeifer, R., Scheier, C.: Understanding Intelligence. MIT, Boston (1999)

    Google Scholar 

  • Piaget, J.: The Origins of Intelligence in Childhood. International University Press, New York (1952)

    Google Scholar 

  • Redgrave, P., Gurney, K., Stafford, T., Thirkettle, M., Lewis, J.: The role of the basal ganglia in discovering novel actions. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 129–149. Springer, Berlin (2012)

    Google Scholar 

  • Ring, M.: Continual learning in reinforcement environments. Ph.D. Thesis, University of Texas at Austin, Austin (1994)

    Google Scholar 

  • Rochat, P.: Object manipulation and exploration in 2- to 5-month-old infants Dev. Psychol. 25, 871–884 (1989)

    Google Scholar 

  • Rolf, M., Steil, J., Gienger, M.: Goal babbling permits direct learning of inverse kinematics. IEEE Trans. Auton. Mental Dev. 2(3), 216–229 (2010)

    Google Scholar 

  • Ryan, R.M., Deci, E.L.: Intrinsic and extrinsic motivations: Classic definitions and new directions. Contem. Educ. Psychol. 25(1), 54–67 (2000)

    Google Scholar 

  • Schaal, S., Atkeson, C.G.: Robot juggling: An implementation of memory-based learning. Control Syst. Mag. 57–71 (1994)

    Google Scholar 

  • Schaal, S., Atkeson, C.G.: Robot learning by nonparametric regression, In: Proceedings of Intelligent Robots and Systems 1994 (IROS 94) pp. 137–153 (1995)

    Google Scholar 

  • Schembri, M., Mirolli, M., Baldassare, G.: Evolving internal reinforcers for an intrinsically motivated reinforcement learning robot. In: Demiris, Y., Scassellati, B., Mareschal, D. (eds.) Proceedings of the 6th IEEE International Conference on Development and Learning (ICDL2007) (2007a)

    Google Scholar 

  • Schembri, M., Mirolli, M., G., B.: Evolution and learning in an intrinsically motivated reinforcement learning robot. In: Springer (ed.) Advances in Artificial Life. Proceedings of the 9th European Conference on Artificial Life, Berlin, pp. 294–333 (2007b)

    Google Scholar 

  • Schlesinger, M.: Heterochrony: It’s (all) about time! In: Studies, L.U.C. (ed.) Proceedings of the Eighth International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, Sweden, pp. 111–117 (2008)

    Google Scholar 

  • Schlesinger, M.: Investigating the origins of intrinsic motivations in human infants. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 367–392. Springer, Berlin (2012)

    Google Scholar 

  • Schmidhuber, J.: Curious model-building control systems. Proc. Int. Joint Conf. Neural Netw. 2, 1458–1463 (1991)

    Google Scholar 

  • Schmidhuber, J.: Exploring the predictable. In: Ghosh, S., Tsutsui, S. (eds.) Advances in Evolutionary Computing: Theory and Applications, pp. 579–612. Springer, New York (2002)

    Google Scholar 

  • Schmidhuber, J.: Optimal artificial curiosity, developmental robotics, creativity, music, and the fine arts. Connect. Sci. 18(2), 173–187 (2006)

    Google Scholar 

  • Schmidhuber, J.: Formal theory of creativity. IEEE Trans. Auton. Mental Dev. 2(3), 230–247 (2010)

    Google Scholar 

  • Schmidhuber, J.: Maximizing fun by creating data with easily reducible subjective complexity. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 95–128. Springer, Berlin (2012)

    Google Scholar 

  • Scholz, J., Klein, M., Behrens, T., Johansen-Berg, H.: Training induces changes in white-matter architecture. Nat. Neurosci. 12(11), 1367–1368 (2009)

    Google Scholar 

  • Sekuler, R., Blake, R.: Perception. McGraw-Hill, New York (1994)

    Google Scholar 

  • Sigaud, O., Salaũn, C., Padois, V.: On-line regression algorithms for learning mechanical models of robots: A survey. Robot. Auton. Syst. 59(12), 1115–1129 (2011)

    Google Scholar 

  • Singh, S., Lewis, R., Barto, A., Sorg, J.: Intrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Trans. Auton. Mental Dev. 2(2), 70–82 (2010)

    Google Scholar 

  • Stout, A., Barto, A.: Competence based intrinsic motivation. In: Proceedings of IEEE International Conference on Development and Learning (ICDL 2010) (2010)

    Google Scholar 

  • Sutton, R.: Integrated architectures for learning, planning, and reacting based on approximating integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In: Proceedings of the International Machine Learning Conference, pp. 212–218 (1990)

    Google Scholar 

  • Sutton, R., Barto, A.: Reinforcement learning: An introduction. MIT, Cambridge (1998)

    MATH  Google Scholar 

  • Sutton, R., Precup, D., Singh, S.: Between mdpss and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artif. Intell. 112, 181–211 (1999)

    MathSciNet  MATH  Google Scholar 

  • Szita, I., Lorincz, A.: The many faces of optimism: A unifying approach. In: Proceedings of ICML’08 (2008)

    Google Scholar 

  • Thomaz, A., Breazeal, C.: Experiments in socially guided exploration: Lessons learned in building robots that learn with and without human teachers. Connect. Sci. 20(2–3), 91–110 (2008)

    Google Scholar 

  • Thrun, S.: The role of exploration in learning control. In: White, D., Sofge, D. (eds.) Handbook for Intelligent Control: Neural, Fuzzy and Adaptive Approaches. Van Nostrand Reinhold, Florence (1992)

    Google Scholar 

  • Thrun, S., Moller, K.: Active exploration in dynamic environments. In: J. Moody, S., Hanson, R.L. (ed.) Proceedings of the Advances of Neural Information Processing Systems, vol. 4 (1992)

    Google Scholar 

  • Ting, L., McKay, J.: Neuromechanics of muscle synergies for posture and movement. Curr. Opin. Neurobiol. 17, 622–628 (2007)

    Google Scholar 

  • Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: Proceedings of the Ninth ACM International Conference on Multimedia, MULTIMEDIA’01, pp. 107–118. ACM (2001)

    Google Scholar 

  • Turkewitz, G., Kenny, P.: The role of developmental limitations of sensory input on sensory/perceptual organization. J. Dev. Behav. Pediatr. 6(5), 302–306 (1985)

    Google Scholar 

  • Weiss, E., Flanders, M.: Muscular and postural synergies of the human hand. J. Neurophysiol. 92, 523–535 (2004)

    Google Scholar 

  • Weng, J., McClelland, J., Pentland, A., Sporns, O., Stockman, I., Sur, M., Thelen, E.: Autonomous mental development by robots and animals. Science 291, 599–600 (2001)

    Google Scholar 

  • White, R.: Motivation reconsidered: The concept of competence. Psychol. Rev. 66, 297–333 (1959)

    Google Scholar 

  • Whitehead, S.: A Study of Cooperative Mechanisms for Faster Reinforcement Learning. Tr-365, University of Rochester, Rochester (1991)

    Google Scholar 

  • Wiering, M., Schmidhuber, J.: Hq-learning. Adap. Behav. 6, 219–246 (1997)

    Google Scholar 

  • Wundt, W.: Grundzuge der physiologischen Psychologie. Engelmann, Leipzig (1874)

    Google Scholar 

  • Yokoi, H., Hernandez, A., Katoh, R., Yu, W., Watanabe, I., Maruishi, M.: Mutual adaptation in a prosthetics application. In: Embodied Artificial Intelligence. LNAI, vol. 3139. Springer, Heidelberg (2004)

    Google Scholar 

  • Ziegler, M., Iida, F., Pfeifer, R.: Cheap underwater locomotion: Roles of morphological properties and behavioural diversity. In: Proceedings of the International Conference on Climbing and Walking Robots (2006)

    Google Scholar 

Download references

Acknowledgements

Many of the ideas presented in this chapter benefited from discussions and joint work with our colleagues, in particular, Jérome Béchu, Fabien Benureau, Thomas Cederborg, Fabien Danieau, Haylee Fogg, David Filliat, Paul Fudal, Verena V. Hafner, Matthieu Lapeyre, Manuel Lopes, Olivier Ly, Olivier Mangin, Mai Nguyen, Luc Steels, Pierre Rouanet, and Andrew Whyte. This research was partially funded by ERC Starting Grant EXPLORER 240007.

Author information

Authors and Affiliations

  1. Bordeaux, France

    Pierre-Yves Oudeyer & Adrien Baranes

  2. EPFL-CRAFT, Lausanne, Switzerland

    Frédéric Kaplan

Authors
  1. Pierre-Yves Oudeyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adrien Baranes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frédéric Kaplan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pierre-Yves Oudeyer .

Editor information

Editors and Affiliations

  1. Istituto di Scienze e Tecnologie della Cognizione Consiglio Nazionale delle Ricerche, Rome, Italy

    Gianluca Baldassarre  & Marco Mirolli  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Oudeyer, PY., Baranes, A., Kaplan, F. (2013). Intrinsically Motivated Learning of Real-World Sensorimotor Skills with Developmental Constraints. In: Baldassarre, G., Mirolli, M. (eds) Intrinsically Motivated Learning in Natural and Artificial Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32375-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32375-1_13

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32374-4

  • Online ISBN: 978-3-642-32375-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation