Abstraction Levels for Robotic Imitation: Overview and Computational Approaches

  • Manuel Lopes
  • Francisco Melo
  • Luis Montesano
  • José Santos-Victor

Abstract

This chapter reviews several approaches to the problem of learning by imitation in robotics. We start by describing several cognitive processes identified in the literature as necessary for imitation. We then proceed by surveying different approaches to this problem, placing particular emphasys on methods whereby an agent first learns about its own body dynamics by means of self-exploration and then uses this knowledge about its own body to recognize the actions being performed by other agents. This general approach is related to the motor theory of perception, particularly to the mirror neurons found in primates. We distinguish three fundamental classes of methods, corresponding to three abstraction levels at which imitation can be addressed. As such, the methods surveyed herein exhibit behaviors that range from raw sensory-motor trajectory matching to high-level abstract task replication. We also discuss the impact that knowledge about the world and/or the demonstrator can have on the particular behaviors exhibited.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abbeel, P., Ng, A.Y.: Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the 21st International Conference on Machine Learning (ICML 2004), pp. 1–8 (2004)Google Scholar
  2. 2.
    Alissandrakis, A., Nehaniv, C.L., Dautenhahn, K.: Imitating with alice: Learning to imitate corresponding actions across dissimilar embodiments. IEEE Transactions on Systems, Man, & Cybernetics, Part A: Systems and Humans 32(4), 482–496 (2002)CrossRefGoogle Scholar
  3. 3.
    Alissandrakis, A., Nehaniv, C.L., Dautenhahn, K.: Towards robot cultures? - learning to imitate in a robotic arm test-bed with dissimilarly embodied agents. Interaction Studies 5(1), 3–44 (2004)CrossRefGoogle Scholar
  4. 4.
    Alissandrakis, A., Nehaniv, C.L., Dautenhahn, K.: Action, state and effect metrics for robot imitation. In: 15th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2006), Hatfield, United Kingdom, pp. 232–237 (2006)Google Scholar
  5. 5.
    Arbib, M.A., Billard, A., Iacoboni, M., Oztop, E.: Synthetic brain imaging: grasping, mirror neurons and imitation. Neural Networks 13, 975–997 (2000)CrossRefGoogle Scholar
  6. 6.
    Argall, B., Chernova, S., Veloso, M.: A survey of robot learning from demonstration. Robotics and Autonomous Systems 57(5), 469–483 (2009)CrossRefGoogle Scholar
  7. 7.
    Asada, M., Ogino, M., Matsuyama, S., Ooga, J.: Imitation learning based on visuo-somatic mapping. In: Marcelo, O.K., Ang, H. (eds.) 9th Int. Symp. Exp. Robot., vol. 21, pp. 269–278. Springer, Berlin (2006)CrossRefGoogle Scholar
  8. 8.
    Asada, M., Yoshikawa, Y., Hosoda, K.: Learning by observation without three-dimensional reconstruction. In: Intelligent Autonomous Systems (IAS-6), pp. 555–560 (2000)Google Scholar
  9. 9.
    Atkeson, C.G., Schaal, S.: Robot learning from demonstration. In: 14th International Conference on Machine Learning, pp. 12–20. Morgan Kaufmann, San Francisco (1997)Google Scholar
  10. 10.
    Baker, C.L., Tenenbaum, J.B., Saxe, R.R.: Bayesian models of human action understanding. In: Advances in Neural Information Processing Systems, vol. 18 (2006)Google Scholar
  11. 11.
    Bakker, P., Kuniyoshi, Y.: Robot see, robot do: An overview of robot imitation. In: Fogarty, T.C. (ed.) AISB-WS 1996. LNCS, vol. 1143, pp. 3–11. Springer, Heidelberg (1996)Google Scholar
  12. 12.
    Bandera, J.P., Marfil, R., Molina-Tanco, L., Rodríguez, J.A., Bandera, A., Sandoval, F.: Robot Learning by Active Imitation, pp. 519–544. ARS Publ. (2007)Google Scholar
  13. 13.
    Bekkering, H., Wohlschläger, A., Gattis, M.: Imitation of gestures in children is goal-directed. Quarterly J. Experimental Psychology 53A, 153–164 (2000)CrossRefGoogle Scholar
  14. 14.
    Bellagamba, F., Tomasello, M.: Re-enacting intended acts: Comparing 12- and 18-month-olds. Infant Behavior and Development 22(2), 277–282 (1999)CrossRefGoogle Scholar
  15. 15.
    Billard, A.: Imitation: A means to enhance learning of a synthetic proto-language in an autonomous robot. In: Imitation in Animals and Artifacs, pp. 281–311. MIT Press, Cambridge (1999)Google Scholar
  16. 16.
    Billard, A., Calinon, S., Dillman, R., Schaal, S.: Robot Programming by Demonstration. In: Handbook of Robotics, ch. 59. Springer, Heidelberg (2007)Google Scholar
  17. 17.
    Billard, A., Epars, Y., Calinon, S., Cheng, G., Schaal, S.: Discovering optimal imitation strategies. Robotics and Autonomous Systems 47(2-3) (2004)Google Scholar
  18. 18.
    Boucenna, S., Gaussier, P., Andry, P.: What should be taught first: the emotional expression or the face? In: 8th International conference on Epigenetic Robotics, Brighton, UK (2008)Google Scholar
  19. 19.
    Brass, M., Schmitt, R.M., Spengler, S., Gergely, G.: Investigating action understanding: Inferential processes versus action simulation. Current Biology 17(24), 2117–2121 (2007)CrossRefGoogle Scholar
  20. 20.
    Breazeal, C.: Imitation as social exchange between humans and robots. In: AISB Symp. Imitation in Animals and Artifacts, pp. 96–104 (1999)Google Scholar
  21. 21.
    Brugger, A., Lariviere, L.A., Mumme, D.L., Bushnell, E.W.: Doing the right thing: Infants’ selection of actions to imitate from observed event sequences. Child Development 78(3), 806–824 (2007)CrossRefGoogle Scholar
  22. 22.
    Bruner, J.: Nature and use of immaturity. American Psychologist 27, 687–708 (1972)CrossRefGoogle Scholar
  23. 23.
    Byrne, R.W.: Imitation without intentionality using string parsing to copy the organization of behaviour. Animal Cognition 2, 63–72 (1999)CrossRefGoogle Scholar
  24. 24.
    Calinon, S., Billard, A.: Learning of gestures by imitation in a humanoid robot. In: Dautenhahn, K., Nehaniv, C.L. (eds.) Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions, pp. 153–177. Cambridge University Press, Cambridge (2007)CrossRefGoogle Scholar
  25. 25.
    Call, J., Carpenter, M.: Three sources of information in social learning. In: Imitation in animals and artifacts. MIT Press, Cambridge (2002)Google Scholar
  26. 26.
    Cantin-Martinez, R., Lopes, M., Melo, F.: Inverse reinforcement learning with noisy observations. Tech. rep., Institute for Systems and Robotics, Lisbon, Portugal (2009)Google Scholar
  27. 27.
    Cantin-Martinez, R., Lopes, M., Montesano, L.: Active body schema learning. Tech. rep., Institute for Systems and Robotics, Lisbon, Portugal (2009)Google Scholar
  28. 28.
    Carpenter, M., Call, J., Tomasello, M.: Twelve- and 18-month-olds copy actions in terms of goals. Developmental Science 1(8), F13–F20 (2005)CrossRefGoogle Scholar
  29. 29.
    Chalodhorn, R., Rao, R.P.N.: Learning to imitate human actions through Eigenposes. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 357–381. Springer, Heidelberg (2010)Google Scholar
  30. 30.
    Chernova, S., Veloso, M.: Teaching collaborative multi-robot tasks through demonstration. In: IEEE-RAS International Conference on Humanoid Robots, Korea (2008)Google Scholar
  31. 31.
    Chernova, S., Veloso, M.: Interactive policy learning through confidence-based autonomy. J. Artificial Intelligence Research 34, 1–25 (2009)MATHMathSciNetGoogle Scholar
  32. 32.
    Cos-Aguilera, I., Cañamero, L., Hayes, G.: Using a SOFM to learn object affordances. In: Workshop of Physical Agents (WAF), Girona, Spain (2004)Google Scholar
  33. 33.
    Csibra, G., Gergely, G.: ”Obsessed with goals”: Functions and mechanisms of teleological interpretation of actions in humans. Acta Psychologica 124, 60–78 (2007)CrossRefGoogle Scholar
  34. 34.
    Demiris, J., Hayes, G.: Imitative learning mechanisms in robots and humans. In: European Workshop on Learning Robots, pp. 9–16 (1996)Google Scholar
  35. 35.
    Demiris, J., Rougeaux, S., Hayes, G.M., Berthouze, L., Kuniyoshi, Y.: Deferred imitation of human head movements by an active stereo vision head. In: 6th IEEE Int. Workshop on Robot Human Communication, pp. 88–93 (1997)Google Scholar
  36. 36.
    Demiris, Y., Dearden, A.: From motor babbling to hierarchical learning by imitation: a robot developmental pathway. In: EPIROB 2005, Japan, pp. 31–37 (2005)Google Scholar
  37. 37.
    Demiris, Y., Hayes, G.: Imitation as a dual-route process featuring predictive and learning components: a biologically-plausible computational model. In: Dautenhahn, K., Nehaniv, C. (eds.) Imitation in Animals and Artifacts. MIT Press, Cambridge (2002)Google Scholar
  38. 38.
    Demiris, Y., Khadhouri, B.: Hierarchical attentive multiple models for execution and recognition of actions. Robotics and Autonomous Systems 54, 361–369 (2006)CrossRefGoogle Scholar
  39. 39.
    Detry, R., Baseski, E., Popovi, M., Touati, Y., Kruger, N., Kroemer, O., Peters, J., Piater, J.: Learning object-specific grasp affordance densities. In: IEEE 8th International Conference on Development and Learning (2009)Google Scholar
  40. 40.
    Doshi, F., Pineau, J., Roy, N.: Reinforcement learning with limited reinforcement: using bayes risk for active learning in pomdps. In: Proceedings of the 25th international conference on Machine learning (ICML 2008), pp. 256–263 (2008)Google Scholar
  41. 41.
    D’Souza, A., Vijayakumar, S., Schaal, S.: Learning inverse kinematics. In: International Conference on Intelligent Robots and Systems, Hawaii, USA, pp. 298–303 (2001)Google Scholar
  42. 42.
    Erlhagen, W., Mukovskiy, A., Chersi, F., Bicho, E.: On the development of intention understanding for joint action tasks. In: 6th IEEE International Conference on Development and Learning (ICDL 2007), London, UK (2007)Google Scholar
  43. 43.
    Erlhagen, W., Mukovsky, A., Bicho, E.: A dynamic model for action understanding and goal-directed imitation. Brain Research 1083, 174–188 (2006)CrossRefGoogle Scholar
  44. 44.
    Erlhagen, W., Mukovsky, A., Bicho, E., Panin, G., Kiss, C., Knoll, A., van Schie, H., Bekkering, H.: Goal-directed imitation in robots: a bio-inspired approach to action understanding and skill learning. Robotics and Autonomous Systems 54(5), 353–360 (2006)CrossRefGoogle Scholar
  45. 45.
    Field, T., Field, T., Sanders, C., Nadel, J.: Children with autism become more social after repeated imitation sessions. Autism, 317–324 (2001)Google Scholar
  46. 46.
    Fitzpatrick, P., Metta, G., Natale, L., Rao, S., Sandini., G.: Learning about objects through action: Initial steps towards artificial cognition. In: IEEE International Conference on Robotics and Automation, Taipei, Taiwan (2003)Google Scholar
  47. 47.
    Fogassi, L., Gallese, V., Buccino, G., Craighero, L., Fadiga, L., Rizzolatti, G.: Cortical mechanism for the visual guidance of hand grasping movements in the monkey: A reversible inactivation study. Brain 124(3), 571–586 (2001)CrossRefGoogle Scholar
  48. 48.
    Fritz, G., Paletta, L., Breithaupt, R., Rome, E., Dorffner, G.: Learning predictive features in affordance based robotic perception systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China (2006)Google Scholar
  49. 49.
    Furse, E.: A model of imitation learning of algorithms from worked examples. Cybernetics and Systems 32, 121–154 (2001)CrossRefGoogle Scholar
  50. 50.
    Galantucci, B., Fowler, C.A., Turvey, M.T.: The motor theory of speech perception reviewed. Psychonomic Bulletin & Review 13(3), 361–377 (2006)Google Scholar
  51. 51.
    Gallese, V., Fadiga, L., Fogassi, L., Rizolaatti, G.: Action recognition in the premotor cortex. Brain 119, 593–609 (1996)CrossRefGoogle Scholar
  52. 52.
    Gardner, M.: Imitation and egocentric perspective transformation. In: Virtual Poster associated with Perspectives on Imitation Conference (2002)Google Scholar
  53. 53.
    Gaussier, P., Moga, S., Quoy, J.B.,, M.: From perception-action loops to imitation processes: A bottom-up approach of learning by imitation. Applied Artificial Intelligence: An Internation Journal 1(7) (1997)Google Scholar
  54. 54.
    Gergely, G., Bekkering, H., Király, I.: Rational imitation in preverbal infants. Nature 415, 755 (2002)Google Scholar
  55. 55.
    Gibson, J.J.: The Ecological Approach to Visual Perception. Houghton Mifflin, Boston (1979)Google Scholar
  56. 56.
    Gopnik, A.: Cells that read minds? what the myth of mirror neurons gets wrong about the human brain. Slate: Brains: A special issue on neuroscience and neuroculture (2007)Google Scholar
  57. 57.
    Grimes, D.B., Rashid, D.R., Rao, R.P.: Learning nonparametric models for probabilistic imitation. In: Advances in NIPS (2007)Google Scholar
  58. 58.
    Hart, S., Grupen, R., Jensen, D.: A relational representation for procedural task knowledge. In: Proceedings of the 2005 American Association for Artificial Intelligence (AAAI) Conference, Pittsburgh, USA (2005)Google Scholar
  59. 59.
    Hayes, G.M., Demiris, J.: A robot controller using learning by imitation. In: Int. Symp. Intelligent Robotic Systems, pp. 198–204 (1994)Google Scholar
  60. 60.
    Heckerman, D., Geiger, D., Chickering, M.: Learning bayesian networks: the combination of knowledge and statistical data. Machine Learning (1995)Google Scholar
  61. 61.
    Hersch, M., Sauser, E., Billard, A.: Online learning of the body schema. International Journal of Humanoid Robotics 5(2), 161–181 (2008)CrossRefGoogle Scholar
  62. 62.
    Horner, V., Whiten, A.: Causal knowledge and imitation/emulation switching in chimpanzees (Pan troglodytes) and children (Homo sapiens). Animal Cognition 8, 164–181 (2005)CrossRefGoogle Scholar
  63. 63.
    Hovland, G., Sikka, P., McCarragher, B.: Skill acquisition from human demonstration using a hidden markov model. In: IEEE International Conference on Robotics and Automation, Minneapolis, MN, pp. 2706–2711 (1996)Google Scholar
  64. 64.
    Jansen, B., Belpaeme, T.: A model for inferring the intention in imitation tasks. In: 15th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2006), Hatfield, UK (2006)Google Scholar
  65. 65.
    Jenkins, O., Bodenheimer, R., Peters, R.: Manipulation manifolds: Explorations into uncovering manifolds in sensory-motor spaces. In: International Conference on Development and Learning (ICDL 2006), Bloomington, IN, USA (2006)Google Scholar
  66. 66.
    Jenkins, O.C., Matarić, M.J., Weber, S.: Primitive-based movement classification for humanoid imitation. In: IEEE International Conference on Humanoid Robots, Humanoids 2000 (2000)Google Scholar
  67. 67.
    Johnson, S., Booth, A., O’Hearn, K.: Inferring the goals of a nonhuman agent. Cognitive Development 16(1), 637–656 (2001)CrossRefGoogle Scholar
  68. 68.
    Kaplan, F., Oudeyer, P.Y.: The progress drive hypothesis: an interpretation of early imitation. In: Models and Mechanims of Imitation and Social Learning: Behavioural, Social and Communication Dimensions, pp. 361–377. Cambridge University Press, Cambridge (2007)CrossRefGoogle Scholar
  69. 69.
    Konczak, J., Meeuwsen, H., Cress, M.: Changing affordances in stair climbing: The perception of maximum climbability in young and older adults. Journal of Experimental Psychology: Human Perception & Performance 19, 691–697 (1992)CrossRefGoogle Scholar
  70. 70.
    Kozima, H.: Infanoid: An experimental tool for developmental psycho-robotics. In: International Workshop on Developmental Study, Tokyo, Japan (2000)Google Scholar
  71. 71.
    Kozima, H., Nakagawa, C., Yano, H.: Emergence of imitation mediated by objects. In: 2nd Int. Workshop on Epigenetic Robotics (2002)Google Scholar
  72. 72.
    Kozima, H., Nakagawa, C., Yano, H.: Attention coupling as a prerequisite for social interaction. In: IEEE International Workshop on Robot and Human Interactive Communication, Millbrae, USA (2003)Google Scholar
  73. 73.
    Kozima, H., Yano, H.: Designing a robot for contingency-detection game. In: International Workshop on Robotic and Virtual Agents in Autism Therapy, Hertfordshire, England (2001)Google Scholar
  74. 74.
    Kulić, D., Nakamura, Y.: Incremental Learning of Full Body Motion Primitives. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 383–406. Springer, Heidelberg (2010)Google Scholar
  75. 75.
    Kuniyoshi, Y., Inaba, M., Inoue, H.: Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Trans. on Robotics and Automation 10(6), 799–822 (1994)CrossRefGoogle Scholar
  76. 76.
    Kuniyoshi, Y., Yorozu, Y., Inaba, M., Inoue, H.: From visuo-motor self learning to early imitation-a neural architecture for humanoid learning. In: IEEE Int. Conf. Robotics and Automation, vol. 3, pp. 3132–3139 (2003)Google Scholar
  77. 77.
    Liberman, A.M., Mattingly, I.G.: The motor theory of speech perception revised. Cognition 21, 1–36 (1985)CrossRefGoogle Scholar
  78. 78.
    Lopes, M., Beira, R., Praça, M., Santos-Victor, J.: An anthropomorphic robot torso for imitation: design and experiments. In: International Conference on Intelligent Robots and Systems, Sendai, Japan (2004)Google Scholar
  79. 79.
    Lopes, M., Melo, F., Kenward, B., Santos-Victor, J.: A computational model of social-learning mechanisms. Adaptive Behavior (to be published) Google Scholar
  80. 80.
    Lopes, M., Melo, F.S., Montesano, L.: Affordance-based imitation learning in robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, USA, pp. 1015–1021 (2007)Google Scholar
  81. 81.
    Lopes, M., Melo, F.S., Montesano, L.: Active learning for reward estimation in inverse reinforcement learning. In: European Conference on Machine Learning (ECML/PKDD), Bled, Slovenia (2009)Google Scholar
  82. 82.
    Lopes, M., Santos-Victor, J.: Visual Transformations in Gesture Imitation: What you see is what you do. In: IEEE Int. Conf. Robotics and Automation (2003)Google Scholar
  83. 83.
    Lopes, M., Santos-Victor, J.: Visual learning by imitation with motor representations. IEEE Trans. Systems, Man, and Cybernetics - Part B: Cybernetics 35(3) (2005)Google Scholar
  84. 84.
    Lopes, M., Santos-Victor, J.: A developmental roadmap for learning by imitation in robots. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics 37(2) (2007)Google Scholar
  85. 85.
    Lyons, D.E., Young, A.G., Keil, F.C.: The hidden structure of over imitation. Proceedings of the National Academy of Sciences 104(50), 19751–19756 (2005)CrossRefGoogle Scholar
  86. 86.
    Maistros, G., Marom, Y., Hayes, G.: Perception-action coupling via imitation and attention. In: AAAI Fall Symp. Anchoring Symbols to Sensor Data in Single and Multiple Robot Systems (2001)Google Scholar
  87. 87.
    Matarić, M.J.: Sensory-motor primitives as a basis for learning by imitation: linking perception to action and biology to robotics. In: Imitation in Animals and Artifacts. MIT Press, Cambridge (2002)Google Scholar
  88. 88.
    McGuigan, N., Whiten, A., Flynn, E., Horner, V.: Imitation of causally opaque versus causally transparent tool use by 3- and 5-year-old children. Cognitive Development 22, 353–364 (2007)CrossRefGoogle Scholar
  89. 89.
    Melo, F., Lopes, M., Santos-Victor, J., Ribeiro, M.I.: A unified framework for imitation-like behaviors. In: 4th International Symposium on Imitation in Animals and Artifacts, Newcastle, UK (2007)Google Scholar
  90. 90.
    Meltzoff, A.N.: Infant imitation after a 1-week delay: Long-term memory for novel acts and multiple stimuli. Developmental Psychology 24(4), 470–476 (1988)CrossRefGoogle Scholar
  91. 91.
    Meltzoff, A.N.: Understanding the intentions of others: Re-enactment of intended acts by 18-month-old children. Developmental Psychology 31(5), 838–850 (1995)CrossRefGoogle Scholar
  92. 92.
    Montesano, L., Lopes, M.: Learning grasping affordances from local visual descriptors. In: IEEE 8th International Conference on Development and Learning, China (2009)Google Scholar
  93. 93.
    Montesano, L., Lopes, M., Bernardino, A., Santos-Victor, J.: Affordances, development and imitation. In: IEEE - International Conference on Development and Learning, London, UK (2007)Google Scholar
  94. 94.
    Montesano, L., Lopes, M., Bernardino, A., Santos-Victor, J.: Learning object affordances: From sensory–motor coordination to imitation. IEEE Transactions on Robotics 24(1), 15–26 (2008)CrossRefGoogle Scholar
  95. 95.
    Montesano, L., Lopes, M., Bernardino, A., Santos-Victor, J.: Learning object affordances: From sensory–motor coordination to imitation. IEEE Transactions on Robotics 28(1) (2008)Google Scholar
  96. 96.
    Murata, A., Fadiga, L., Fogassi, L., Gallese, V., Raos, V., Rizzolatti, G.: Object representation in the ventral premotor cortex (area f5) of the monkey. Journal of Neurophysiology 78(4), 2226–2230 (1997)Google Scholar
  97. 97.
    Nadel, J.: Imitation and imitation recognition: functional use in preverbal infants and nonverbal children with autism. In: The imitative mind. Cambridge University Press, Cambridge (2002)Google Scholar
  98. 98.
    Nagai, Y., Asada, M., Hosoda, K.: A developmental approach accelerates learning of joint attention. In: International Conference on Development and Learning (2002)Google Scholar
  99. 99.
    Nakaoka, S., Nakazawa, A., Yokoi, K., Hirukawa, H., Ikeuchi, K.: Generating whole body motions for a biped humanoid robot from captured human dances. In: IEEE International Conference on Robotics and Automation, Taipei, Taiwan (2003)Google Scholar
  100. 100.
    Nehaniv, C., Dautenhahn, K.: Mapping between dissimilar bodies: Affordances and the algebraic foundations of imitation. In: European Workshop on Learning Robots (1998)Google Scholar
  101. 101.
    Nehaniv, C.L., Dautenhahn, K.: Like me? - measures of correspondence and imitation. Cybernetics and Systems 32, 11–51 (2001)MATHCrossRefGoogle Scholar
  102. 102.
    Neu, G., Szepesvári, C.: Apprenticeship learning using inverse reinforcement learning and gradient methods. In: Uncertainty in Artificial Intelligence (UAI), pp. 295–302 (2007)Google Scholar
  103. 103.
    Nielsen, M.: Copying actions and copying outcomes: Social learning through the second year. Developmental Psychology 42(3), 555–565 (2006)CrossRefGoogle Scholar
  104. 104.
    Noble, J., Franks, D.W.: Social learning mechanisms compared in a simple environment. In: Artificial Life VIII: Proceedings of the Eighth International Conference on the Simulation and Synthesis of Living Systems, pp. 379–385. MIT Press, Cambridge (2002)Google Scholar
  105. 105.
    Oudeyer, P.Y.: The Social Formation of Acoustic Codes with ”Something Simpler”. In: Dautenham, K., Nehaniv, C. (eds.) Second International Workshop on Imitation in Animals and Artefacts, AISB 2003, Aberystwyth, Wales (2003)Google Scholar
  106. 106.
    Oztop, E., Kawato, M., Arbib, M.: Mirror neurons and imitation: A computationally guided review. Neural Networks 19(3), 254–271 (2006)MATHCrossRefGoogle Scholar
  107. 107.
    Pelleg, D., Moore, A.W.: X-means: Extending k-means with efficient estimation of the number of clusters. In: International Conference on Machine Learning, San Francisco, CA, USA (2000)Google Scholar
  108. 108.
    Peters, J., Schaal, S.: Reinforcement learning of motor skills with policy gradients. Neural Networks, 682–697 (2008)Google Scholar
  109. 109.
    Peters, J., Vijayakumar, S., Schaal, S.: Natural Actor-Critic. In: Proc. 16th European Conf. Machine Learning, pp. 280–291 (2005)Google Scholar
  110. 110.
    Petreska, B., Billard, A.: A Neurocomputational Model of an Imitation Deficit following Brain Lesion. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 770–779. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  111. 111.
    Pomplun, M., Matarić, M.J.: Evaluation metrics and results of human arm movement imitation. In: IEEE-RAS Int. Conf. Humanoid Robotics (2000)Google Scholar
  112. 112.
    Price, B.: Accelerating reinforcement learning with imitation. Ph.D. thesis, University of British Columbia (2003)Google Scholar
  113. 113.
    Price, B., Boutilier, C.: Implicit imitation in multiagent reinforcement learning. In: Proc. ICML, pp. 325–334 (1999)Google Scholar
  114. 114.
    Price, B., Boutilier, C.: Accelerating reinforcement learning through implicit imitation. J. Artificial Intelligence Research 19, 569–629 (2003)MATHGoogle Scholar
  115. 115.
    Puterman, M.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Chichester (1994)MATHGoogle Scholar
  116. 116.
    Ramachandran, D., Amir, E.: Bayesian inverse reinforcement learning. In: 20th Int. Joint Conf. Artificial Intelligence, India (2007)Google Scholar
  117. 117.
    Ramachandran, V.: Mirror neurons and imitation learning as the driving force behind the great leap forward in human evolution. Edge 69 (2000)Google Scholar
  118. 118.
    Range, F., Viranyi, Z., Huber, L.: Selective imitation in domestic dogs. Current Biology 17(10), 868–872 (2007)CrossRefGoogle Scholar
  119. 119.
    Rao, R., Shon, A., Meltzoff, A.: A Bayesian model of imitation in infants and robots. In: Imitation and social learning in robots, humans, and animals. Cambridge University Press, Cambridge (2007)Google Scholar
  120. 120.
    Ratliff, N., Bagnell, J., Zinkevich, M.: Maximum margin planning. In: Proc. 23rd Int. Conf. Machine Learning, pp. 729–736 (2006)Google Scholar
  121. 121.
    Robins, B., Dautenhahn, K., Dubowski, J.: Robots as isolators or mediators for children with autism? a cautionary tale. In: AISB 2005: Social Intelligence and Interaction in Animals, Robots and Agents, Hatfield, UK, pp. 12–15 (2005)Google Scholar
  122. 122.
    Sahin, E., Cakmak, M., Dogar, M., Ugur, E., Ucoluk, G.: To afford or not to afford: A new formalization of affordances towards affordance-based robot control. Adaptive Behavior 15(5), 447–472 (2007)CrossRefGoogle Scholar
  123. 123.
    Sauser, E., Billard, A.: View sensitive cells as a neural basis for the representation of others in a self-centered frame of reference. In: 3rd Int. Symp. Imitation in Animals & Artifacts (2005)Google Scholar
  124. 124.
    Sauser, E., Billard, A.: Parallel and Distributed Neural Models of the Ideomotor Principle: An Investigation of Imitative Cortical Pathways. Neural Networks 19(3), 285–298 (2006)MATHCrossRefGoogle Scholar
  125. 125.
    Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. In: International Journal of Robotics Research, IJRR (2008)Google Scholar
  126. 126.
    Saxena, A., Wong, L., Ng, A.Y.: Learning grasp strategies with partial shape information. In: AAAI (2008)Google Scholar
  127. 127.
    Schaal, S.: Is imitation learning the route to humanoid robots. Trends in Cognitive Sciences 3(6), 233–242 (1999)CrossRefGoogle Scholar
  128. 128.
    Schaal, S., Ijspeert, A., Billard, A.: Computational approaches to motor learning by imitation. Phil. Trans. of the Royal Society of London: Series B, Biological Sciences 358(1431) (2003)Google Scholar
  129. 129.
    Schaal, S., Peters, J., Nakanishi, J., Ijspeert, A.: Learning movement primitives. In: International Symposium on Robotics Research, ISRR 2003 (2003)Google Scholar
  130. 130.
    Shon, A., Grochow, K., Rao, R.: Robotic imitation from human motion capture using gaussian processes. In: IEEE/RAS International Conference on Humanoid Robots, Humanoids (2005)Google Scholar
  131. 131.
    Shon, A.P., Joshua, S.J., Rao, R.P.N.: Towards a real-time bayesian imitation system for a humanoid robot. In: IEEE - International Conference on Robotics and Automation, ICRA (2007)Google Scholar
  132. 132.
    Shon, A.P., Verma, D., Rao, R.P.N.: Active imitation learning. In: AAAI (2007)Google Scholar
  133. 133.
    Stoytchev, A.: Behavior-grounded representation of tool affordances. In: International Conference on Robotics and Automation, Barcelona, Spain (2005)Google Scholar
  134. 134.
    Stulp, F., Fedrizzi, A., Beetz, M.: Learning and performing place-based mobile manipulation. In: IEEE 8th International Conference on Development and Learning (2009)Google Scholar
  135. 135.
    Sturm, J., Plagemann, C., Burgard, W.: Adaptive body scheme models for robust robotic manipulation. In: RSS - Robotics Science and Systems IV, Zurich, Switzerland (2008)Google Scholar
  136. 136.
    Syed, U., Schapire, R., Bowling, M.: Apprenticeship learning using linear programming. In: Proceedings of the 25th International Conference on Machine Learning (ICML 2008), pp. 1032–1039 (2008)Google Scholar
  137. 137.
    Tani, J., Ito, M., Sugita, Y.: Self-organization of distributedly represented multiple behavior schemata in a mirror system: Reviews of robot experiments using rnnpb. Neural Networks 17(8/9), 1273–1289 (2004)CrossRefGoogle Scholar
  138. 138.
    Tennie, C., Call, J., Tomasello, M.: Push or pull: Imitation vs. emulation in great apes and human children. Ethology 112(12), 1159–1169 (2006)CrossRefGoogle Scholar
  139. 139.
    Thurau, C., Paczian, T., Sagerer, G.: Bayesian imitation learning in game characters. Int. J. Intelligent Systems Technologies and Applications 2(2/3) (2007)Google Scholar
  140. 140.
    Tomasello, M., Kruger, A.C., Ratner, H.H.: Cultural learning. Behavioral and Brain Sciences 16(3), 495–511 (1993)CrossRefGoogle Scholar
  141. 141.
    Turvey, M., Shockley, K., Carello, C.: Affordance, proper function, and the physical basis of perceived heaviness. Cognition 73 (1999)Google Scholar
  142. 142.
    Verma, D., Rao, R.: Goal-based imitation as probabilistic inference over graphical models. In: Advances in NIPS, vol. 18 (2006)Google Scholar
  143. 143.
    Visalberghi, E., Fragaszy, D.: “Do monkeys ape?”: ten years after. In: Imitation in Animals and Artifacts. MIT Press, Cambridge (2002)Google Scholar
  144. 144.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. Trans. PAM, 283–298 (2008)Google Scholar
  145. 145.
    Want, S., Harris, P.: Learning from other people’s mistakes: Causal understanding in learning to use a tool. Child Development 72(2), 431–443 (2001)CrossRefGoogle Scholar
  146. 146.
    Want, S.C., Harris, P.L.: How do children ape? Applying concepts from the study of non-human primates to the development study of “imitation” in children. Developmental Science 5(1), 1–13 (2002)CrossRefGoogle Scholar
  147. 147.
    Whiten, A., Custance, D., Gomez, J.C., Teixidor, P., Bard, K.A.: Imitative learning of artificial fruit processing in children (Homo sapiens) and chimpanzees (Pan troglodytes). Journal of Comparative Psychology 110, 3–14 (1996)CrossRefGoogle Scholar
  148. 148.
    Whiten, A., Horner, V., Litchfield, C.A., Marshall-Pescini, S.: How do apes ape? Learning & Behavior 32(1), 36–52 (2004)Google Scholar
  149. 149.
    Williams, T.G., Rowland, J.J., Lee, M.H.: Teaching from examples in assembly and manipulation of snack food ingredients by robot. In: 2001 IEEE/RSJ, International Conference on Intelligent Robots and Systems, pp. 2300–2305 (2001)Google Scholar
  150. 150.
    Williamson, R.A., Markman, E.M.: Precision of imitation as a function of preschoolers’ understanding of the goal of the demonstration. Developmental Psychology 42(4), 723–731 (2006)CrossRefGoogle Scholar
  151. 151.
    Wolpert, D.M., Doya, K., Kawato, M.: A unifying computational framework for motor control and social interaction. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 358(1431), 593–602 (2003)CrossRefGoogle Scholar
  152. 152.
    Wolpert, D.M., Kawato, M.: Multiple paired forward and inverse models for motor control. Neural Networks 11(7-8), 1317–1329 (1998)CrossRefGoogle Scholar
  153. 153.
    Yang, J., Xu, Y., Chen, C.: Hidden markov model approach to skill learning and its application to telerobotics. IEEE Transations on Robotics and Automation 10(5), 621–631 (1994)CrossRefGoogle Scholar
  154. 154.
    Zentall, T.R.: Imitation in animals: Evidence, function, and mechanisms. Cybernetics and Systems 32(1), 53–96 (2001)MATHCrossRefGoogle Scholar
  155. 155.
    Zhang, J., Rössler, B.: Self-valuing learning and generalization with application in visually guided grasping of complex objects. Robotics and Autonomous Systems 47, 117–127 (2004)CrossRefGoogle Scholar
  156. 156.
    Ziebart, B., Maas, A., Bagnell, J., Dey, A.: Maximum entropy inverse reinforcement learning. In: Proc. 23rd AAAI Conf. Artificial Intelligence, pp. 1433–1438 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Manuel Lopes
    • 1
  • Francisco Melo
    • 2
  • Luis Montesano
    • 3
  • José Santos-Victor
    • 4
  1. 1.Instituto de Sistemas e RobóticaInstituto Superior TécnicoLisbonPortugal
  2. 2.Carnegie Mellon UniversityPittsburghUSA
  3. 3.Universidad de ZaragozaZaragozaSpain
  4. 4.Instituto de Sistemas e RobóticaInstituto Superior TécnicoLisbonPortugal

Personalised recommendations