Cognitive Computation

, Volume 5, Issue 3, pp 340–354 | Cite as

Observational Learning: Basis, Experimental Results and Models, and Implications for Robotics

Article

Abstract

In this paper, we describe a brief survey of observational learning, with particular emphasis on how this could impact on the use of observational learning in robots. We present a set of simulations of a neural model which fits recent experimental data and such that it leads to the basic idea that observational learning uses simulations of internal models to represent the observed activity, so allowing for efficient learning of the observed actions. We conclude with a set of recommendations as to how observational learning might most efficiently be used in developing and training robots for their variety of tasks.

Keywords

Neural model Cognition Perception Action Inverse model Observational learning DARWIN robot 

References

  1. 1.
    Acosta-Calderon CA, Hu H. Imitation towards service robots and systems. International Conference on Intelligent Robots and Systems (IROS 2004), p. 3726–3731, Sendai, Japan, 2004.Google Scholar
  2. 2.
    Abbeel P, Ng AY. Apprenticeship learning via inverse reinforcement learning. In: Proceedings of International Conference on Machine Learning (ICML), 2004.Google Scholar
  3. 3.
    Argall B, Chernova S, Veloso M, Browning B. A survey of robot learning from demonstration. Rob Auton Syst. 2009;57(5):469–83.CrossRefGoogle Scholar
  4. 4.
    Atkeson CG, Schaal S. Learning tasks from a single demonstration. IEEE Int Conf Robot Autom. 1997;2:1706–12.CrossRefGoogle Scholar
  5. 5.
    Bandura A. Social learning theory. Englewood Cliffs, NJ: Prentice Hall; 1977.Google Scholar
  6. 6.
    Bandura A, Ross D, Ross SA. Transmission of aggressions through imitation of aggressive models. J Abnorm Soc Psychol. 1961;63:575–82.PubMedCrossRefGoogle Scholar
  7. 7.
    Braun DA, Aertsen A, Wolpert DM, Mehring C. Motor task variation induces structural learning. Curr Biol. 2009;19:352–7.PubMedCentralPubMedCrossRefGoogle Scholar
  8. 8.
    Braun DA, Mehring C, Wolpert DM. Structure learning in action. Behav Brain Res. 2010;206:157–65.PubMedCentralPubMedCrossRefGoogle Scholar
  9. 9.
    Braun DA, Waldert S, Aertsen A, Wolpert DM, Mehring C. Structure learning in a sensorimotor association task. PLoS ONE. 2010;5:e8973.PubMedCentralPubMedCrossRefGoogle Scholar
  10. 10.
    Billard A, Schaal S (eds). The brain mechanisms of imitation learning. Neural Netw. 2006;19:251–337.Google Scholar
  11. 11.
    Billard A, Dillmann (eds). The social mechanism of robot programming. Rob Auton Syst. 2006;54(5):354–418.Google Scholar
  12. 12.
    Calinon S, Billard A. What is the teachers role in robot programming by demonstration? Toward benchmarks for improved learning. Interact Stud. 2007;8(3):441–64.CrossRefGoogle Scholar
  13. 13.
    Demiris Y, Billard A. Robot learning by observation, demonstration and imitation. IEEE Trans Syst Man Cybern B. 2007;37:254–5.CrossRefGoogle Scholar
  14. 14.
    Fagard J, Lockman JJ. Change in imitation for object manipulation between 10 and 12 months of age. Dev Psychobiol. 2010;52(1):90–9.PubMedGoogle Scholar
  15. 15.
    Fiorito G, Scotto P. Observational learning in Octopus vulgaris. Science. 1992;256:545–7.PubMedCrossRefGoogle Scholar
  16. 16.
    Hartley M, Fagard J, Eseily R, Taylor JG. Observational versus trial and error learning effects in a model of an infant learning paradigm. In: Kurkova V, et al., editors. Lecture notes in computer science 5164. Berlin: Springer; 2008. p. 277–89.Google Scholar
  17. 17.
    Harris CM, Wolpert DM. Signal-dependent noise determines motor planning. Nature. 1998;394:780–4.PubMedCrossRefGoogle Scholar
  18. 18.
    Hongeng S, Wyatt J. Learning causality and intentional actions. In: Proceedings of the 2006 international conference on towards affordance-based robot control, Berlin, Heidelberg: Springer-Verlag; 2008. p. 27–46.Google Scholar
  19. 19.
    Hoppe F, Sommer G. Ensemble learning for hierarchies of locally arranged models. In: Proceedings of IEEE world congress on computational intelligence, Vancouver, Canada, July 16–21, 2006.Google Scholar
  20. 20.
    Kassahun Y, Sommer G. Efficient reinforcement learning through evolutionary acquisition of neural topologies. In: M Verleysen (ed) 13th European Symposium on Artificial Neural Networks, Bruges, Belgium, p. 259–266. D-side, April 2005.Google Scholar
  21. 21.
    Kormushev P, Nenchev D, Calinon S, Caldwell D. Upper-body kinesthetic teaching of a free-standing humanoid robot. In: IEEE International Conference on Robotics and Automation (ICRA), 2011. p. 3970–3975.Google Scholar
  22. 22.
    Lefebvre L. The opening of milk bottles by birds: evidence for accelerating learning rates but against the wave-of-advance model of cultural transmission. Behav Process. 1995;34(1):43–53.CrossRefGoogle Scholar
  23. 23.
    Hartley M, Taylor JG. A simple model of cortical activations during both observation and execution of reach-to-grasp movements. In: Lecture notes in computer science, vol 4669. Berlin: Springer; 2007. p. 899–911.Google Scholar
  24. 24.
    Miall RC. Connecting mirror neurons and forward models. NeuroReport. 2003;14(16):1–3.Google Scholar
  25. 25.
    Montessano L, et al. Learning object affordances: from sensory motor coordination to imitation. IEEE Trans Rob. 2006;24(1):15–26.CrossRefGoogle Scholar
  26. 26.
    Morimoto J, Doya K. Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning. Rob Auton Syst. 2001;36(1):37–51.CrossRefGoogle Scholar
  27. 27.
    Nanayakkara T, Sahin F, Jamshidi M. Intelligent control systems with an introduction to system of systems engineering. London: CRC Press, Taylor and Francis Group; 2009.CrossRefGoogle Scholar
  28. 28.
    Oztop E, Kawato M, Arbib M. Mirror neurons and imitation: a comprehensive review. Neural Netw. 2006;19:254–71.PubMedCrossRefGoogle Scholar
  29. 29.
    Raos V, Evangeliou MN, Savaki H. Observation of action: grasping with the mind’s hand. Neuroimage. 2004;23:193–201.PubMedCrossRefGoogle Scholar
  30. 30.
    Raos V, Evangeliou MN, Savaki H. Mental simulation of action in the service of action perception. J Neurosci. 2007;27:12675–83.PubMedCrossRefGoogle Scholar
  31. 31.
    Ratliff ND, Silver D, Bagnell JA. Learning to search: functional gradient techniques for imitation learning. Auton Rob. 2009;27(1):25–53.CrossRefGoogle Scholar
  32. 32.
    Rizolatti G, Craighero L. The mirror neuron system. Ann Rev Neurosci. 2004;27:169–92.CrossRefGoogle Scholar
  33. 33.
    Rizolatti G, et al. Neurophysiological mechanisms underlying the understanding and imitation of action. Nat Rev Neurosci. 2001;2:670–81.CrossRefGoogle Scholar
  34. 34.
    Schaal S. Learning from demonstration. In: Mozer MC, Jordan M, Petsche T, editors. Advances in neural information processing systems 9. Cambridge, MA: MIT Press; 1997. p. 1040–6.Google Scholar
  35. 35.
    Taylor JG. The perception-conceptualization-knowledge representation- reasoning representation-action cycle: the view from the brain. In: Cutsuridis V, et al., editors. Perception-action cycle: models, algorithms and hardware, Springer series in Cognitive and neural systems I. New York: Springer Science + Business Media, LLC; 2011. p. 243–85.CrossRefGoogle Scholar
  36. 36.
    Taylor JG, Hartley M. Towards a neural model of mental simulation. In: Kurkova V, et al., editors. Lecture notes in computer science 5164. Berlin: Springer; 2008. p. 969–79.Google Scholar
  37. 37.
    Taylor JG, Hartley M, Taylor NR, Panchev C, Kasderidis S. A hierarchical attention-based neural network architecture, based on human brain guidance, for perception, conceptualisation, action and reasoning. Image Vis Comput. 2009;27(11):1641–57.CrossRefGoogle Scholar
  38. 38.
    Young G. Are different affordances subserved by different neural pathways? Brain Cogn. 2006;62:134–42.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • John G. Taylor
    • 1
  • Vassilis Cutsuridis
    • 2
    • 4
  • Matthew Hartley
    • 3
  • Kaspar Althoefer
    • 2
  • Thrishantha Nanayakkara
    • 2
  1. 1.Department of MathematicsKings College LondonStrandUK
  2. 2.Division of EngineeringKings College LondonStrandUK
  3. 3.Department of Computational and Systems BiologyJohn Innes CentreNorwichUK
  4. 4.Institute of Molecular Biology and BiotechnologyFoundation of Research and Technology - Hellas (FORTH)Heracklion, CreteGreece

Personalised recommendations