Cognitive Processing

, Volume 8, Issue 3, pp 151–158 | Cite as

Prediction of intent in robotics and multi-agent systems

  • Yiannis DemirisEmail author


Moving beyond the stimulus contained in observable agent behaviour, i.e. understanding the underlying intent of the observed agent is of immense interest in a variety of domains that involve collaborative and competitive scenarios, for example assistive robotics, computer games, robot–human interaction, decision support and intelligent tutoring. This review paper examines approaches for performing action recognition and prediction of intent from a multi-disciplinary perspective, in both single robot and multi-agent scenarios, and analyses the underlying challenges, focusing mainly on generative approaches.


Forward Model Internal Model Inverse Model Motor Command Mirror Neuron 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



I would like to thank Simon Butler, Bálint Takács, Ant Dearden and Tom Carlson, as well as the anonymous reviewers for their comments.


  1. Acosta-Calderon CA, Hu H (2005) Robot imitation: body schema and body percept. J Appl Bionics Biomech 2(3–):131–48. ISSN 1176–322Google Scholar
  2. Aggarwal A, Cai Q (1999) Human motion analysis: a review. Comput Vis Image Underst 73(3):428–440CrossRefGoogle Scholar
  3. Anderson JR, Boyle CF, Corbett AT, Lewis MW (1990) Cognitive modelling and intelligent tutoring. Artif Intell 42:7–49CrossRefGoogle Scholar
  4. Arkin RC (1998) Behavior based robotics. MIT Press, CambridgeGoogle Scholar
  5. Beetz M, Kirchlechner B, Lames M (2005) Computerised real-time analysis of football games. Pervasive Comput 4(3):33–39CrossRefGoogle Scholar
  6. Bekkering H, Wohlschläger A, Gattis M (2000) Imitation of gestures in children is goal-directed. Q J Exp Psychol 53A:153–164CrossRefGoogle Scholar
  7. Bishop C (2006) Pattern recognition and machine learning. Springer, HeidelbergGoogle Scholar
  8. Blakemore S-J, Decety J (2001) From the perception of action to the understanding of intention. Nat Rev Neurosci 2:561–567PubMedGoogle Scholar
  9. Bratman ME (1990) What is Intention? In: Cohen PR, Morgan JL, Pollack ME (eds) Intentions in communication, MIT Press, Cambridge, pp 15–32Google Scholar
  10. Bratman ME (1992) Shared cooperative activity. Philos Rev 101(2):327–341CrossRefGoogle Scholar
  11. Breazeal C, Berlin M, Brooks A, Gray J, Thomaz A (2006) Using perspective taking to learn from ambiguous demonstrations. Rob Auton Syst 54(5):385–393CrossRefGoogle Scholar
  12. Brooks (1986) A robust layered control system for a mobile robot. IEEE J Robot Autom RA-2, pp 14–3Google Scholar
  13. Brooks R, Meltzoff AN (2002) The importance of eyes: how infants interpret adult looking behavior. Dev Psychol 38:958–966CrossRefPubMedGoogle Scholar
  14. Brooks R, Meltzoff AN (2005) The development of gaze following and its relation to language. Dev Sci 8:535–543CrossRefPubMedGoogle Scholar
  15. Buxton H (2003) Learning and understanding dynamic scene activity: a review. Image Vis Comput 21:125–136CrossRefGoogle Scholar
  16. Byrne RW, Russon AE (1998) Learning by imitation: a hierarchical approach. Behav Brain Sci 21(5):667–684PubMedGoogle Scholar
  17. Cohen PR, Levesque HJ (1990) Intention is choice with commitment. Artif Intell 42:213–261CrossRefGoogle Scholar
  18. Csibra G, Gergely G (1998) The teleological origins of mentalistic action explanations: a developmental hypothesis. Dev Sci 1:255–259CrossRefGoogle Scholar
  19. Csibra G, Gergely G (2007) ‘Obsessed with goals– functions and mechanisms of teleological interpretation of actions in humans. Acta Psychologica 124:60–78CrossRefPubMedGoogle Scholar
  20. Dearden A, Demiris Y (2005) Learning forward models for robots. In: Proceedings of the international joint conference on artificial intelligence (IJCAI), Edinburgh, pp 1440–445 Google Scholar
  21. Demiris Y, Hayes G (2002) Imitation as a dual-route process featuring predictive and learning components: a biologically plausible computational model. In: Dautenhahn K, Nehaniv CL (eds) Imitation in animals and artifacts. MIT Press, Cambridge, pp 327–361Google Scholar
  22. Demiris Y, Johnson M (2003) Distributed, predictive perception of actions: a biologically inspired robotics architecture for imitation and learning. Connect Sci 15:231–243CrossRefGoogle Scholar
  23. Demiris Y, Khadhouri B (2006) Hierarchical attentive multiple models for execution and recognition of actions. Rob Auton Syst 54:361–369CrossRefGoogle Scholar
  24. Demiris Y, Simmons G (2006) Perceiving the unusual: temporal properties of hierarchical motor representations for action perception. Neural Netw 19:272–284CrossRefPubMedGoogle Scholar
  25. Devaney M, Ram A (1998) Needles in a haystack: plan recognition in large spatial domains involving multiple agents. In: Proceedings of the 15th national conference on artificial intelligence, AAAI-98, pp 942–47Google Scholar
  26. Flanagan JR, Johansson RS (2003) Action plans used in action observation. Nature 424:769–771CrossRefPubMedGoogle Scholar
  27. Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119:593–609CrossRefPubMedGoogle Scholar
  28. Gleissner B, Meltzoff AN, Bekkering H (2000) Children’s coding of human action: cognitive factors influencing imitation in 3-year-olds. Dev Sci 3:405–414CrossRefGoogle Scholar
  29. Grosz BJ, Hunsberger L (2006) The dynamics of intention in collaborative activity. Cogn Syst Res 7:259–272CrossRefGoogle Scholar
  30. Hamilton A, Grafton ST (2007) The motor hierarchy: from kinematics to goals and intentions. In: Haggard P, Rossetti Y, Kawato M (eds) Sensorimotor foundations of higher cognition, attention and performance XXII, chap 18 (in press)Google Scholar
  31. Hesslow G (2002) Conscious thought as simulation of behaviour and perception. Trends Cogn Sci 6(6):242–247CrossRefPubMedGoogle Scholar
  32. Hommel B, Musseler J, Aschersleben G, Prinz W (2001) The theory of event coding (TEC): a framework for perception and action planning. Behav Brain Sci 24:849–937PubMedCrossRefGoogle Scholar
  33. Isham V (1981) An introduction to spatial point processes and markov random fields. Int Stat Rev 49:21–43CrossRefGoogle Scholar
  34. Jackson PL, Meltzoff AN, Decety J (2006) Neural circuits involved in imitation and perspective-taking. Neuroimage 31:429–439CrossRefPubMedGoogle Scholar
  35. Jain AK, Zhong Y, Dubuisson-Jolly M-P (1998) Deformable template models: a review. Signal Processing 71:109–129CrossRefGoogle Scholar
  36. Jansen B, Belpaeme T (2006) A computational model of intention reading in imitation. Rob Auton Syst 54(5):394–402CrossRefGoogle Scholar
  37. Johnson MR, Demiris Y (2005) Perceptual perspective taking and action recognition. Int J Adv Rob Syst 2:301–308Google Scholar
  38. Kanno T, Nakata K, Furuta K (2003) A method for team intention inference. Int J Hum Comput Stud 58:393–413CrossRefGoogle Scholar
  39. Karniel A (2002) Three creatures named forward model. Neural Netw 15:305–307CrossRefPubMedGoogle Scholar
  40. Kott A, McEneaney WM (eds) (2006) Adversarial reasoning: computational approaches to reading the opponent’s mind. Chapman & Hall/CRC, LondonGoogle Scholar
  41. Liberman AM, Cooper FS, Shankweiler DP, Studdert-Kennedy M (1967) Perception of the speech code. Psychol Rev 74:431–361CrossRefPubMedGoogle Scholar
  42. Meltzoff AN (1995) Understanding the intentions of others: re-enactment of intended acts by 18-month-old children. Dev Psychol 31:838–850CrossRefGoogle Scholar
  43. Meltzoff AN (2005) Imitation and other minds: the “Like Me–hypothesis. In: Hurley S, Chater N (eds) Perspectives on imitation: from neuroscience to social science. MIT Press, Cambridge, vol 2, pp 55–7Google Scholar
  44. Meltzoff AN (2007a) ‘Like me– a foundation for social cognition. Dev Sci 10:126–134CrossRefPubMedGoogle Scholar
  45. Meltzoff AN (2007b) The ‘like me–framework for recognizing and becoming an intentional agent. Acta Psychologica 124:26–43CrossRefPubMedGoogle Scholar
  46. Meltzoff AN, Brooks R (2004) Developmental changes in social cognition with an eye towards gaze following. In: Carpenter M, Tomasello M (eds) Action-based measures of infants–understanding of others–intentions and attention. Symposium conducted at the Biennial meeting of the International Conference on Infant Studies, ChicagoGoogle Scholar
  47. Miall RC, Wolpert DM (1996) Forward models for physiological motor control. Neural Netw 9:1265–1279CrossRefPubMedGoogle Scholar
  48. Moeslund TB, Granum E (2000) A survey of computer vision-based human motion capture. Comput Vis Image Underst 81(3):231–268CrossRefGoogle Scholar
  49. Moeslund TB, Hilton A, Kruger V (2006) A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Underst 104:90–126CrossRefGoogle Scholar
  50. Narendra KS, Balakrishnan J (1997) Adaptive control using multiple models. IEEE Trans Autom Control 42(2):171–187CrossRefGoogle Scholar
  51. Pezzulo G, Calvi G (2006) A schema based model of the praying mantis. From animals to animats. In: Proceedings of the 9th international conference on simulation of adaptive behaviour. Springer LNAI, vol 4095, pp 211–23Google Scholar
  52. Roweis S, Ghahramani Z (1999) A unifying review of linear Gaussian models. Neural Comput 11(2):305–345CrossRefPubMedGoogle Scholar
  53. Schaal S (1999) Is Imitation learning the route to humanoid robots?. Trends Cogn Sci 3:233–242CrossRefPubMedGoogle Scholar
  54. Schaal S, Ijspeert A, Billard A (2003) Computational approaches to motor learning by imitation. Philos Trans R Soc Lond B Biol Sci 358:537–547CrossRefPubMedGoogle Scholar
  55. Scott SK, Johnsrude IS (2003) The neuroanatomical and functional organisation of speech perception. Trends Neurosci 26(2):100–107CrossRefPubMedGoogle Scholar
  56. Sonenberg L, Tidhar G (1999) Observations on team-oriented mental state recognition. In: Proceedings of the IJCAI-1999 workshop on team modelling and plan recognitionGoogle Scholar
  57. Sukthankar G, Sycara K (2006) Simultaneous team assignment and behavior recognition from spatio-temporal agent traces. In: Proceedings of 21st national conference on artificial intelligence (AAAI-06)Google Scholar
  58. Tambe M (1996) Tracking dynamic team activity. In: Proceedings of the national conference on artificial intelligence (AAAI)Google Scholar
  59. Tani J, Nolfi S (1999) Learning to perceive the world as articulated: an approach for hierarchical learning in sensory motor systems. Neural Netw 12:1131–1141CrossRefPubMedGoogle Scholar
  60. Tomasello M, Carpenter M, Call K, Behne T, Moll H (2005) Understanding and sharing intentions: the origins of cultural cognition. Behav Brain Sci 28:675–735PubMedGoogle Scholar
  61. Trafton J, Cassimatis N, Bugajska M, Brock D, Mintz F, Schultz A (2005) Enabling effective human–robot interaction using perspective taking in robots. IEEE Trans Syst Man Cybern A Syst Hum 35(4):460–470CrossRefGoogle Scholar
  62. Wohlschlager A, Gattis M, Bekkering H (2003) Action generation and action perception in imitation: an instance of the ideomotor principle. Philos Trans R Soc Lond B Biol Sci 358:501–515CrossRefPubMedGoogle Scholar
  63. Wolpert DM, Kawato M (1998) Multiple paired forward and inverse models for motor control. Neural Netw 11:1317–1329CrossRefPubMedGoogle Scholar
  64. Wolpert DM, Doya K, Kawato M (2003) A unifying computational framework for motor control and social interaction. Philos Trans R Soc Lond B Biol Sci 358:593–602CrossRefPubMedGoogle Scholar

Copyright information

© Marta Olivetti Belardinelli and Springer-Verlag 2007

Authors and Affiliations

  1. 1.Department of Electrical and Electronic EngineeringImperial College LondonLondonUK

Personalised recommendations