Interpreting Robot Pointing Behavior

  • Mary-Anne Williams
  • Shaukat Abidi
  • Peter Gärdenfors
  • Xun Wang
  • Benjamin Kuipers
  • Benjamin Johnston
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8239)


The ability to draw other agents’ attention to objects and events is an important skill on the critical path to effective human-robot collaboration. People use the act of pointing to draw other people’s attention to objects and events for a wide range of purposes. While there is significant work that aims to understand people’s pointing behavior, there is little work analyzing how people interpret robot pointing. Since robots have a wide range of physical bodies and cognitive architectures, interpreting pointing will be determined by a specific robot’s morphology and behavior. Humanoids and robots whose heads, torso and arms resemble humans that point may be easier for people to interpret, however if such robots have different perceptual capabilities to people then misinterpretation may occur. In this paper we investigate how ordinary people interpret the pointing behavior of a leading state-of-the-art service robot that has been designed to work closely with people. We tested three hypotheses about how robot pointing is interpreted. The most surprising finding was that the direction and pitch of the robot’s head was important in some conditions.


Human-robot interaction human-robot collaboration sociocognitive skills attention joint attention pointing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, R.: Cognitive psychology and its implications, 6th edn. Worth Publishers (2004)Google Scholar
  2. 2.
    Axelrod, R.: The Evolution of Cooperation. Basic Books, NY (1984)Google Scholar
  3. 3.
    Wright, R.D., Ward, L.M.: Orienting of Attention. Oxford University Press (2008)Google Scholar
  4. 4.
    Wolfe, J.M.: Guided search 2.0: a revised model of visual search. Psychonomic Bulletin Review 1(2), 202–238 (1994)CrossRefGoogle Scholar
  5. 5.
    Brinck, I.: Attention and the evolution of intentional communication. Pragmatics & Cognition 9(2), 255–272 (2001)CrossRefGoogle Scholar
  6. 6.
    Boyd, R., Richerson, P.J.: Culture and Evolutionary Process, Chicago (1985)Google Scholar
  7. 7.
    Castiello, U., Umilta, C.: Size of the attentional focus and efficiency of processing. Acta Psychologica 73(3), 195–209 (1990), doi:10.1016/0001-6918(90)90022-8; Corballis, M.C.: From Hand to Mouth. The Origins of Language, Princeton, NJ (2002)Google Scholar
  8. 8.
    Deutsch, J.A., Deutsch, D.: Attention: some theoretical considerations. Psychological Review 70, 80–90 (1963)CrossRefGoogle Scholar
  9. 9.
    Donald, M.: Origins of the Modern Mind. Three Stages in the Evolution of Culture and Cognition, Cambridge, MA (1991)Google Scholar
  10. 10.
    Cohen, P.R., Leveque, H.: Intention Is Choice with Commitment. Artificial Intelligence 42, 213–261 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Eriksen, C., St James, J.: Visual attention within and around the field of focal attention: A zoom lens model. Perception & Psychophysics 40(4), 225–240 (1986); Eriksen, C.W., Hoffman, J.E.: The extent of processing of noise elements during selective encoding from visual displays. Perception & Psychophysics 14(1), 155–160 (1973)Google Scholar
  12. 12.
    Givon, T., Malle, B. (eds.): The Evolution of Language out of Pre-Language, Philadelphia (2002)Google Scholar
  13. 13.
    Hauser, M.: The Evolution of Communication, Cambridge (1996)Google Scholar
  14. 14.
    Jonides, J.: Further towards a model of the mind’s eye’s movement. Bulletin of the Psychonomic Society 21(4), 247–250 (1983)CrossRefGoogle Scholar
  15. 15.
    Kopp, L., Gärdenfors, P.: Attention as a minimal criterion of intentionality in robots. Cognitive Science Quarterly 2, 302–319Google Scholar
  16. 16.
    Lavie, N., Hirst, A., de Fockert, J.W., Viding, E.: Load theory of selective attention and cognitive control. J. of Experimental Psychology 133(3), 339–354 (2004)CrossRefGoogle Scholar
  17. 17.
    McNeill, D.: Hand and Mind. What Gestures Reveal About Thought, Chicago (1992)Google Scholar
  18. 18.
    McNeill, D.: Gesture & Thought. U. of Chicago Press (2005)Google Scholar
  19. 19.
    Novianto, R., Johnston, B., Williams, M.-A.: Attention in the ASMO Cognitive Architecture. In: International Conference on Biologically Inspired Cognitive Architectures (2010)Google Scholar
  20. 20.
    Rizzolatti, G., Arbib, M.A.: Language within Our Grasp. Trends in Neuroscience 21, 188–194 (1998)CrossRefGoogle Scholar
  21. 21.
    Schilbach, L., Wilms, M., Eickhoff, S.B., Romanzetti, S., Tepest, R., Bente, G., Shah, N.J., Fink, G.R., Vogeley, K.: Minds made for sharing: initiating joint attention recruits reward-related neurocircuitry. Journal of Cognitive Neuroscience 22(12), 2702–2715 (2010)CrossRefGoogle Scholar
  22. 22.
    Tomasello, M.: Why don’t apes point? In: Enfield, N.J., Levinson, S.C. (eds.) Roots of Human Sociality: Culture, Cognition and Interaction, Berg, pp. 506–524 (2006)Google Scholar
  23. 23.
    Tomasello, M.: Origins of Human Communication. Cambridge University Press (2009)Google Scholar
  24. 24.
    Treisman, A., Gelade, G.: A feature-integration theory of attention. Cognitive Psychology 12(1), 97–136 (1980)CrossRefGoogle Scholar
  25. 25.
    Williams, M.-A.: Robot Social Intelligence. In: Ge, S.S., Khatib, O., Cabibihan, J.-J., Simmons, R., Williams, M.-A. (eds.) ICSR 2012. LNCS (LNAI), vol. 7621, pp. 45–55. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  26. 26.
    Wnuczko, M., Kennedy, J.M.: Pivots for pointing: Visually-monitored pointing has higher arm elevations than pointing blindfolded. Journal of Experimental Psychology: Human Perception and Performance 37, 1485–1491 (2011)Google Scholar
  27. 27.
    Camaioni, L., Perucchini, P., Bellagamba, F., Colonnesi, C.: The Role of Declarative Pointing in Developing a Theory of Mind. Infancy 5(3), 291–308 (2004), doi:10.1207/s15327078in0503_3CrossRefGoogle Scholar
  28. 28.
    Carpenter, M., Nagell, K., Tomasello, M.: Social cognition, joint attention, and communicative competence from 9 to 15 months of age. Monographs of the Society for Research in Child Development 63(4, Serial No. 255) (1998)Google Scholar
  29. 29.
    Sodian, B., Thoermer, C.: Infants’ under-standing of looking, pointing, and reaching as cues to goal-directed action. J. of Cognition and Development 5, 289–316 (2004)CrossRefGoogle Scholar
  30. 30.
    Gärdenfors, P.: How Homo Became Sapiens. Oxford University Press, Oxford (2003)Google Scholar
  31. 31.
    Gärdenfors, P., Warglien, P.: The development of semantic space for pointing and verbal communication. To Appear in Conceptual Spaces and the Construal of Spatial Meaning: Empirical Evidence from Human Communication. Cambridge University Press, CambridgeGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Mary-Anne Williams
    • 1
  • Shaukat Abidi
    • 1
  • Peter Gärdenfors
    • 2
  • Xun Wang
    • 1
  • Benjamin Kuipers
    • 3
  • Benjamin Johnston
    • 1
  1. 1.Social Robotics StudioUniversity of TechnologySydneyAustralia
  2. 2.University of LundSweden
  3. 3.University of MichiganUSA

Personalised recommendations