International Journal of Social Robotics

, Volume 9, Issue 1, pp 33–49 | Cite as

A Model for Generating Socially-Appropriate Deictic Behaviors Towards People

  • Phoebe LiuEmail author
  • Dylan F. Glas
  • Takayuki Kanda
  • Hiroshi Ishiguro
  • Norihiro Hagita


Pointing behaviors are essential in enabling social robots to communicate about a particular object, person, or space. Yet, pointing to a person can be considered rude in many cultures, and as robots collaborate with humans in increasingly diverse environments, they will need to effectively refer to people in a socially-appropriate way. We confirmed in an empirical study that although people would point precisely to an object to indicate where it is, they were reluctant to do so when pointing to another person. We propose a model for selecting utterances and pointing behaviors towards people in terms of a balance between understandability and social appropriateness. Calibrating our proposed model based on empirical human behavior, we developed a system able to autonomously select among six deictic behaviors and execute them on a humanoid robot. We evaluated the system in an experiment in a shopping mall, and the results show that the robot’s deictic behavior was perceived by both the listener and the referent as more polite, more natural, and better overall when using our model, as compared with a model considering understandability alone.


Human–robot interaction Social robots Pointing gestures Deictic behavior 



We would like to thank Satoshi Koizumi for facilitating the smooth operation of the experiments. This study was funded in part by the Ministry of Internal Affairs and Communications of Japan and in part by JSPS KAKENHI Grant Number 25240042.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethical Approval

This research was conducted in compliance with the standards and regulations of our company’s ethical review board, which requires every experiment we conduct to be subject to a review and approval procedure according to strict ethical guidelines.


  1. 1.
    Liu P, Glas DF, Kanda T, Ishiguro H, Hagita N (2013) It’s not polite to point: generating socially-appropriate deictic behaviors towards people. In: 8th ACM/IEEE international conference on human–robot interaction, pp 267–274Google Scholar
  2. 2.
    Bennewitz M, Faber F, Joho D, Schreiber M, Behnke S (2005) Towards a humanoid museum guide robot that interacts with multiple persons. In: 2005 5th IEEE-RAS international conference on humanoid robots, pp 418–423Google Scholar
  3. 3.
    Shiomi M, Kanda T, Ishiguro H, Hagita N (2007) Interactive humanoid robots for a science museum. IEEE Intell Syst 22:25–32CrossRefGoogle Scholar
  4. 4.
    Nieuwenhuisen M, Behnke S (2013) Human-like interaction skills for the mobile communication robot robotinho. Int J Soc Robot 5:549–561CrossRefGoogle Scholar
  5. 5.
    Kanda T, Sato R, Saiwaki N, Ishiguro H (2007) A two-month field trial in an elementary school for long-term human–robot interaction. IEEE Trans Robot 23:962–971CrossRefGoogle Scholar
  6. 6.
    Berns K, Mehdi SA (2010) Use of an autonomous mobile robot for elderly care. In: Advanced technologies for enhancing quality of life (AT-EQUAL), pp 121–126Google Scholar
  7. 7.
    Sabelli AM, Kanda T, Hagita N (2011) A conversational robot in an elderly care center: an ethnographic study. In: 2011 6th ACM/IEEE international conference on Human–robot interaction (HRI), pp 37–44Google Scholar
  8. 8.
    Semwal VB, Katiyar SA, Chakraborty R, Nandi G (2015) Biologically-inspired push recovery capable bipedal locomotion modeling through hybrid automata. Robot Auton Syst 70:181–190CrossRefGoogle Scholar
  9. 9.
    Sugiyama O, Kanda T, Imai M, Ishiguro H, Hagita N, Anzai Y (2006) Humanlike conversation with gestures and verbal cues based on a three-layer attention-drawing model. Connect Sci 18:379–402CrossRefGoogle Scholar
  10. 10.
    Schmidt J, Hofemann N, Haasch A, Fritsch J, Sagerer G (2008) Interacting with a mobile robot: evaluating gestural object references. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 3804–3809Google Scholar
  11. 11.
    Sakurai S, Sato E, Yamaguchi T (2007) Recognizing pointing behavior using image processing for human–robot interaction. In: 2007 IEEE/ASME international conference on advanced intelligent mechatronics, pp 1–6Google Scholar
  12. 12.
    Holladay RM, Dragan AD, Srinivasa SS (2014) Legible robot pointing. In: 2014 RO-MAN: the 23rd IEEE international symposium on robot and human interactive communication, pp 217–223Google Scholar
  13. 13.
    Salem M, Kopp S, Wachsmuth I, Rohlfing K, Joublin F (2012) Generation and evaluation of communicative robot gesture. Int J Soc Robot 4:201–217CrossRefGoogle Scholar
  14. 14.
    Spexard T, Li S, Wrede B, Fritsch J, Sagerer G, Booij O et al (2006) BIRON, where are you? Enabling a robot to learn new places in a real home environment by integrating spoken dialog and visual localization. In: 2006 IEEE/RSJ international conference on intelligent robots and systems, pp 934–940Google Scholar
  15. 15.
    Hato Y, Satake S, Kanda T, Imai M, Hagita N (2010) Pointing to space: modeling of deictic interaction referring to regions. In: 2010 5th ACM/IEEE international conference on human–robot interaction (HRI), pp 301–308Google Scholar
  16. 16.
    Kendon A (2004) Gesture: visible action as utterance. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  17. 17.
    Van Der Sluis I, Krahmer E (2001) Generating referring expressions in a multimodal context an empirically oriented approach. Lang Comput 37:158–176Google Scholar
  18. 18.
    Haywood SL, Pickering MJ, Branigan HP (2005) Do speakers avoid ambiguities during dialogue? Psychol Sci 16:362–366CrossRefGoogle Scholar
  19. 19.
    Paraboni I, van Deemter K, Masthoff J (2007) Generating referring expressions: making referents easy to identify. Comput Linguist 33:229–254CrossRefGoogle Scholar
  20. 20.
    Vaish A, Kumari P (2014) A comparative study on machine learning algorithms in emotion state recognition using ECG. In: Proceedings of the second international conference on soft computing for problem solving (SocProS 2012), 28–30 December 2012, pp 1467–1476Google Scholar
  21. 21.
    Kumari P, Vaish A (2015) Information-theoretic measures on intrinsic mode function for the individual identification using EEG sensors. IEEE Sens J 15:4950–4960Google Scholar
  22. 22.
    Bangerter A., Chevalley, E (2007) Pointing and describing in referential communication: When are pointing gestures used to communicate? In: MOG 2007 workshop on multimodal output generation, p 17Google Scholar
  23. 23.
    Brooks AG, Breazeal C (2006) Working with robots and objects: Revisiting deictic reference for achieving spatial common ground. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on human–robot interaction, pp 297–304Google Scholar
  24. 24.
    Schultz AC, Trafton JG (2005) Towards collaboration with robots in shared space: spatial perspective and frames of reference. Interactions 12:22–24CrossRefGoogle Scholar
  25. 25.
    Glas DF, Miyashita T, Ishiguro H, Hagita N (2012) Human–robot interaction in social robotics. In: Kanda T, Ishiguro H (eds) Laser-based tracking of human position and orientation using parametric shape modeling. CRC Press, Boca Raton, pp 158–236Google Scholar
  26. 26.
    Burgoon JK, Buller DB, Hale JL, Turck MA (1984) Relational messages associated with nonverbal behaviors. Hum Commun Res 10:351–378CrossRefGoogle Scholar
  27. 27.
    Trout DL, Rosenfeld HM (1980) The effect of postural lean and body congruence on the judgment of psychotherapeutic rapport. J Nonverbal Behav 4:176–190CrossRefGoogle Scholar
  28. 28.
    Edelmann RJ (1982) The effect of embarrassed reactions upon others. Aust J Psychol 34:359–367CrossRefGoogle Scholar
  29. 29.
    Wolfe JM (1994) Guided search 2.0 A revised model of visual search. Psychon Bull Rev 1:202–238CrossRefGoogle Scholar
  30. 30.
    Sternberg S (1966) High-speed scanning in human memory. Science 153:652–654CrossRefGoogle Scholar
  31. 31.
    Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12:97–136CrossRefGoogle Scholar
  32. 32.
    Kranstedt, A, Lücking A, Pfeiffer T, Rieser H, Wachsmuth I (2006) Deixis: how to determine demonstrated objects using a pointing cone. In: Gesture in human–computer interaction and simulation. Springer, Heidelberg, pp 300–311Google Scholar
  33. 33.
    Kühnlein P, Stegmann J (2003) Empirical issues in deictic gesture: referring to objects in simple identification tasks. Report 2003/3, SFB, vol 360Google Scholar
  34. 34.
    Wolfe JM, Horowitz TS, Kenner N, Hyle M, Vasan N (2004) How fast can you change your mind? The speed of top-down guidance in visual search. Vis Res 44:1411–1426CrossRefGoogle Scholar
  35. 35.
    Wang Q, Cavanagh P, Green M (1994) Familiarity and pop-out in visual search. Percept Psychophys 56:495–500CrossRefGoogle Scholar
  36. 36.
    Glas DF, Miyashita T, Ishiguro H, Hagita N (2009) Laser-based tracking of human position and orientation using parametric shape modeling. Adv Robot 23:405–428CrossRefGoogle Scholar
  37. 37.
    Müller C (2004) Forms and uses of the Palm Up Open Hand: a case of a gesture family. In: Posner R, Müller C (eds) The semantics and pragmatics of everyday gestures. Weidler Buchverlag, Berlin, pp 234–256Google Scholar
  38. 38.
    Ishii S (1973) Characteristics of Japanese nonverbal communicative behavior. J Commun Assoc Pacific 2:43–60Google Scholar
  39. 39.
    Richie D (1998) A lateral view: essays on culture and style in contemporary Japan. Stone Bridge Press, BerkeleyGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • Phoebe Liu
    • 1
    Email author
  • Dylan F. Glas
    • 1
  • Takayuki Kanda
    • 1
  • Hiroshi Ishiguro
    • 1
  • Norihiro Hagita
    • 1
  1. 1.Advanced Telecommunications Research Institute InternationalKyotoJapan

Personalised recommendations