International Journal of Social Robotics

, Volume 9, Issue 2, pp 181–198 | Cite as

Personal Greetings: Personalizing Robot Utterances Based on Novelty of Observed Behavior

  • Dylan F. GlasEmail author
  • Kanae Wada
  • Masahiro Shiomi
  • Takayuki Kanda
  • Hiroshi Ishiguro
  • Norihiro Hagita


One challenge in creating conversational service robots is how to reproduce the kind of individual recognition and attention that a human can provide. We believe that interactions can be made to seem more warm and humanlike by using sensors to observe a person’s behavior or appearance over time, and programming the robot to comment when it observes a novel feature, such as a new hairstyle, or a consistent behavior, such as visiting every afternoon. To create a system capable of recognizing such novelty and typicality, we collected one month of training data from customers in a shopping mall and recorded features of people’s visits, such as time of day and group size. We then trained SVM classifiers to identify each feature as novel, typical, or neither, based on the inputs of a human coder, and we trained an additional classifier to choose an appropriate topic for a personalized greeting. An utterance generator was developed to generate text for the robot to speak, based on the selected topic and sensor data. A cross-validation analysis showed that the trained classifiers could accurately reproduce human novelty judgments with 88% accuracy and topic selection with 95% accuracy. We then deployed a teleoperated robot using this system to greet customers in a shopping mall for three weeks, and we present example interactions and results from interviews showing that customers appreciated the robot’s personalized greetings and felt a sense of familiarity with the robot.


Human–robot interaction Social robots Novelty detection Greetings Personalized interaction 



We would like to thank Dr. Satoshi Koizumi, Tony Han, Benoit Toulmé, Peace Cho, and the management of the APiTA Town Keihanna shopping mall for their help in the organization and execution of the data collection.

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethical Approval

This research was conducted in compliance with the standards and regulations of our company’s ethical review board, which requires every experiment we conduct to be subject to a review and approval procedure according to strict ethical guidelines.


  1. 1.
    Friedman B, Kahn PH, Hagman J (2003) Hardware companions?-What online AIBO discussion forums reveal about the Human-Robotic relationship. Paper presented at the ACM conference on human factors in computing systems (CHI2003)Google Scholar
  2. 2.
    Kahn PH, Kanda T, Ishiguro H, Freier NG, Severson RL, Gill BT, Ruckert JH, Shen S (2012) “Robovie, you’ll have to go into the closet now”: children’s social and moral relationships with a humanoid robot. Dev Psychol 48(2):303CrossRefGoogle Scholar
  3. 3.
    Wada K, Shibata T (2007) Living with seal robots-its sociopsychological and physiological influences on the elderly at a care house. IEEE Trans Robot 23(5):972–980CrossRefGoogle Scholar
  4. 4.
    Sabelli AM, Kanda T, Hagita N (2011) A conversational Robot in an Elderly Care Center: an Ethnographic Study. Paper presented at the ACM/IEEE international conference on human-robot interaction (HRI2011)Google Scholar
  5. 5.
    Eyssel F, Reich N (2013) Loneliness makes the heart grow fonder (of robots)—on the effects of loneliness on psychological anthropomorphism. In: 8th ACM/IEEE international conference on human-robot interaction (HRI), pp 121–122Google Scholar
  6. 6.
    Bickmore TW, Picard RW (2005) Establishing and maintaining long-term human-computer relationships. ACM Trans Comput Hum Int 12(2):293–327CrossRefGoogle Scholar
  7. 7.
    Gockley R, Bruce A, Forlizzi J, Michalowski M, Mundell A, Rosenthal S, Sellner B, Simmons R, Snipes K, Schultz AC, Jue W (2005) Designing robots for long-term social interaction. In: Intelligent robots and systems (IROS 2005). 2005 IEEE/RSJ international conference on, 2-6 Aug. 2005, pp 1338-1343. doi: 10.1109/iros.2005.1545303
  8. 8.
    Leite I, Castellano G, Pereira A, Martinho C, Paiva A (2012) Long-term interactions with empathic robots: evaluating perceived support in children. Soc Robot 298–307Google Scholar
  9. 9.
    Kidd CD (2008) Designing for long-term human-robot interaction and application to weight loss. Massachusetts Institute of Technology, Cambridge, MAGoogle Scholar
  10. 10.
    Pan Y, Okada H, Uchiyama T, Suzuki K (2015) On the reaction to Robot’s speech in a hotel public space. Int J Soc Robot 7(5):911–920. doi: 10.1007/s12369-015-0320-0 CrossRefGoogle Scholar
  11. 11.
    Riek LD, Paul PC, Robinson P (2009) When my robot smiles at me: enabling human-robot rapport via real-time head gesture mimicry. J Multimodal User Interfaces 3(1–2):99–108Google Scholar
  12. 12.
    Sakamoto D, Kanda T, Ono T, Kamashima M, Imai M, Ishiguro H (2004) Cooperative embodied communication emerged by interactive humanoid robot. Int J Hum Comput Stud 62:247–265CrossRefGoogle Scholar
  13. 13.
    Sabelli AM, Kanda T, Hagita N (2011) A conversational robot in an elderly care center: an ethnographic study. In: 6th ACM/IEEE international conference on human-robot interaction (HRI), 8-11 March 2011, pp 37–44Google Scholar
  14. 14.
    Kanda T, Sato R, Saiwaki N, Ishiguro H (2007) A two-month field trial in an elementary school for long-term human-robot interaction. IEEE Trans Robot 23(5):962–971. doi: 10.1109/tro.2007.904904 CrossRefGoogle Scholar
  15. 15.
    Kanda T, Shiomi M, Miyashita Z, Ishiguro H, Hagita N (2010) A communication robot in a shopping mall. Trans Robot 26(5):897–913. doi: 10.1109/tro.2010.2062550 CrossRefGoogle Scholar
  16. 16.
    Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv 41(3):1–58. doi: 10.1145/1541880.1541882 CrossRefGoogle Scholar
  17. 17.
    Huber PJ (1981) Robust statistics, vol 1., Wiley series in probability and mathematical statisticsWiley, New YorkCrossRefzbMATHGoogle Scholar
  18. 18.
    Marsland S, Nehmzow U, Shapiro J (2005) On-line novelty detection for autonomous mobile robots. Robot Auton Syst 51(2):191–206CrossRefGoogle Scholar
  19. 19.
    Andry P, Gaussier P, Moga S, Banquet JP, Nadel J (2001) Learning and communication via imitation: an autonomous robot perspective. IEEE Trans Syst Man Cybern Part A 31(5):431–442. doi: 10.1109/3468.952717 CrossRefGoogle Scholar
  20. 20.
    Bonaccorsi M, Fiorini L, Cavallo F, Saffiotti A, Dario P (2016) A cloud robotics solution to improve social assistive robots for active and healthy aging. Int J Soc Robot 8(3):393–408. doi: 10.1007/s12369-016-0351-1 CrossRefGoogle Scholar
  21. 21.
    Glas DF, Miyashita T, Ishiguro H, Hagita N (2009) Laser-based tracking of human position and orientation using parametric shape modeling. Adv Robot 23(4):405–428. doi: 10.1163/156855309x408754 CrossRefGoogle Scholar
  22. 22.
    Lao S, Kawade M (2005) Vision-based face understanding technologies and their applications. In: Li SZ, Lai J, et al. (eds) Advances in biometric person authentication. Springer, New York, pp 339–348Google Scholar
  23. 23.
    Yücel Z, Zanlungo F, Ikeda T, Miyashita T, Hagita N (2013) Deciphering the crowd: modeling and identification of pedestrian group motion. Sensors 13(1):875–897CrossRefGoogle Scholar
  24. 24.
    Chang CC, Lin CJ (2001) LIBSVM: a library for support vector machinesGoogle Scholar
  25. 25.
    Shiwa T, Kanda T, Imai M, Ishiguro H, Hagita N How quickly should communication robots respond? In: Proceedings of the 3rd ACM/IEEE international conference on human robot interaction, Amsterdam, The Netherlands, 2008. ACM, pp 153-160. doi: 10.1145/1349822.1349843
  26. 26.
    Brščić D, Kanda T, Ikeda T, Miyashita T (2013) Person tracking in large public spaces using 3-D range sensors. IEEE Trans Hum Mach Syst 43(6):522–534. doi: 10.1109/thms.2013.2283945 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • Dylan F. Glas
    • 1
    Email author
  • Kanae Wada
    • 1
  • Masahiro Shiomi
    • 1
  • Takayuki Kanda
    • 1
  • Hiroshi Ishiguro
    • 2
  • Norihiro Hagita
    • 1
  1. 1.ATRKeihanna Science CityJapan
  2. 2.Osaka UniversityToyonakaJapan

Personalised recommendations