Transmitting Human Presence Through Portable Teleoperated Androids: A Minimal Design Approach

  • Hiroshi IshiguroEmail author


In this article, results from the JST/CREST project “Studies on cellphone-type teleoperated androids transmitting human presence” are described. The goal of this project is to develop new robotic communication devices that can provide a strong feeling of human presence, called “sonzaikan”, by distance and to clarify the principles of human nature behind this. We took a synthetic approach, exploring three bidirectional processes: system development, field studies, and evaluation. We first define sonzaikan , which forms the core concept of this study, and then in the following sections describe various implementations of sonzaikan media prototypes and supporting technologies, evaluations of sonzaikan media, and field studies using prototypes we developed. We discuss the results in detail and then conclude this article with a brief summary of our essential findings and a mention of future work.


Teleoperated android Minimal design Human presence Robotic media Telenoid Elfoid Hugvie 



This project has been a collaborative work between several researchers from Advanced Telecommunications Research Institute International (ATR), Osaka University, and Tottori University. As principle investigator, I, Hiroshi Ishiguro, deeply appreciate the active collaborative work of everyone in the project. Dr. Shuichi Nishio at ATR took the initiative with the Telenoid field tests in Denmark and Japan. In addition, he helped me to summarize this report. Dr. Takashi Minato at ATR developed various versions of Telenoid and Elfoid . Dr. Carlos T. Ishi developed the software system that generates lip and head movements from the operator’s voice. Dr. Hidenobu Sumioka took the initiative with the Hugvie studies in elementary schools. Dr. Ryuji Yamazaki at ATR contributed to the field tests with Telenoid in Denmark. Prof. Hideyuki Nakanishi at Osaka University took the initiative with the psychological studies with Telenoid. Prof. Kazumasa Tanaka at Osaka University worked with Prof. Hideyuki Nakanishi and contributed to the psychological studies. Prof. Aya Nakae at Osaka University worked with Dr. Hidenobu Sumioka on the hormone tests with Hugvie. Prof. Yoshio Iwai at Tottori University developed the vision system of Telenoid. Prof. Hiroki Yoshimura and Prof. Maiya Hori worked with Prof. Yoshio Iwai on the vision system. In addition to these researchers, our students at ATR, Osaka University, and Tottori University made contributions that are much appreciated. This project has been very successful and has obtained a good reputation, especially in Japan and Denmark. We could not have made this achievement without the efforts of all these collaborators. This work has been supported by JST/CREST.


  1. 1.
    M. McPherson, L. Smith-Lovin, M.E. Brashears, Social isolation in america: changes in core discussion networks over two decades. Am. Sociol. Rev. 71(3), 353–375 (2006)CrossRefGoogle Scholar
  2. 2.
    M.O. Ernst, H.H. Bulthoff, Merging the senses into a robust percept. Trends Cogn. Sci. 8(4), 162–169 (2004)CrossRefGoogle Scholar
  3. 3.
    L. Shams, R. Kim, Crossmodal influences on visual perception. Phys. Life Rev. 7(3), 269–284 (2010)CrossRefGoogle Scholar
  4. 4.
    M.H. Giard, F. Peronnet, Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cognitive Neurosci. 11(5), 473–490 (1999)CrossRefGoogle Scholar
  5. 5.
    M.O. Ernst, M.S. Banks, Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870), 429–433 (2002)CrossRefGoogle Scholar
  6. 6.
    J.M. Hillis, M.O. Ernst, M.S. Banks, M.S. Landy, Combining sensory information: mandatory fusion within, but not between, senses. Science 298(5598), 1627–1630 (2002)CrossRefGoogle Scholar
  7. 7.
    H. Ishiguro, S. Nishio, Building artificial humans to understand humans. J. Artif. Organs 10(3), 133–142 (2007). doi: 10.1007/s10047-007-0381-4 CrossRefGoogle Scholar
  8. 8.
    S. Nishio, H. Ishiguro, N. Hagita, Geminoid: Teleoperated android of an existing person. in Humanoid Robots: New Developments ed. by A.C. de Pina Filho (I-Tech Education and Publishing, Vienna, Austria, 2007) pp. 343–352Google Scholar
  9. 9.
    B.R. Duffy, Anthropomorphism and the social robot. Robot. Auton. Syst. 42(3), 177–190 (2003)CrossRefzbMATHGoogle Scholar
  10. 10.
    B. Reeves, C. Nass, The Media Equation (Cambridge university press, Cambridge, 1996)Google Scholar
  11. 11.
    N. Matsumoto, H. Fujii, M. Okada, Minimal design for human-agent communication. J. AROB 10(1), 49–54 (2006)Google Scholar
  12. 12.
    H. Osawa, J. Mukai, M. Imai, Anthropomorphization framework for human-object communication. J. Adv. Comp. Intell. Intell. Inform. 11(8), 1007–1014 (2007)CrossRefGoogle Scholar
  13. 13.
    M. Ogata, Y. Sugiura, H. Osawa, M. Imai, in Proceedings of the Asia Pacific Conference on Computer Human Interaction. Pygmy: a ring-shaped robotic device that promotes the presence of an agent on human hand (2012), pp. 85–92Google Scholar
  14. 14.
    J.M. Beer, L. Takayama, in Proceedings of the International Conference on Human-robot Interaction, Mobile remote presence systems for older adults: acceptance, benefits, and concerns, pp. 19–26 (2011)Google Scholar
  15. 15.
    P. Thacker, Physician-robot makes the rounds. JAMA: J. Am. Med. Assoc. 293(2), 150–150 (2005)CrossRefGoogle Scholar
  16. 16.
    M. Lee, L. Takayama, in Proceedings of the Conference on Human Factors in Computing Systems. Now, i have a body: uses and social norms for mobile remote presence in the workplace, pp. 33–42 (2011)Google Scholar
  17. 17.
    E. Paulos, J. Canny, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Prop: personal roving presence (1998), pp. 296–303Google Scholar
  18. 18.
    T. Kashiwabara, H. Osawa, K. Shinozawa, M. Imai, in Proceedings of the ACM SIGCHI Conference on Human Factors in Computer Systems, Teroos: a wearable avatar to enhance joint activities, pp. 2001–2004 (2012)Google Scholar
  19. 19.
    Y. Tsumaki, Y. Fujita, A. Kasai, C. Sato, D. Nenchev, M. Uchiyama, in Proceedings of the International Workshop on Robot and Human Interactive Communication. Telecommunicator: a novel robot system for human communications, pp. 35 – 40, (2002). doi: 10.1109/ROMAN.2002.1045594
  20. 20.
    K. Ogawa, S. Nishio, K. Koda, G. Balistreri, T. Watanabe, H. Ishiguro, Exploring the natural reaction of young and aged person with telenoid in a real world. J. Adv. Comput. Intell. Intell. Inform. 15(5), 592–597 (2011)CrossRefGoogle Scholar
  21. 21.
    A. Kendon, Do gestures communicate? a review. Res. Lang. Soc. Interact. 27(3), 175–200 (1994)CrossRefGoogle Scholar
  22. 22.
    A. Mehrabian, Silent messages (Wadsworth Pub, Belmont, California, 1971)Google Scholar
  23. 23.
    K.R. Scherer, Vocal communication of emotion: a review of research paradigms. Speech Commun. 40(1–2), 227–256 (2003). doi: 10.1016/S0167-6393(02)00084-5 CrossRefzbMATHGoogle Scholar
  24. 24.
    D.I. Perrett, D. Burt, I.S. Penton-Voak, K.J. Lee, D.A. Rowland, R. Edwards, Symmetry and human facial attractiveness. Evol. Human Behav. 20(5), 295–307 (1999). doi: 10.1016/S1090-5138(99)00014-8 CrossRefGoogle Scholar
  25. 25.
    I. Straub, S. Nishio, H. Ishiguro, in IEEE International Symposium on Robot and Human Interactive Communication. Incorporated identity in interaction with a teleoperated android robot: a case study, Viareggio, Italy, pp. 139–144, (2010). doi: 10.1109/ROMAN.2010.5598695
  26. 26.
    N. Yee, J. Bailenson, The proteus effect: the effect of transformed self-representation on behavior. Hum. Commun. Res. 33, 271–290 (2007)CrossRefGoogle Scholar
  27. 27.
    K. Kuwamura, K. Sakai, T. Minato, S. Nishio, H. Ishiguro, in IEEE International Symposium on Robot and Human Interactive Communication. Hugvie: a medium that fosters love (Gyeongju, Korea, 2013) pp. 70–75, doi: 10.1109/ROMAN.2013.6628533
  28. 28.
    J. Nakanishi, K. Kuwamura, T. Minato, S. Nishio, H. Ishiguro, in the First International Conference on Human-Agent Interaction. Evoking affection for a communication partner by a robotic communication medium, (Hokkaido University, Sapporo, Japan, 2013), pp. III–1–4Google Scholar
  29. 29.
    T. Minato, H. Sumioka, S. Nishio, H. Ishiguro, in the RO-MAN 2012 workshop on social robotic telepresence. Studying the influence of handheld robotic media on social communications, (France, Paris, 2012), pp. 15–16Google Scholar
  30. 30.
    H. Sumioka, K. Koda, S. Nishio, T. Minato, H. Ishiguro, in IEEE International Symposium on Robot and Human Interactive Communication. Revisiting ancient design of human form for communication avatar: design considerations from chronological development of dogu, Gyeongju, Korea, pp. 726–731, (2013a). doi: 10.1109/ROMAN.2013.6628399
  31. 31.
    S. Kaner, D. Bailey (2009) The power of Dogū: Ceramic figures from ancient JapanGoogle Scholar
  32. 32.
    MIHO Museum (ed.) Dogu, a Cosmos (Hatori Press, 2012)Google Scholar
  33. 33.
    C.T. Ishi, C. Liu, H. Ishiguro, N. Hagita, Speech-driven lip motion generation for tele-operated humanoid robots, Processing of the the International Conference on Audio-Visual Speech (Volterra, Italy, 2011), pp. 131–135Google Scholar
  34. 34.
    C. Liu, C.T. Ishi, H. Ishiguro, N. Hagita (2013) Generation of nodding, head tilting and gazing for human-robot speech interaction. Int. J. Humanoid Rob. 10(1), 1350,009(1–19) (2013), doi: 10.1142/S0219843613500096
  35. 35.
    K. Sakai, H. Sumioka, T. Minato, S. Nishio, H. Ishiguro, Motion design of interactive small humanoid robot with visual illusion. Int. J. Innovative Comput. Inf. Control 9(12), 4725–4736 (2013)Google Scholar
  36. 36.
    Y. Fujie, M. Hori, H. Yoshimura, Y. Iwai, in Proceedings of the HRI2013 Workshop on Design of Humanlikeness in HRI from uncanny valley to minimal design, Emotion transmission by color effects for a teleoperated mobile communication robot, pp. 19–25 (2013)Google Scholar
  37. 37.
    R. Plutchik, Emotion: A Psychoevolutionary Synthesis (Harper and Row, New York 1980)Google Scholar
  38. 38.
    K. Tanaka, H. Nakanishi, H. Ishiguro, Appearance, Motion, and Embodiment: Unpacking Avatars by Fine-Grained Communication Analysis (Practice and Experience, Concurrency and Computation, 2015a). doi: 10.1002/cpe.3442 Google Scholar
  39. 39.
    K. Tanaka, H. Nakanishi, H. Ishiguro, Physical embodiment can produce robot operator’s pseudo presence. Frontiers in ICT 2(8), (2015). doi: 10.3389/fict.2015.00008
  40. 40.
    K. Kuwamura, T. Minato, S. Nishio, H. Ishiguro, Personality distortion in communication through teleoperated robots, IEEE Int. Symp. Robot Hum. Interact. Commun. (France, Paris, 2012), pp. 49–54Google Scholar
  41. 41.
    H. Gough, A. Heilbrun, The adjective check list manual (Consulting Psychologists Press, Palo Alto, 1983)Google Scholar
  42. 42.
    S. Wada, Construction of the Big Five Scales of personality trait terms and concurrent validity with NPI. Jpn. J. Psychol. 67(1), 61–67 (1996). in JapaneseCrossRefGoogle Scholar
  43. 43.
    D.G. Dutton, A.P. Aron, Some evidence for heightened sexual attraction under conditions of high anxiety. J. Pers. SoC Psychol. 30(4), 510–517 (1974). doi: 10.1037/h0037031 CrossRefGoogle Scholar
  44. 44.
    N. Nishimura, A. Ishi, M. Sato, S. Fukushima, H. Kajimoto, in CHI ’12 Extended Abstracts on Human Factors in Computing Systems. Facilitation of affection by tactile feedback of false heratbeat, pp. 2321–2326 (2012). doi: 10.1145/2212776.2223796
  45. 45.
    H. Sumioka, A. Nakae, R. Kanai, H. Ishiguro, Huggable communication medium decreases cortisol levels. Sci. Rep. 3(3034), (2013b). doi: 10.1038/srep03034
  46. 46.
    D.H. Hellhammer, S. Wust, B.M. Kudielka, Salivary cortisol as a biomarker in stress research. Psychoneuroendocrinology 34(2), 163–171 (2009)CrossRefGoogle Scholar
  47. 47.
    S. Cohen, Social relationships and health. Am. Psychol. 59, 676684 (2004)CrossRefGoogle Scholar
  48. 48.
    B. Ditzen, I.D. Neumann, G. Bodenmann, B. von Dawans, R.A. Turner, U. Ehlert, M. Heinrichs, Effects of different kinds of couple interaction on cortisol and heart rate responses to stress in women. Psychoneuroendocrinology 32(5), 565–574 (2007)CrossRefGoogle Scholar
  49. 49.
    T. Field, M. Hernandez-Reif, M. Diego, S. Schanberg, C. Kuhn, Cortisol decreases and serotonin and dopamine increase following massage therapy. Int. J. Neurosci. 115(10), 1397–1413 (2005)CrossRefGoogle Scholar
  50. 50.
    J. Holt-Lunstad, W.A. Birmingham, K.C. Light, Influence of a “warm touch” support enhancement intervention among married couples on ambulatory blood pressure, oxytocin, alpha amylase, and cortisol. Psychosom. Med. 70(9), 976–985 (2008)CrossRefGoogle Scholar
  51. 51.
    K.M. Grewen, S.S. Girdler, J. Amico, K.C. Light, Effects of partner support on resting oxytocin, cortisol, norepinephrine, and blood pressure before and after warm partner contact. Psychosom. Med. 67(4), 531–538 (2005)CrossRefGoogle Scholar
  52. 52.
    K.C. Light, K.M. Grewen, J.A. Amico, More frequent partner hugs and higher oxytocin levels are linked to lower blood pressure and heart rate in premenopausal women. Biol. Psychol. 69(1), 5–21 (2005). doi: 10.1016/j.biopsycho.2004.11.002. Current Trends in Women’s Health ResearchCrossRefGoogle Scholar
  53. 53.
    J.L. Edens, K.T. Larkin, J.L. Abel, The effect of social support and physical touch on cardiovascular reactions to mental stress. J. Psychosom. Res. 36(4), 371–381 (1992)CrossRefGoogle Scholar
  54. 54.
    W.J. Nilsen, S.R. Vrana, Some touching situations: the relationship between gender and contextual variables in cardiovascular responses to human touch. Ann. Behav. Med. 20(4), 270–276 (1998)CrossRefGoogle Scholar
  55. 55.
    P.E. Bee, P. Bower, K. Lovell, S. Gilbody, D. Richards, L. Gask, P. Roach, Psychotherapy mediated by remote communication technologies: a meta-analytic review. BMC Psychiatry 8, 60 (2008)CrossRefGoogle Scholar
  56. 56.
    R. Yamazaki, S. Nishio, K. Ogawa, H. Ishiguro, Teleoperated android as an embodied communication medium: a case study with demented elderlies in a care facility, IEEE Int. Symp. Robot Human Interact. Commun. (France, Paris, 2012), pp. 1066–1071Google Scholar
  57. 57.
    R. Yamazaki, S. Nishio, H. Ishiguro, M. Nørskov, N. Ishiguro, G. Balistreri, Acceptability of a teleoperated android by senior citizens in Danish society: a case study on the application of an embodied communication medium to home care. Int. J. Soc. Robot. 6(3), 429–442 (2014). doi: 10.1007/s12369-014-0247-x CrossRefGoogle Scholar
  58. 58.
    R. Yamazaki, S. Nishio, K. Ogawa, K. Matsumura, T. Minato, H. Ishiguro, T. Fujinami, M. Nishikawa, Promoting socialization of schoolchildren using a teleoperated android: an interaction study. Int. J. Humanoid Rob. 10(1), 1350,007(1–25) (2013), doi: 10.1142/S0219843613500072
  59. 59.
    J. Nakanishi, H. Sumioka, M. Shiomi, D. Nakamichi, K. Sakai, H. Ishiguro, in Proceedings of the Second International Conference on Human-agent Interaction. Huggable communication medium encourages listening to others, pp. 249–252 (2014). doi: 10.1145/2658861.2658934
  60. 60.
    A.D. Wolvin, Listening and Human Communication in the 21st Century (Wiley Online Library, 2010)Google Scholar
  61. 61.
    A. Wolvin, C. Coakley, Listening. (McGraw Hill, New York, 1996)Google Scholar
  62. 62.
    K. Nowak, in Presence 2001 4th Annual International Workshop. Defining and differentiating copresence, social presence and presence as transportation (2001)Google Scholar

Copyright information

© Springer Japan 2016

Authors and Affiliations

  1. 1.Advanced Telecommunications Research Institute InternationalKyotoJapan

Personalised recommendations