Cognitive Human–Robot Interaction

  • Bilge MutluEmail author
  • Nicholas Roy
  • Selma Šabanović
Part of the Springer Handbooks book series (SHB)


A key research challenge in robotics is to design robotic systems with the cognitive capabilities necessary to support human–robot interaction. These systems will need to have appropriate representations of the world; the task at hand; the capabilities, expectations, and actions of their human counterparts; and how their own actions might affect the world, their task, and their human partners. Cognitive human–robot interaction is a research area that considers human(s), robot(s), and their joint actions as a cognitive system and seeks to create models, algorithms, and design guidelines to enable the design of such systems. Core research activities in this area include the development of representations and actions that allow robots to participate in joint activities with people; a deeper understanding of human expectations and cognitive responses to robot actions; and, models of joint activity for human–robot interaction. This chapter surveys these research activities by drawing on research questions and advances from a wide range of fields including computer science, cognitive science, linguistics, and robotics.


Joint Attention Social Robot Robot Interaction Robot Behavior Human Partner 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



behavior mark-up language


functional magnetic resonance imaging


focus of attention


field of view


human–computer interaction


HRI operating system


human–robot interaction


interaction agent


interaction unit


Markov decision process


open agent architecture


out of field


pattern-based mixed-initiative


partially observable Markov decision process


Robotics Science and Systems


spatial reasoning agent


extensible markup language


  1. 71.1
    T. Fong, C. Kunz, L.M. Hiatt, M. Bugajska: The human-robot interaction operating system, Proc. 1st ACM SIGCHI/SIGART Conf. HRI, Salt Lake City (2006) pp. 41–48Google Scholar
  2. 71.2
    J.G. Trafton, A.C. Schultz, N.L. Cassimatis, L.M. Hiatt, D. Perzanowski, D.P. Brock, M.D. Bugajska, W. Adams: Communicating and collaborating with robotic agents. In: Cognition and Multi-Agent Interaction, ed. by R. Sun (Cambridge Univ. Press, New York 2006) pp. 252–278Google Scholar
  3. 71.3
    C.-M. Huang, B. Mutlu: Robot behavior toolkit: Generating effective social behaviors for robots, Proc. 7th ACM/IEEE Intl. Conf. HRI, Boston (2012) pp. 25–32Google Scholar
  4. 71.4
    A.H. Vera, H.A. Simon: Situated action: A symbolic interpretation, Cogn. Sci. 17(1), 7–48 (1993)CrossRefGoogle Scholar
  5. 71.5
    T. Winograd, F. Flores: Understanding Computers and Cognition: A New Foundation for Design (Ablex Publ., New York 1986)zbMATHGoogle Scholar
  6. 71.6
    L.A. Suchman: Plans and Situated Actions: The Problem of Human-Machine Communication (Cambridge Univ. Press, Cambridge 1987)Google Scholar
  7. 71.7
    A.N. Leont'ev: The problem of activity in psychology, J. Russ. East Eur. Psychol. 13(2), 4–33 (1974)CrossRefGoogle Scholar
  8. 71.8
    E. Hutchins: The social organization of distributed cognition. In: Perspectives on Socially Shared Cognition, ed. by L.B. Resnick, J.M. Levine, S.D. Teasley (American Psychological Association, Wachington, DC 1991)Google Scholar
  9. 71.9
    B. Gates: A robot in every home, Sci. Am. 296, 58–65 (2007)CrossRefGoogle Scholar
  10. 71.10
    C. Pantofaru, L. Takayama, T. Foote, B. Soto: Exploring the role of robots in home organization, Proc. 7th Annu. ACM/IEEE Intl. Conf. HRI, Boston (2012) pp. 327–334Google Scholar
  11. 71.11
    K.M. Tsui, M. Desai, H.A. Yanco, C. Uhlik: Exploring use cases for telepresence robots, ACM/IEEE 6th Int. Conf. HRI, Lausanne (2011) pp. 11–18Google Scholar
  12. 71.12
    F. Tanaka, A. Cicourel, J.R. Movellan: Socialization between toddlers and robots at an early childhood education center, Proc. Natl. Acad. Sci. USA 104(46), 17954–17958 (2007)CrossRefGoogle Scholar
  13. 71.13
    T. Kanda, R. Sato, N. Saiwaki, H. Ishiguro: A two-month field trial in an elementary school for long-term human-robot interaction, IEEE Trans. Robotics 23(5), 962–971 (2007)CrossRefGoogle Scholar
  14. 71.14
    B. Reeves, C. Nass: The Media Equation: How People Treat Computers, Television and New Media Like Real People and Places (Cambridge University Press, Cambridge 1996)Google Scholar
  15. 71.15
    S. Turkle, O. Daste, C. Breazeal, B. Scassellati: Encounters with Kismet and Cog: Children respond to relational artifacts, Proc. IEEE-RAS/RSJ Int. Conf. Humanoid Robots, Los Angeles (2004) pp. 1–20Google Scholar
  16. 71.16
    S. Turkle: Alone Together: Why We Expect More from Technology and Less from Each Other (Basic Books, New York, 2011)Google Scholar
  17. 71.17
    C. Nass, Y. Moon: Machines and mindlessness: Social responses to computers, J. Soc. Issues 56(1), 81–103 (2000)CrossRefGoogle Scholar
  18. 71.18
    S. Turkle: Evocative Objects: Things We Think With (MIT Press, Cambridge 2011)Google Scholar
  19. 71.19
    K. Wada, T. Shibata, Y. Kawaguchi: Long-term robot therapy in a health service facility for the aged – A case study for 5 years, Proc. 11th IEEE Int. Conf. Rehabil. Robotics, Kyoto (2009) pp. 930–933Google Scholar
  20. 71.20
    B.R. Duffy: Anthropomorphism and the social robot, Robotics Auton. Syst. 42(3-4), 177–190 (2003)zbMATHCrossRefGoogle Scholar
  21. 71.21
    S. Kiesler, A. Powers, S.R. Fussell, C. Torrey: Anthropomorphic interactions with a software agent and a robot, Soc. Cogn. 26(2), 168–180 (2008)CrossRefGoogle Scholar
  22. 71.22
    S.R. Fussell, S. Kiesler, L.D. Setlock, V. Yew: How people anthropomorphize robots, Proc. 3rd ACM/IEEE Int. Conf. HRI, Amsterdam (2008) pp. 145–152Google Scholar
  23. 71.23
    D.S. Syrdal, K. Dautenhahn, S.N. Woods, M.L. Walters, K.L. Koay: Looking good? Appearance preferences and robot personality inferences at zero acquaintance, AAAI Spring Symp.: Multidiscip. Collab. Socially Assist. Robotics, Stanford (2007) pp. 86–92Google Scholar
  24. 71.24
    K.F. MacDorman, T. Minato, M. Shimada, S. Itakura, S. Cowley, H. Ishiguro: Assessing human likeness by eye contact in an android testbed, Proc. XXVII Annu. Meet. Conf. Cogn. Sci. Soc., Stresa (2005) pp. 1373–1378Google Scholar
  25. 71.25
    F. Hegel, S. Krach, T. Kircher, B. Wrede, G. Sagerer: Understanding social robots: A user study on anthropomorphism, Proc. 17th IEEE Int. Symp. Robot Hum. Interact. Commun., Munich (2008) pp. 574–579Google Scholar
  26. 71.26
    M. Mori: The uncanny valley, Energy 7(4), 33–35 (1970)Google Scholar
  27. 71.27
    C. Bartneck, T. Kanda, H. Ishiguro, N. Hagita: My robotic Doppelganger – A critical look at the Uncanny Valley Theory, IEEE 18th Intl. Symp. Robot Hum. Interact. Commun., Toyama (2009) pp. 269–276Google Scholar
  28. 71.28
    M.L. Walters, D.S. Syrdal, K. Dautenhahn, R. te Boekhorst, K.L. Koay: Avoiding the uncanny valley: Robot appearance, personality and consistency of behavior in an attention-seeking home scenario for a robot companion, Auton. Robots 24(2), 159–178 (2008)CrossRefGoogle Scholar
  29. 71.29
    A.P. Saygin, T. Chaminade, H. Ishiguro, J. Driver, C. Frith: The thing that should not be: Predictive coding and the uncanny valley in perceiving human and humanoid robot actions, Soc. Cogn. Affect. Neurosci. 7(4), 413–422 (2012)CrossRefGoogle Scholar
  30. 71.30
    W. Mitchell, K.A. Szerszen Sr., A.S. Lu, P.W. Schermerhorn, M. Scheutz, K.F. MacDorman: A mismatch in the human realism of face and voice produces an uncanny valley, i-Perception 2, 10–12 (2011)CrossRefGoogle Scholar
  31. 71.31
    C. Breazeal: Designing Sociable Robots (MIT Press, Cambridge 2002)zbMATHGoogle Scholar
  32. 71.32
    N. Matsumoto, H. Fujii, M. Okada: Minimal design for human-agent communication, Artif. Life Robotics 10(1), 49–54 (2006)CrossRefGoogle Scholar
  33. 71.33
    H. Kozima, H. Yano: A Robot that Learns to Communicate with Human Caregivers, Proc. 1st Int. Workshop Epigenetic Robotics, Lund (2001) pp. 47–52Google Scholar
  34. 71.34
    A. Powers, A.D.I. Kramer, S. Lim, J. Kuo, S-l. Lee, S. Kiesler: Eliciting information from people with a gendered humanoid robot, IEEE 14th Int. Workshop Robot Hum. Interact. Commun., Nashville (2005) pp. 158–163Google Scholar
  35. 71.35
    K.M. Lee, N. Park, H. Song: Can a robot be perceived as a developing creature?: Effects of a robot's long-term cognitive developments on its social presence and people's social responses toward it, Human Commun. Res. 31(4), 538–563 (2005)Google Scholar
  36. 71.36
    J. Goetz, S. Kiesler, A. Powers: Matching robot appearance and behavior to tasks to improve human–robot cooperation, Proc. 12th IEEE Int. Workshop Robot Hum. Interact. Commun., Silicon Valley (2003) pp. 55–60Google Scholar
  37. 71.37
    M.K. Lee, S. Kiesler, J. Forlizzi, S. Srinivasa, P. Rybski: Gracefully mitigating breakdowns in robotic services, Proc. 6th ACM/IEEE Int. Conf. HRI, Lausanne (2010) pp. 203–210Google Scholar
  38. 71.38
    B. Shore: Culture in Mind: Cognition, Culture, and the Problem of Meaning (Oxford Univ. Press, Oxford 1996)Google Scholar
  39. 71.39
    V. Evers, H. Maldonado, T. Brodecki, P. Hinds: Relational vs. group self-construal: Untangling the role of national culture in HRI, Proc. 3rd ACM/IEEE Int. Conf. HRI, Amsterdam (2008)Google Scholar
  40. 71.40
    L. Wang, P.-L.P. Rau, V. Evers, B.K. Robinson, P. Hinds: When in Rome: The role of culture and context in adherence to robot recommendations, Proc. 5th ACM/IEEE Int. Conf. HRI, Osaka (2010) pp. 359–366Google Scholar
  41. 71.41
    S. Sabanovic: Robots in society, society in robots – Mutual shaping of society and technology as a framework for social robot design, Int. J. Soc. Robotics 2(4), 439–450 (2010)CrossRefGoogle Scholar
  42. 71.42
    G. Shaw-Garlock: Looking forward to sociable robots, Int. J. Soc. Robotics 1(3), 249–260 (2009)CrossRefGoogle Scholar
  43. 71.43
    P.H. Kahn, A.L. Reichert, H.E. Gary, T. Kanda, H. Ishiguro, S. Shen, J.H. Ruckert, B. Gill: The new ontological category hypothesis in human–robot interaction, Proc. 6th ACM/IEEE Int. Conf. HRI, Lausanne (2011) pp. 159–160Google Scholar
  44. 71.44
    P.H. Kahn, N.G. Freier, B. Friedman, R.L. Severson, E.N. Feldman: Social and moral relationships with robotic others?, IEEE 13th Int. Workshop Robot Hum. Interact. Commun., Kurashiki (2004) pp. 545–550Google Scholar
  45. 71.45
    S. Turkle: A Nascent Robotics Culture: New Complicities for Companionship (AAAI, Boston 2006)Google Scholar
  46. 71.46
    B. Scassellati: How developmental psychology and robotics complement each other, NSF/DARPA Workshop Dev. Learn. (MIT Press, CSAIL, Cambridge 2006)Google Scholar
  47. 71.47
    H. Ishiguro: Android science – toward a new cross-interdisciplinary framework, ICCS/CogSci Workshop Toward Soc. Mech. Android Sci., Stresa (2005) pp. 1–6Google Scholar
  48. 71.48
    K.F. MacDorman, H. Ishiguro: The uncanny advantage of using androids in cognitive and social science research, Interact. Stud. 7(3), 297–337 (2006)CrossRefGoogle Scholar
  49. 71.49
    M. Stanley, J. Sabini: On maintaining social norms: A field experiment in the subway. In: Advances in Environmental Psychology: The Urban Environment, ed. by A. Baum, J.E. Singer, S. Valins (Erlbaum Associates, Hillsdale 1978) pp. 31–40Google Scholar
  50. 71.50
    H. Kozima, M.P. Michalowski, C. Nakagawa: Keepon: A playful robot for research, therapy, and entertainment, Int. J. Soc. Robotics 1(1), 3–18 (2009)CrossRefGoogle Scholar
  51. 71.51
    K.F. MacDorman: Introduction to the special issue on android science, Connect. Sci. 18(4), 313–317 (2006)CrossRefGoogle Scholar
  52. 71.52
    J.J. Gibson: The Ecological Approach to Visual Perception (Houghton Mifflin, Boston 1979)Google Scholar
  53. 71.53
    E.S. Reed: Encountering the World: Toward an Ecological Psychology (Oxford Univ. Press, Oxford 1996)Google Scholar
  54. 71.54
    Y. Yamaji, T. Miyake, Y. Yoshiike, P.R.S. De Silva, M. Okada: STB: Human-dependent sociable trash box, Proc. 5th ACM/IEEE Int. Conf. HRI, Osaka (2010) pp. 197–198Google Scholar
  55. 71.55
    H. Ishiguro: Android science: Conscious and subconscious recognition, Connect. Sci. 18(4), 319–332 (2006)CrossRefGoogle Scholar
  56. 71.56
    S. Nishio, H. Ishiguro, N. Hagita: Geminoid: Teleoperated android of an existing person. In: Humanoid Robots, New Developments, ed. by A.C. De Pina Filho (InTech, Vienna 2007) pp. 343–352Google Scholar
  57. 71.57
    S. Nishio, H. Ishiguro, N. Hagita: Can a teleoperated robot represent personal presence? – A case study with children, Psychologia 50(4), 330–342 (2007)CrossRefGoogle Scholar
  58. 71.58
    M. Shimada, K. Yamauchi, T. Minato, H. Ishiguro, S. Itakura: Studying the influence of the chameleon effect on humans using an android, IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Nice (2008)Google Scholar
  59. 71.59
    J. Wainer, D.J. Feil-Seifer, D.A. Shell, M.J. Mataric: Embodiment and human-robot interaction: A taskbased perspective, Proc. 2nd ACM/IEEE Int. Conf. HRI, Washington (2007) pp. 872–877Google Scholar
  60. 71.60
    P. Schermerhorn, M. Scheutz: Disentangling the effects of robot affect, embodiment, and autonomy on human team members in a mixed-initiative task, Proc. 4th Int. Conf. Adv. Comput.–Hum. Interact., Gosier (2011) pp. 235–241Google Scholar
  61. 71.61
    W.A. Bainbridge, J.W. Hart, E.S. Kim, B. Scassellati: The benefits of interactions with physically present robots over video-displayed agents, Int. J. Soc. Robotics 1(2), 41–52 (2010)Google Scholar
  62. 71.62
    B. Mutlu: Designing embodied cues for dialog with robots, AI Magazine 32(4), 17–30 (2011)Google Scholar
  63. 71.63
    E.T. Hall: The Hidden Dimension (Anchor Books, New York 1966)Google Scholar
  64. 71.64
    L. Takayama, C. Pantofaru: Influences on proxemic behaviors in human–robot interaction, IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), St. Louis (2009) pp. 5495–5502Google Scholar
  65. 71.65
    D.S. Syrdal, K.L. Koay, M.L. Walters, K. Dautenhahn: A personalized robot companion? The role of individual differences on spatial preferences in HRI scenarios, Proc. 16th IEEE Int. Symp. Robot Hum. Interact. Commun., Jeju Island (2007) pp. 1143–1148Google Scholar
  66. 71.66
    J. Mumm, B. Mutlu: Human-robot proxemics: Physical and psychological distancing in human-robot interaction, Proc. 6th ACM/IEEE Int. Conf. HRI, Lausanne (2011) pp. 331–338Google Scholar
  67. 71.67
    M.L. Walters, M.A. Oskoei, D.S. Syrdal, K. Dautenhahn: A long-term human–robot proxemic study, Proc. 20th IEEE Int. Symp. Robot Hum. Interact. Commun., Atlanta (2011) pp. 137–142Google Scholar
  68. 71.68
    C. Yu, M. Scheutz, P. Schermerhorn: Investigating multimodal real-time patterns of joint attention in an HRI word learning task, Proc. 5th ACM/IEEE Int. Conf. HRI, Osaka (2010) pp. 309–316Google Scholar
  69. 71.69
    T. Yonezawa, H. Yamazoe, A. Utsumi, S. Abe: Gaze-communicative behavior of stuffed-toy robot with joint attention and eye contact based on ambient gaze-tracking, Proc. 9th Int. Conf. Multimodal Interfaces, Nagoya (2007) pp. 140–145CrossRefGoogle Scholar
  70. 71.70
    Y. Yoshikawa, K. Shinozawa, H. Ishiguro, N. Hagita, T. Miyamoto: Responsive robot gaze to interaction partner, Robotics Sci. Syst., Philadelphia (2006)Google Scholar
  71. 71.71
    B. Mutlu, T. Shiwa, T. Kanda, H. Ishiguro, N. Hagita: Footing in human-robot conversations: How robots might shape participant roles using gaze cues, Proc. 4th ACM/IEEE Int. Conf. HRI, San Diego, California (2009) pp. 61–68Google Scholar
  72. 71.72
    M. Staudte, M.W. Crocker: Visual attention in spoken human-robot interaction, Proc. 4th ACM/IEEE Int. Conf. HRI, San Diego (2009) pp. 77–84CrossRefGoogle Scholar
  73. 71.73
    B. Mutlu, J. Forlizzi, J.K. Hodgins: A storytelling robot: Modeling and evaluation of human-like gaze behavior, IEEE-RAS Conf. Humanoid Robots, Genoa (2006) pp. 518–523Google Scholar
  74. 71.74
    C. Yu, P. Schermerhorn, M. Scheutz: Adaptive eye gaze patterns in interactions with human and artificial agents, ACM Trans. Interact. Intell. Syst. 1(2), 1–25 (2012)CrossRefGoogle Scholar
  75. 71.75
    H. Admoni, C. Bank, J. Tan, M. Toneva, B. Scassellati: Robot gaze does not reflexively cue human attention, Proc. 33rd Annu. Conf. Cogn. Sci. Soc., Boston (2011) pp. 1983–1988Google Scholar
  76. 71.76
    W.S. Condon: Cultural microrhythms. In: Interaction Rhythms: Periodicity in Communicative Behavior, ed. by M. Davis (Human Sciences Press, New York 1982) pp. 53–76Google Scholar
  77. 71.77
    E. Goffman: Some context for content analysis: A view of the origins of structural studies of face-to-face interaction. In: Conducting Interaction: Patterns of Behavior in Focused Encounters, ed. by A. Kendon (Cambridge Univ. Press, Cambridge 1990) pp. 15–49Google Scholar
  78. 71.78
    C. Trevarthen: Can a robot hear music? Can a robot dance? Can a robot tell what it knows or intends to do? Can it feel pride or shame in company? – Questions of the nature of human vitality, Proc. 2nd Int. Workshop Epigenet. Robotics, Edinburgh (2002)Google Scholar
  79. 71.79
    M. Michalowski, S. Sabanovic, H. Kozima: A dancing robot for rhythmic social interaction, Proc. 2nd ACM/IEEE Int. Conf. HRI, Washington DC (2007) pp. 89–96Google Scholar
  80. 71.80
    M.P. Michalowski, R. Simmons, H. Kozima: Rhythmic attention in child-robot dance play, Proc. 18th IEEE Int. Symp. Robot Hum. Interact. Commun., Toyama (2009) pp. 816–821Google Scholar
  81. 71.81
    E. Avrunin, J. Hart, A. Douglas, B. Scassellati: Effects related to synchrony and repertoire in perceptions of robot dance, Proc. 6th ACM/IEEE Int. Conf. HRI, Lausanne (2011) pp. 93–100Google Scholar
  82. 71.82
    G. Hoffman, C. Breazeal: Anticipatory perceptual simulation for human-robot joint practice: Theory and application study, Proc. 23rd AAAI Conf. Artif. Intell., Chicago (2008) pp. 1357–1362Google Scholar
  83. 71.83
    G. Hoffman, G. Weinberg: Interactive improvisation with a robotic marimba player, Auton. Robots 31(2-3), 133–153 (2011)CrossRefGoogle Scholar
  84. 71.84
    G. Deàk, I. Fasel, J. Movellan: The emergence of shared attention: Using robots to test developmental theories, Proc. 1st Int. Workshop Epigenet. Robotics, Lund (2001) pp. 95–104Google Scholar
  85. 71.85
    K. Dautenhahn: Roles and functions of robots in human society: Implications from research in autism therapy, Robotica 21(4), 443–452 (2003)CrossRefGoogle Scholar
  86. 71.86
    H. Kozima, C. Nakagawa, Y. Yasuda: Wowing together: What facilitates social interactions in children with autistic spectrum disorders, Proc. 6th Int. Workshop Epigenet. Robotics Model. Cogn. Dev. Robotics Syst., Paris (2006) p. 177Google Scholar
  87. 71.87
    B. Scassellati: How social robots will help us to diagnose, treat, and understand autism, Proc. 12th Int. Symp. Robotics Res., San Francisco, ed. by S. Thrun, R.A. Brooks, H. Durrant-Whyte (Springer, Berlin, Heidelberg 2005) pp. 552–563Google Scholar
  88. 71.88
    D.J. Feil-Seifer, M.J. Mataric: B3IA: An architecture for autonomous robot-assisted behavior intervention for children with autism spectrum disorders, Proc. 17th IEEE Int. Workshop Robot Hum. Interact. Commun., Munich (2008) pp. 328–333Google Scholar
  89. 71.89
    H. Kozima, C. Nakagawa, Y. Yasuda: Interactive robots for communication-care: A case-study in autism therapy, Proc. 14th IEEE Int. Workshop Robot Hum. Interact. Commun., Nashville (2005) pp. 341–346Google Scholar
  90. 71.90
    H. Kozima, Y. Yasuda, C. Nakagawa: Social interaction facilitated by a minimally-designed robot: Findings from longitudinal therapeutic practices for autistic children, Proc. 16th IEEE Int. Symp. Robot Hum. interact. Commun., Jeju Island (2007) pp. 599–604Google Scholar
  91. 71.91
    H.A. Simon: The Sciences of the Artificial (MIT Press, Cambridge 1969)Google Scholar
  92. 71.92
    B. Adams, C.L. Breazeal, R.A. Brooks, B. Scassellati: Humanoid robots: A new kind of tool, IEEE Intell. Syst. Appl. 15(4), 25–31 (2000)CrossRefGoogle Scholar
  93. 71.93
    L.W. Barsalou, C. Breazeal, L.B. Smith: Cognition as coordinated non-cognition, Cogn. Process. 8(2), 79–91 (2007)CrossRefGoogle Scholar
  94. 71.94
    A.L. Thomaz, M. Berlin, C. Breazeal: An embodied computational model of social referencing, Proc. 14th IEEE Int. Workshop Robot Hum. Interact. Commun., Nashville (2005) pp. 591–598Google Scholar
  95. 71.95
    G. Hoffman, C. Breazeal: Robotic partners? Bodies and minds: An embodied approach to fluid human-robot collaboration, Proc. 5th Int. Workshop Cogn. Robotics, Boston (2006) pp. 95–102Google Scholar
  96. 71.96
    G. Hoffman: Effects of anticipatory action on human-robot teamwork efficiency, fluency, and perception of team, Proc. 2nd ACM/IEEE Int. Conf. HRI, Washington D.C. (2007) pp. 1–8Google Scholar
  97. 71.97
    Y. Demiris, A. Meltzoff: The robot in the crib: A developmental analysis of imitation skills in infants and robots, Infant Child Dev. 17(1), 43–53 (2008)CrossRefGoogle Scholar
  98. 71.98
    C. Nehaniv, K. Dautenhahn (Eds.): Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions (Cambridge Univ. Press, Cambridge 2009)Google Scholar
  99. 71.99
    B. Scassellati: Imitation and mechanisms of joint attention: A developmental structure for building social skills on a humanoid robot, Lect. Notes Comput. Sci. 1562, 176–195 (1999)CrossRefGoogle Scholar
  100. 71.100
    Y. Nagai, K. Hosoda, A. Morita, M. Asada: A constructive model for the development of joint attention, Connect. Sci. 15(4), 211–229 (2003)CrossRefGoogle Scholar
  101. 71.101
    F. Kaplan, V. Hafner: The challenges of joint attention, Proc. 4th Int. Workshop Epigenet. Robotics, Lund (2004) pp. 67–74Google Scholar
  102. 71.102
    C. Crick, M. Munz, B. Scassellati: Synchronization in social tasks: Robotic drumming, Proc. 15th IEEE Int. Workshop Robot Hum. Interact. Commun., Hatfield (2006) pp. 97–102Google Scholar
  103. 71.103
    P. Bakker, Y. Kuniyoshi: Robot see, robot do: An overview of robot imitation, AISB-96 Workshop Learn. Robots Animals, Brighton (1996) pp. 3–11Google Scholar
  104. 71.104
    R.S. Jackendoff: On beyond zebra: The relation of linguistic and visual information, Cognition 26, 89–114 (1987)CrossRefGoogle Scholar
  105. 71.105
    B. Landau, R.S. Jackendoff: What and where in spatial language and spatial cognition, Behav. Brain Sci. 16, 217–265 (1993)CrossRefGoogle Scholar
  106. 71.106
    L. Talmy: The fundamental system of spatial schemas in language. In: From Perception to Meaning: Image Schemas in Cognitive Linguistics, ed. by B. Hamp (Mouton de Gruyter, Berlin 2005)Google Scholar
  107. 71.107
    T.P. Regier: The Acquisition of Lexical Semantics for Spatial Terms: A Connectionist Model of Perceptual Categorization, Ph.D. Thesis (University of California at Berkeley, Berkeley 1992)Google Scholar
  108. 71.108
    J.D. Kelleher, F.J. Costello: Applying computational models of spatial prepositions to visually situated dialog, Comput. Linguist. 35(2), 271–306 (2008)CrossRefGoogle Scholar
  109. 71.109
    T.P. Regier, L.A. Carlson: Grounding spatial language in perception: An empirical and computational investigation, J. Exp. Psychol. 130(2), 273–298 (2001)CrossRefGoogle Scholar
  110. 71.110
    G. Bugmann, E. Klein, S. Lauria, T. Kyriacou: Corpus-based robotics: A route instruction example, Proc. 8th Conf. Intell. Auton. Syst. (IAS-8), Amsterdam (2004) pp. 96–103Google Scholar
  111. 71.111
    M. Levit, D. Roy: Interpretation of spatial language in a map navigation task, IEEE Trans. Syst. Man Cybern. B 37(3), 667–679 (2007)CrossRefGoogle Scholar
  112. 71.112
    M. MacMahon, B. Stankiewicz, B. Kuipers: Walk the talk: Connecting language, knowledge, and action in route instructions, Proc. Natl. Conf. Artif. Intell., Boston (2006) pp. 1475–1482Google Scholar
  113. 71.113
    H. Kress-Gazit, G.E. Fainekos: Translating structured English to robot controllers, Adv. Robotics 22, 1343–1359 (2008)CrossRefGoogle Scholar
  114. 71.114
    C. Matuszek, D. Fox, K. Koscher: Following directions using statistical machine translation, Proc. 5th ACM/IEEE Int. Conf. HRI, Nara (2010) pp. 251–258Google Scholar
  115. 71.115
    A. Vogel, D. Jurafsky: Learning to follow navigational directions, Proc. 48th Annu. Meet. Assoc. Comput. Linguist., Uppsala (2010) pp. 806–814Google Scholar
  116. 71.116
    S. Harnad: The symbol grounding problem, Physica D 43, 335–346 (1990)MathSciNetCrossRefGoogle Scholar
  117. 71.117
    T. Winograd: Procedures as a Representation for Data in a Computer Program for Understanding Natural Language, MIT Tech. Rep. TMAC-TR-84 (MIT, Cambridge 1971)Google Scholar
  118. 71.118
    K.Y. Hsiao, N. Mavridis, D. Roy: Coupling perception and simulation: Steps towards conversational robotics, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Las Vegas (2003) pp. 928–933Google Scholar
  119. 71.119
    D. Roy, K.Y. Hsiao, N. Mavridis: Conversational Robots: Building blocks for grounding word meanings, Proc. HLT-NAACL 2003 Workshop Learn. Word Mean. Non-Linguist. Data, Stroudsburg (2003) pp. 70–77CrossRefGoogle Scholar
  120. 71.120
    D. Roy: Semiotic schemas: A framework for grounding language in action and perception, Artif. Intell. 167(1-2), 170–205 (2005)CrossRefGoogle Scholar
  121. 71.121
    J. Dzifcak, M. Scheutz, C. Baral, P. Schermerhorn: What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution, IEEE Int. Conf. Robotics Autom. (ICRA), Kobe (2009) pp. 4163–4168Google Scholar
  122. 71.122
    Y. Sugita, J. Tani: Learning semantic combinatoriality from the interaction between linguistic and behavioral processes, Adapt. Behav. – Animals Animat. Softw. Agents Robots Adapt. Syst. 13(1), 33–52 (2005)Google Scholar
  123. 71.123
    J. Modayil, B. Kuipers: Autonomous development of a grounded object ontology by a learning robot, Proc. 22nd AAAI Conf. Artif. Intell., Vancouver (2007) pp. 1095–1101Google Scholar
  124. 71.124
    D. Marocco, A. Cangelosi, K. Fischer, T. Belpaeme: Grounding action words in the sensorimotor interaction with the world: Experiments with a simulated iCub humanoid robot, Front. Neurorobotics 4, 1–15 (2010)Google Scholar
  125. 71.125
    R. Ge, R.J. Mooney: A statistical semantic parser that integrates syntax and semantics, Proc. 9th Conf. Comput. Nat. Lang. Learn., Ann Arbor (2005) pp. 9–16Google Scholar
  126. 71.126
    N. Shimizu, A. Haas: Learning to follow navigational route instructions, Proc. 21st Int. Jt. Conf. Artif. Intell., Pasadena (2009) pp. 1488–1493Google Scholar
  127. 71.127
    S.R.K. Branavan, H. Chen, L.S. Zettlemoyer, R. Barzilay: Reinforcement learning for mapping instructions to actions, Proc. 47th Jt. Conf. Annu. Meet. Assoc. Comput. Linguist. 4th Int. Jt. Conf. Nat. Lang. Process. (AFNLP), Singapore (2009) pp. 82–90Google Scholar
  128. 71.128
    S.R.K. Branavan, D. Silver, R. Barzilay: Learning to win by reading manuals in a Monte-Carlo framework, Proc. 49th Annu. Meet. Assoc. Comput. Linguist. Hum. Lang. Technol., Portland (2011)Google Scholar
  129. 71.129
    T. Kollar, S. Tellex, D. Roy, N. Roy: Toward understanding natural language directions, Proc. 5th ACM/IEEE Int. Conf. HRI, Osaka (2010) pp. 259–266Google Scholar
  130. 71.130
    D. Bailey: When Push Comes to Shove: A Computational Model of the Role of Motor Control in the Acquisition of Action Verbs, Ph.D. Thesis (Univ. of California, Berkeley 1997)Google Scholar
  131. 71.131
    T. Kollar, S. Tellex, D. Roy, N. Roy: Grounding verbs of motion in natural language commands to robots, Proc. Int. Symp. Exp. Robotics, New Delhi (2010) pp. 31–47Google Scholar
  132. 71.132
    S. Tellex, T. Kollar, S. Dickerson, M.R. Walter, A.G. Banerjee, S. Teller, N. Roy: Understanding natural language commands for robotic navigation and mobile manipulation, Proc. Natl. Conf. Artif. Intell., San Francisco (2011)Google Scholar
  133. 71.133
    J.L. Burke, R.R. Murphy, M.D. Coovert, D.L. Riddle: Moonlight in Miami: Field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise, Hum.–Comput. Interact. 19(1/2), 85–116 (2004)CrossRefGoogle Scholar
  134. 71.134
    K. Stubbs, P.J. Hinds, D. Wettergreen: Autonomy and common ground in human-robot interaction: A field study, IEEE Intell. Syst. 22(2), 42–50 (2007)CrossRefGoogle Scholar
  135. 71.135
    S. Kiesler: Fostering common ground in human-robot interaction, Proc. 14th IEEE Int. Workshop Robot Hum. Interact. Commun., Nashville (2005) pp. 729–734Google Scholar
  136. 71.136
    T. Fong, C. Thorpe, C. Baur: Collaboration, dialogue, human-robot interaction, Robotics Res. 6, 255–266 (2003)CrossRefGoogle Scholar
  137. 71.137
    M.E. Foster, T. By, M. Rickert, A. Knoll: Human-robot dialogue for joint construction tasks, Proc. 8th Int. Conf. Mulltimodal Interfaces, Banff (2006) pp. 68–71Google Scholar
  138. 71.138
    S. Li, B. Wrede, G. Sagerer: A computational model of multi-modal grounding for human robot interaction, Proc. 7th SIGdial Workshop Discourse Dialogue, Sydney (2009) pp. 153–160Google Scholar
  139. 71.139
    P.R. Cohen, H.J. Levesque: Teamwork, Nous 25(4), 487–512 (1991)CrossRefGoogle Scholar
  140. 71.140
    J. Peltason, B. Wrede: Pamini: A framework for assembling mixed-initiative human-robot interaction from generic interaction patterns, Proc. 11th SIGdial Annu. Meet. Special Interest Group Discourse Dialogue, Tokyo (2010) pp. 229–232Google Scholar
  141. 71.141
    J. Peltason, B. Wrede: The curious robot as a case-study for comparing dialog systems, AI Magazine 32(4), 85–99 (2011)Google Scholar
  142. 71.142
    M.F. Schober: Spatial perspective-taking in conversation, Cognition 47(1), 1–24 (1993)CrossRefGoogle Scholar
  143. 71.143
    J.E. Hanna, M.K. Tanenhaus, J.C. Trueswell: The effects of common ground and perspective on domains of referential interpretation, J. Mem. Lang. 49(1), 43–61 (2003)CrossRefGoogle Scholar
  144. 71.144
    J.G. Trafton, N.L. Cassimatis, M.D. Bugajska, D.P. Brock, F.E. Mintz, A.C. Schultz: Enabling effective human–robot interaction using perspective-taking in robots, IEEE Trans. Syst. Man Cybern. A 35(4), 460–470 (2005)CrossRefGoogle Scholar
  145. 71.145
    M. Berlin, J. Gray, A.L. Thomaz, C. Breazeal: Perspective taking: An organizing principle for learning in human–robot interaction, Proc. 21st Natl. Conf. Artif. Intell., Boston (2006) p. 1444Google Scholar
  146. 71.146
    R. Moratz, K. Fischer, T. Tenbrink: Cognitive modeling of spatial reference for human-robot interaction, Int. J. Artif. Intell. Tools 10(04), 589–611 (2001)CrossRefGoogle Scholar
  147. 71.147
    R. Ros, S. Lemaignan, E.A. Sisbot, R. Alami, J. Steinwender, K. Hamann, F. Warneken: Which one? Grounding the referent based on efficient human-robot interaction, Proc. 19th IEEE Int. Symp. Robot Hum. Interact. Commun., Viareggio (2010) pp. 570–575CrossRefGoogle Scholar
  148. 71.148
    A. Holroyd, C. Rich, C.L. Sidner, B. Ponsler: Generating connection events for human-robot collaboration, Proc. 20th IEEE Int. Symp. Robot Hum. Interact. Commun., Atlanta (2011) pp. 241–246Google Scholar
  149. 71.149
    G. Butterworth, L. Grover: Joint visual attention, manual pointing, and preverbal communication in human infancy. In: Attention and Performance, Vol. 13: Motor Representation and Control, ed. by M. Jeannerod (Lawrence Erlbaum Assoc., Mahwah 1990) pp. 605–624Google Scholar
  150. 71.150
    C. Rich, P. Ponsler, A. Holroyd, C.L. Sidner: Recognizing engagement in human-robot interaction, Proc. 5th ACM/IEEE Int. Conf. HRI, Osaka (2010) pp. 375–382Google Scholar
  151. 71.151
    V. Gallese, A. Goldman: Mirror neurons and the simulation theory of mind-reading, Trends Cogn. Sci. 2(12), 493–501 (1998)CrossRefGoogle Scholar
  152. 71.152
    E. Bicho, W. Erlhagen, L. Louro, E. Costa e Silva: Neuro-cognitive mechanisms of decision making in joint action: A human–robot interaction study, Hum. Mov. Sci. 30(5), 846–868 (2011)CrossRefGoogle Scholar
  153. 71.153
    J. Gray, C. Breazeal, M. Berlin, A. Brooks, J. Lieberman: Action parsing and goal inference using self as simulator, Proc. 14th IEEE Int. Workshop Robot Hum. Interact. Commun., Nashville (2005) pp. 202–209Google Scholar
  154. 71.154
    M.N. Nicolescu, M.J. Mataric: Linking perception and action in a control architecture for human-robot domains, Proc. 36th Annu. Hawaii Int. Conf. Syst. Sci., Big Island (2003) pp. 10–20Google Scholar
  155. 71.155
    C. Breazeal, G. Hoffman, A. Lockerd: Teaching and working with robots as a collaboration, Proc. 3rd Int. Jt. Conf. Auton. Agents Multiagent Syst., New York, Vol. 3 (2004) pp. 1030–1037Google Scholar
  156. 71.156
    R. Alami, A. Clodic, V. Montreuil, E.A. Sisbot, R. Chatila: Task planning for human–robot interaction, Proc. 2005 Jt. Conf. Smart Obj. Ambient Intell. Innov. Context-Aware Serv. Usages Technol., Grenoble (2005) pp. 81–85Google Scholar
  157. 71.157
    R. Alami, A. Clodic, V. Montreuil, E.A. Sisbot, R. Chatila: Toward human-aware robot task planning, AAAI Spring Symp.: To Boldly Go where No Human-Robot Team Has Gone Before, Palo Alto (2006) pp. 39–46Google Scholar
  158. 71.158
    R. Bellman: Dynamic Programming (Princeton Univ. Press, Princeton 1957)zbMATHGoogle Scholar
  159. 71.159
    N. Roy, J. Pineau, S. Thrun: Spoken dialog management for robots, Proc. Assoc. Comput. Linguist., Hong Kong (2000) pp. 93–100Google Scholar
  160. 71.160
    J. Hoey, P. Poupart, C. Boutilier, A. Mihailidis: POMDP models for assistive technology, Proc. AAAI Fall Symp. Caring Mach., AI in Eldercare (2005)Google Scholar
  161. 71.161
    J. Williams, S. Young: Scaling up POMDPs for dialogue management: The summary POMDP method, Proc. IEEE Autom. Speech Recognit. Underst. Workshop, Cancun (2005)Google Scholar
  162. 71.162
    D. Litman, S. Singh, M. Kearns, M. Walker: NJFun: A reinforcement learning spoken dialogue system, Proc. ANLP/NAACL 2000 Workshop Conversat. Syst., Seattle (2000) pp. 17–20CrossRefGoogle Scholar
  163. 71.163
    F. Broz, I. Nourbakhsh, R. Simmons: Planning for human-robot interaction using time-state aggregated POMDPs, Proc. 23rd Conf. Artif. Intell., Chicago (2008) pp. 1339–1344Google Scholar
  164. 71.164
    F. Doshi, N. Roy: The permutable POMDP: Fast solutions to POMDPs for preference elicitation, Proc. 7th Int. Conf. Auton. Agents Multiagent Syst., Estoril (2008) pp. 493–500Google Scholar
  165. 71.165
    R. Wilcox, S. Nikolaidis, J. Shah: Optimization of temporal dynamics for adaptive human-robot interaction in assembly manufacturing, Proc. Robotics Sci. Syst., Sydney (2012) p. 441Google Scholar
  166. 71.166
    T. Prommer, H. Holzapfel, A. Waibel: Rapid simulation-driven reinforcement learning of multimodal dialog strategies in human–robot interaction, 9th Int. Conf. Spoken Lang. Process., Pittsburgh (2006)Google Scholar
  167. 71.167
    J.M. Porta, N. Vlassis, M. Spaan, P. Poupart: Point-based value iteration for continuous POMDP, J. Mach. Learn. Res. 7, 2329–2367 (2006)MathSciNetzbMATHGoogle Scholar
  168. 71.168
    F. Doshi, N. Roy: Efficient model learning for dialog management, Proc. 2nd ACM/IEEE Int. Conf. HRI, Arlington (2007) pp. 65–72Google Scholar
  169. 71.169
    F. Doshi, N. Roy: Spoken language interaction with model uncertainty: An adaptive human-robot interaction system, Connect. Sci. 20(4), 299–319 (2008)CrossRefGoogle Scholar
  170. 71.170
    M. Cakmak, A.L. Thomaz: Designing robot learners that ask good questions, Proc. 7th Annu. ACM/IEEE Int. Conf. HRI, Boston (2012) pp. 17–24Google Scholar
  171. 71.171
    M.A. Goodrich, D.R. Olsen: Seven principles of efficient human robot interaction, IEEE Int. Conf. Syst. Man Cybern., Washington D.C. (2003) pp. 3942–3948Google Scholar
  172. 71.172
    J.W. Crandall, M.A. Goodrich, D.R. Olsen, C.W. Nielsen: Validating human-robot interaction schemes in multitasking environments, IEEE Trans. Syst. Man Cybern. A 35(4), 438–449 (2005)CrossRefGoogle Scholar
  173. 71.173
    R. Sun (Ed.): Cognition and Multiagent Interaction: From Cognitive Modeling to Social Simulation (Cambridge Univ. Press, Cambridge 2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Department of Computer SciencesUniversity of Wisconsin–MadisonMadisonUSA
  2. 2.Department of Aeronautics and AstronauticsMassachusetts Institute of TechnologyCambridgeUSA
  3. 3.School of Informatics and ComputingIndiana University BloomingtonBloomingtonUSA

Personalised recommendations