The Future of Human Robot Teams in the Army: Factors Affecting a Model of Human-System Dialogue Towards Greater Team Collaboration

  • A. William EvansEmail author
  • Matthew Marge
  • Ethan Stump
  • Garrett Warnell
  • Joseph Conroy
  • Douglas Summers-Stay
  • David Baran
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 499)


Understanding of intent is one of the most complex traits of highly efficient teams. Combining elements of verbal and non-verbal communication along with shared mental models about mission goals and team member capabilities, intent requires knowledge about both task and teammate. Beginning with the traditional models of communication, accounting for teaming factors, such as situation awareness, and incorporating the sensing, reasoning, and tactical capabilities available via autonomous systems, a revised model of team communication is needed to accurately describe the unique interactions and understanding of intent which will occur in human-robot teams. This paper focuses on examining the issue from a system capability viewpoint, identifying which system capabilities can mirror the abilities of humans through the sensor and computing strengths of autonomous systems, thus creating a team environment which is robust and adaptable while maintaining focus on mission goals.


Human-robot teaming Intent understanding Shared mental model Human-robot communication 


  1. 1.
    Intent. (n.d.): Retrieved February 21, 2016, from
  2. 2.
    Wang, J., Lewis, M.: Human control for cooperating robot teams. In: 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 9–16. IEEE, Mar 2007Google Scholar
  3. 3.
    Chen, J.Y., Barnes, M.J.: Human-agent teaming for multirobot control: a review of human factors issues. IEEE Trans. Hum. Mach. Syst. 44(1), 13–29 (2014)CrossRefGoogle Scholar
  4. 4.
    Doare, R., Danet, D., Hanon, J.P.: Robots on the battlefield: contemporary issues and implications for the future. Maroon Ebooks (2014)Google Scholar
  5. 5.
    Chen, T., Campbell, D., Gonzalez, F., Coppin, G.: The effect of autonomy transparency in human-robot interactions: a preliminary study on operator cognitive workload and situation awareness in multiple heterogeneous UAV management. In: Proceedings of Australasian Conference on Robotics and Automation 2014, pp. 1–10. Australian Robotics & Automation Association ARAA, Dec 2014Google Scholar
  6. 6.
    Zhang, Y., Narayanan, V., Chakraborti, T., Kambhampati, S.: A human factors analysis of proactive support in human-robot teaming. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2015)Google Scholar
  7. 7.
    Chen, J.Y., Barnes, M.J., Qu, Z.: RoboLeader: an agent for supervisory control of multiple robots. In: Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction, pp. 81–82. IEEE Press, Mar 2010Google Scholar
  8. 8.
    Schramm, W.: How communication works. In: Schramm, W. (ed.) The Process and Effects of Communication, pp. 3–26. University of Illinois Press, Champaign (1954)Google Scholar
  9. 9.
    Berlo, D.K.: The Process of Communication: An Introduction To Theory And Practice. Holt, Rinehart and Winston, New York (1960)Google Scholar
  10. 10.
    Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Human Factors J. Hum. Factors Ergon. Soc. 37(1), 32–64 (1995)CrossRefGoogle Scholar
  11. 11.
    Salas, E., Prince, C., Baker, D.P., Shrestha, L.: Situation awareness in team performance: implications for measurement and training. Hum. Factors J. Hum. Factors Ergon. Soc. 37(1), 123–136 (1995)CrossRefGoogle Scholar
  12. 12.
    Salas, E., Cooke, N.J., Rosen, M.A.: On teams, teamwork, and team performance: discoveries and developments. Hum. Factors J. Hum. Factors Ergon. Soc. 50(3), 540–547 (2008)CrossRefGoogle Scholar
  13. 13.
    Salas, E., Dickinson, T.L., Converse, S.A., Tannenbaum, S.I.: Toward an understanding of team performance and training (1992)Google Scholar
  14. 14.
    Converse, S.: Shared mental models in expert team decision making. Individ. Group Decis. Making Current 1993, 221 (1993)Google Scholar
  15. 15.
    Lim, B.C., Klein, K.J.: Team mental models and team performance: a field study of the effects of team mental model similarity and accuracy. J. Organ. Behav. 27(4), 403–418 (2006)CrossRefGoogle Scholar
  16. 16.
    Mathieu, J.E., Heffner, T.S., Goodwin, G.F., Salas, E., Cannon-Bowers, J.A.: The influence of shared mental models on team process and performance. J. Appl. Psychol. 85(2), 273 (2000)CrossRefGoogle Scholar
  17. 17.
    Rouse, W.B., Cannon-Bowers, J.A., Salas, E.: The role of mental models in team performance in complex systems. IEEE Trans. Syst. Man Cybern. 22(6), 1296–1308 (1992)CrossRefGoogle Scholar
  18. 18.
    Stout, R.J., Cannon-Bowers, J.A., Salas, E., Milanovich, D.M.: Planning, shared mental models, and coordinated performance: An empirical link is established. Hum. Factors J. Hum. Factors Ergon. Soc. 41(1), 61–71 (1999)CrossRefGoogle Scholar
  19. 19.
    Gurtner, A., Tschan, F., Semmer, N.K., Nägele, C.: Getting groups to develop good strategies: effects of reflexivity interventions on team process, team performance, and shared mental models. Organ. Behav. Hum. Decis. Process. 102(2), 127–142 (2007)CrossRefGoogle Scholar
  20. 20.
    Nikolaidis, S., Shah, J.: Human-robot teaming using shared mental models. ACM/IEEE HRI (2012)Google Scholar
  21. 21.
    Clark, H.H.: Using Language. Cambridge University Press, Cambridge (1996)Google Scholar
  22. 22.
    Clark, H.H., Brennan, S.E.: Grounding in communication. Perspect. Soc. Shared Cogn. 13(1991), 127–149 (1991)CrossRefGoogle Scholar
  23. 23.
    Bohus, D., Horvitz, E.: On the challenges and opportunities of physically situated dialog. In: AAAI Fall Symposium: Dialog with Robots, Nov 2010Google Scholar
  24. 24.
    Lemon, O., Bracy, A., Gruenstein, A., Peters, S.: The WITAS multi-modal dialogue system I. In: INTERSPEECH, pp. 1559–1562, Sept 2001Google Scholar
  25. 25.
    Marge, M., Pappu, A., Frisch, B., Harris, T.K., Rudnicky, A.I.: Exploring spoken dialog interaction in human-robot teams. In: Proceedings of Robots, Games, and Research: Success Stories in USARSim IROS Workshop (2009)Google Scholar
  26. 26.
    Marge, M., Rudnicky, A.I.: Miscommunication recovery in physically situated dialogue. In: Proceedings of SIGdial (2015)Google Scholar
  27. 27.
    Skantze, G.: Exploring human error recovery strategies: implications for spoken dialogue systems. Speech Commun. 45(3), 325–341 (2005)CrossRefGoogle Scholar
  28. 28.
    Streeck, J., Knapp, M.L.: The interaction of visual and verbal features in human communication. Adv. Nonverbal Commun. 3–23 (1992)Google Scholar
  29. 29.
    Baron-Cohen, S., Campbell, R., Karmiloff-Smith, A., Grant, J., Walker, J.: Are children with autism blind to the mentalistic significance of the eyes? Br. J. Dev. Psychol. 13(4), 379–398 (1995)CrossRefGoogle Scholar
  30. 30.
    Hoffman, M.W., Grimes, D.B., Shon, A.P., Rao, R.P.: A probabilistic model of gaze imitation and shared attention. Neural Netw. 19(3), 299–310 (2006)CrossRefzbMATHGoogle Scholar
  31. 31.
    Lockerd, A., Breazeal, C.: Tutelage and socially guided robot learning. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS 2004. Proceedings, vol. 4, pp. 3475–3480. IEEE, Sept 2004Google Scholar
  32. 32.
    Strabala, K.W., Lee, M.K., Dragan, A.D., Forlizzi, J.L., Srinivasa, S., Cakmak, M., Micelli, V.: Towards seamless human-robot handovers. J. Hum. Robot Interact. 2(1), 112–132 (2013)CrossRefGoogle Scholar
  33. 33.
    Huang, C.M., Andrist, S., Sauppé, A., Mutlu, B.: Using gaze patterns to predict task intent in collaboration. Front. Psychol. 6 (2015)Google Scholar
  34. 34.
    Handwerk, B.: 5 Surprising drone uses (Besides Amazon Delivery), National Geographic. Retrieved March 2, 2016 from
  35. 35.
    Charniak, E., Goldman, R.P.: Probabilistic Abduction for Plan Recognition. Brown University, Department of Computer Science (1991)Google Scholar
  36. 36.
    Sakita, K., Ogawara, K., Murakami, S., Kawamura, K., Ikeuchi, K.: Flexible cooperation between human and robot by interpreting human intention from gaze information. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS 2004. Proceedings, vol. 1, pp. 846–851. IEEE, Sept 2004Google Scholar
  37. 37.
    Park, H.S., Jain, E., Sheikh, Y.: 3d social saliency from head-mounted cameras. In: Advances in Neural Information Processing Systems, pp. 431–439 (2012)Google Scholar
  38. 38.
    Soo Park, H., Shi, J.: Social saliency prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4777–4785 (2015)Google Scholar
  39. 39.
    Hopcroft, J., Motwani, R., Ullman, J.: Introduction to Automata Theory, Languages, and Computation. Pearson, London (2013)Google Scholar
  40. 40.
    Matuszek, C., Herbst, E., Zettlemoyer, L., Fox, D.: Learning to parse natural language commands to a robot control system. In: Proceedings of the International Symposium on Experimental Robotics, pp. 403–415 (2013)Google Scholar
  41. 41.
    Dantam, N., Stilman, M.: The motion grammar: analysis of a linguistic method for robot control. IEEE Trans. Robot. 29(3), 704–718 (2013)CrossRefGoogle Scholar
  42. 42.
    Kress-Gazit, H., Fainekos, G.E., Pappas, G.J.: Temporal-logic-based reactive mission and motion planning. IEEE Trans. Robot. 25(6), 1370–1381 (2009)CrossRefGoogle Scholar
  43. 43.
    Parks, D., Borji, A., Itti, L.: Augmented saliency model using automatic 3d head pose detection and learned gaze following in natural scenes. Vision. Res. 116, 113–126 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • A. William Evans
    • 1
    Email author
  • Matthew Marge
    • 1
  • Ethan Stump
    • 1
  • Garrett Warnell
    • 1
  • Joseph Conroy
    • 1
  • Douglas Summers-Stay
    • 1
  • David Baran
    • 1
  1. 1.US Army Research LaboratoryAdelphiUSA

Personalised recommendations