Advertisement

Experience in System Design for Human-Robot Teaming in Urban Search and Rescue

  • G. J. M. KruijffEmail author
  • M. Janíček
  • S. Keshavdas
  • B. Larochelle
  • H. Zender
  • N. J. J. M. Smets
  • T. Mioch
  • M. A. Neerincx
  • J. V. Diggelen
  • F. Colas
  • M. Liu
  • F. Pomerleau
  • R. Siegwart
  • V. Hlaváč
  • T. Svoboda
  • T. Petříček
  • M. Reinstein
  • K. Zimmermann
  • F. Pirri
  • M. Gianni
  • P. Papadakis
  • A. Sinha
  • P. Balmer
  • N. Tomatis
  • R. Worst
  • T. Linder
  • H. Surmann
  • V. Tretyakov
  • S. Corrao
  • S. Pratzler-Wanczura
  • M. Sulk
Chapter
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 92)

Abstract

The paper describes experience with applying a user-centric design methodology in developing systems for human-robot teaming in Urban Search and Rescue. A human-robot team consists of several semi-autonomous robots (rovers/UGVs, microcopter/UAVs), several humans at an off-site command post (mission commander, UGV operators) and one on-site human (UAV operator). This system has been developed in close cooperation with several rescue organizations, and has been deployed in a real-life tunnel accident use case. The human-robot team jointly explores an accident site, communicating using a multi-modal team interface, and spoken dialogue. The paper describes the development of this complex socio-technical system per se, as well as recent experience in evaluating the performance of this system.

Keywords

Situation Awareness Autonomous Navigation Robot Behavior Command Post Automatic Speech Recognizer 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    J.F. Allen, C.R. Perrault, Analyzing intention in utterances. Artif. Intell. 15(3) (1980)Google Scholar
  2. 2.
    J.F. Allen, D.J. Litman. Discourse processing and common sense plans. In P.R. Cohen, J. Morgan, M. Pollack (eds.), Intentions and Communication. The MIT Press, 1990Google Scholar
  3. 3.
    A. Birk, S. Carpin, Rescue robotics—a crucial milestone on the road to autonomous systems. Adv. Robot. 20(5), 595–605 (2006)CrossRefGoogle Scholar
  4. 4.
    J.L. Burke, R.R. Murphy, M. Coovert, D. Riddle. Moonlight in Miami: An ethnographic study of human-robot interaction in USAR. Hum. Comput. Interact. 19(1–2), 85–116 (2004)Google Scholar
  5. 5.
    J. Carlson, R.R. Murphy, How UGVs physically fail in the field. IEEE Trans. Rob. 21(3), 423–437 (2005)Google Scholar
  6. 6.
    H. Clark, Using Language (Cambridge University Press, Cambridge, 1996)Google Scholar
  7. 7.
    A. Finzi, F. Pirri, Representing flexible temporal behaviors in the situation calculus, in Proceedings of the 19th International Joint Conference on Artificial Intelligence, IJCAI’05, (Morgan Kaufmann Publishers Inc, San Francisco, 2005), pp. 436–441Google Scholar
  8. 8.
    T. Fong, C. Thorpe, C. Baur, Multi-robot remote driving with collaborative control. IEEE Trans. Indus. Electron. 54(4), 699–704 (2003)CrossRefGoogle Scholar
  9. 9.
    M. Gianni, P. Papadakis, F. Pirri, M. Liu, F. Pomerleau, F. Colas, K. Zimmerman, T. Svoboda, T. Petricek, G. J. M. Kruijff, H. Zender, Khambhaita, A unified framework for planning and execution-monitoring of mobile robots, in Proceedings of the AAAI-11 Workshop on Automated Action Planning for Autonomous Mobile Robots, 2011Google Scholar
  10. 10.
    M. Gianni, P. Papadakis, F. Pirri, M. Pizzoli, Awareness in mixed initiative planning, in AAAI Fall Symposium Series, 2011Google Scholar
  11. 11.
    B.J. Grosz, C.L. Sidner, Attention, intention and the structure of discourse. Computat. Linguist. 12(3), 175–204 (1986)Google Scholar
  12. 12.
    C. Jansen, J.B.F. van Erp. Telepresence control of unmanned systems. in M. Barnes, F. Jentsch, eds. Human-Robot Interactions in Future Military Operations (Ashgate Publishing Limited, Hampshire, 2010), pp. 251–270Google Scholar
  13. 13.
    M. Johnson, J.M. Bradshaw, P.J. Feltovich, R.R. Hoffman, C. Jonker, B. van Riemsdijk, M. Sierhuis, Beyond cooperative robotics: the central role of interdependence in coactive design. IEEE Intelligent Systems, pp. 81–88, May/June 2011Google Scholar
  14. 14.
    S. Keshavdas, H. Zender, G.J.M Kruijff, M. Liu, F. Colas, Functional mapping: spatial inferencing to aid human-robot rescue efforts in unstructured disaster environments, in Proceedings of the 2012 AAAI Spring Symposium on Designing Intelligent Robots (2012)Google Scholar
  15. 15.
    H. Khambhaita, G.J.M. Kruijff, M. Mancas, M. Gianni, P. Papadakis, F. Pirri, M. Pizzoli, Help me to help you: how to learn intentions, actions and plans, in Proceedings of AAAI Spring Symposium Help Me Help You: Bridging the Gaps in Human-Agent Collaboration, March 2011Google Scholar
  16. 16.
    G. Klein, D.D. Woods, J.M. Bradshaw, R. Hoffman, P. Feltovich, Ten challenges for making automation a team player in joint human-agent activity. IEEE Intell. Syst. 19(6), 91–95, Nov–Dec 2004Google Scholar
  17. 17.
    T. Komatsu, S. Yamada, Adaptation gap hypothesis: how differences between users’ expected and perceived agent functions affect their subjective impression. J. Syst. Cybern. Inform. 9(1), 67–74 (2011)Google Scholar
  18. 18.
    G.J.M. Kruijff, M. Janíček, Using doctrines for human-robot collaboration to guide ethical behaviour. in Proceedings of the AAAI 2011 Fall Symposium Robot-Human Team-Work in Dynamic Adverse Environments. AAAI, 2011Google Scholar
  19. 19.
    G.J.M. Kruijff, M. Janíček, P. Lison, Continual processing of situated dialogue in human-robot collaborative activities. in Proceedings of the 19th International Symposium on Robot and Human Interactive Communication (RO-MAN 2010). IEEE, 2010Google Scholar
  20. 20.
    G.J.M. Kruijff, M. Janíček, and H. Zender, Situated communication for joint activity in human-robot teams. IEEE Intell. Syst. 272, 27–35 (2012) forthcomingGoogle Scholar
  21. 21.
    B. Larochelle, G.J.M. Kruijff, N. Smets, T. Mioch, P. Groenewegen, Establishing human situation awareness using a multi-modal operator control unit in an urban search and rescue human-robot team. in Proceedings of the 20th IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 2011Google Scholar
  22. 22.
    M. Liu, F. Colas, R. Siegwart. Regional topological segmentation based on mutual information graphs. in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2011Google Scholar
  23. 23.
    M. Lohse. The role of expectations and situations in human-robot interaction. In K. Dautenhahn, J. Saunders, eds. New Frontiers in Human-Robot Interaction (John Benjamins, Amsterdam, 2011) pp. 35–56Google Scholar
  24. 24.
    C. Miller, R. Parasuraman, Designing for flexible interaction between humans and automation: delegation interfaces for supervisory control. Hum. Factors 49, 57–75 (2007)CrossRefGoogle Scholar
  25. 25.
    R.R. Murphy, Human-robot interaction in rescue robotics. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 34(2), 138–153 (2004)CrossRefGoogle Scholar
  26. 26.
    R.R. Murphy, J.L. Burke, The safe human-robot ratio. in M.J. Barnes, F. Jentsch, eds. Human-Robot Interactions in Future Military Operations, Human Factors in Defence (Ashgate, Florida, 2010), pp. 31–49Google Scholar
  27. 27.
    R.R. Murphy, S. Tadokoro, D. Nardi, A. Jacoff, P. Fiorini, H. Choset, A.M. Erkmen. Search and rescue robotics. in B. Siciliano, O. Khatib, eds. Springer Handbook of Robotics (Springer, Berlin, 2008), Part F, pp. 1151–1173Google Scholar
  28. 28.
    T. Oron-Gilad, P.A. Hancock, Multi-modal information display under stress. in P.A. Hancock, J.L. Szalma, eds. Performance Under Stress, Human Factors in Defence (Ashgate, Florida, 2008), pp. 251–270Google Scholar
  29. 29.
    P. Papadakis, I. Pratikakis, S. Perantonis, T. Theoharis, Efficient 3d shape matching and retrieval using a concrete radialized spherical projection representation. Pattern Recognit. 40(9), 2437–2452 (2007)CrossRefzbMATHGoogle Scholar
  30. 30.
    E.G. Papadopoulos, D.A. Rey. A new measure of tipover stability margin for mobile manipulators. in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3111–3116, 1996Google Scholar
  31. 31.
    R. Parasuraman, M. Barnes, K. Cosenzo, Adaptive automation for human-robot teaming in future command and control systems. Int. J. Command Control 1(2), 43–68 (2007)Google Scholar
  32. 32.
    F. Pirri, The well-designed logical robot: learning and experience from observations to the situation calculus. Artif. Intell. 175(1), 378–415 (2011)Google Scholar
  33. 33.
    F. Pomerleau, S. Magnenat, F. Colas, M. Liu, R. Siegwart, Tracking a depth camera: parameter exploration for fast icp. in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2011)Google Scholar
  34. 34.
    P.M. Salmon, N.A. Stanton, G.H. Walker, D.P. Jenkins, Distributed Situation Awareness: Theory, Measurement, and Application to Teamwork (Ashgate, Human Factors in Defence, 2009)Google Scholar
  35. 35.
    K. Stubbs, P.J. Hinds, D. Wettergreen, Autonomy and common ground in human-robot interaction: A field study. IEEE Intell. Syst. 22, 42–50 (2007) (Special Issue on Interacting with Autonomy)Google Scholar
  36. 36.
    J. van Diggelen, K. van Drimmelen, A. Heuvelink, P.J.M. Kerbusch, M.A. Neerincx, S.M.A van Trijp, E.M. Ubink, and B. van der Vecht. Mutual empowerment in mobile soldier support. Special Issue of the International Journal of Battlefield Technology on Human Factors and Battlefield Technologies. submitted.Google Scholar
  37. 37.
    D.D. Woods, J. Tittle, M. Feil, A. Roesler, Envisioning human-robot coordination for future operations. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 34(2), 210–218 (2004)Google Scholar
  38. 38.
    K. Zimmermann, D. Hurych, T. Svoboda, Improving cascade of classifiers by sliding window alignment in between. in The 5th International Conference on Automation, Robotics and Applications. IEEE, 2011Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • G. J. M. Kruijff
    • 1
    Email author
  • M. Janíček
    • 1
  • S. Keshavdas
    • 1
  • B. Larochelle
    • 1
  • H. Zender
    • 1
  • N. J. J. M. Smets
    • 2
  • T. Mioch
    • 2
  • M. A. Neerincx
    • 2
  • J. V. Diggelen
    • 2
  • F. Colas
    • 3
  • M. Liu
    • 3
  • F. Pomerleau
    • 3
  • R. Siegwart
    • 3
  • V. Hlaváč
    • 4
  • T. Svoboda
    • 4
  • T. Petříček
    • 4
  • M. Reinstein
    • 4
  • K. Zimmermann
    • 4
  • F. Pirri
    • 5
  • M. Gianni
    • 5
  • P. Papadakis
    • 5
  • A. Sinha
    • 5
  • P. Balmer
    • 6
  • N. Tomatis
    • 6
  • R. Worst
    • 7
  • T. Linder
    • 7
  • H. Surmann
    • 7
  • V. Tretyakov
    • 7
  • S. Corrao
    • 8
  • S. Pratzler-Wanczura
    • 9
  • M. Sulk
    • 9
  1. 1.Nuance Communications Deutschland GmbHR&D—NLP ResearchAachenGermany
  2. 2.TNOSoesterbergThe Netherlands
  3. 3.ETH ZürichZürichSwitzerland
  4. 4.CTU in PraguePragueCzech Republic
  5. 5.ROMARomeItaly
  6. 6.BLUELausanneSwitzerland
  7. 7.FraunhoferSt. AugustinGermany
  8. 8.VVFFRomeItaly
  9. 9.FDDODortmundGermany

Personalised recommendations