Remote HRI and Mixed Reality, an Ontology

  • Carolina Cani D.L.Email author
  • Felipe B. Breyer
  • Judith Kelner
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9748)


This work builds a panorama of resources in Remote HRI identified in a systematic literature review with focus on Mixed reality solutions. This study builds a panorama of HRI with mixed reality through the definition of the terms presented in this ontology which serves as a reference to facilitate to create new robotic solutions relying on this resource.


HRI Human robot interaction Remote control Teleoperation Augmented reality Virtual reality Mixed reality Augmented virtuality 


  1. 1.
    Goodrich, M.A., Schultz, A.C.: Human-robot interaction: a survey. Trends Hum. Comput. Interact. 1(3), 203–275 (2007)CrossRefzbMATHGoogle Scholar
  2. 2.
    Heemskerk, C., Eendebak, P., Schropp, G., Hermes, H., Elzendoorn, B., Magielsen, A.: Introducing artificial depth cues to improve task performance in ITER maintenance actions. Fus. Eng. Des. 88(9–10), 1969–1972 (2013). Proceedings of the 27th Symposium on Fusion Technology, Belgium, pp. 24–28 (2012)CrossRefGoogle Scholar
  3. 3.
    Lamb, P., Owen, D.: human performance in space telerobotic manipulation. In: ACM Symposium on Virtual Reality Software and Technology, VRST 2005, pp. 31–37. ACM, New York (2005)Google Scholar
  4. 4.
    Casper, J., Murphy, R.R.: Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center. IEEE Trans. Syst. Man Cybern. Part B Cybern. 33(3), 367–385 (2003)CrossRefGoogle Scholar
  5. 5.
    Liu, Y., Nejat, G.: Robotic urban search and rescue: a survey from the control perspective. J. Intell. Robot. Syst. 72(2), 147–165 (2013)CrossRefGoogle Scholar
  6. 6.
    Green, S., Billinghurst, M., Chen, X., Chase, G.: Human-robot collaboration: a literature review and augmented reality approach in design. J. ARS, 1–18 (2007)Google Scholar
  7. 7.
    Sheridan, T.B., Verplanck, W.L.: Human and computer control of undersea teleoperators. Technical report, MIT Man-Machine Laboratory, Cambridge, MA (1978)Google Scholar
  8. 8.
    Sheridan, T.B.: Adaptive automation, level of automation, allocation authority, supervisory control, and adaptive control: distinctions and modes of adaptation. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 41(4), 662–667 (2011)CrossRefGoogle Scholar
  9. 9.
    Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. 77(12), 1321–1329 (1994)Google Scholar
  10. 10.
    Noy, N.F., McGuinness, D.L., et al.: Ontology development 101: a guide to creating your first ontology. Volume 15, Stanford knowledge systems laboratory technical report KSL-01-05 and Stanford medical informatics technical report SMI-2001-0880, Stanford, CA (2001)Google Scholar
  11. 11.
    Chen, J.Y., Barnes, M.J.: Robotics operator performance in a military multi-tasking environment. In: 3rd ACM/IEEE International Conference on Human Robot Interaction, HRI 2008, pp. 279–286. ACM, New York (2008)Google Scholar
  12. 12.
    Nielsen, C.W., Goodrich, M.A., Ricks, R.W.: Ecological interfaces for improving mobile robot teleoperation. IEEE Trans. Robot. 23(5), 927–941 (2007)CrossRefGoogle Scholar
  13. 13.
    Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., Goodrich, M.: Common metrics for human-robot interaction. In: 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, HRI 2006, pp. 33–40. ACM, New York (2006)Google Scholar
  14. 14.
    Sanguino, J.T.M., Márquez, M.J.A., Carlson, T., Millán, J.: Improving skills and perception in robot navigation by an augmented virtuality assistance system. J. Intell. Robot. Syst. 76(2), 255–266 (2014)CrossRefGoogle Scholar
  15. 15.
    Parasuraman, R., Sheridan, T.B., Wickens, C.D.: A model for types and levels of human interaction with automation. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 30(3), 286–297 (2000)CrossRefGoogle Scholar
  16. 16.
    Michaud, F., Boissy, P., Labonté, D., Brière, S., Perreault, K., Corriveau, H., Grant, A., Lauria, M., Cloutier, R., Roux, M.-A., Iannuzzi, D., Royer, M.-P., Ferland, F., Pomerleau, F., Létourneau, D.: Exploratory design and evaluation of a homecare teleassistive mobile robotic system. Mechatronics 20(7), 751–766 (2010). Special Issue on Design and Control Methodologies in Telerobotics (2010)CrossRefGoogle Scholar
  17. 17.
    Azuma, R.T.: A survey of augmented reality. Presence Teleoper. Virtual Environ. 6(4), 355–385 (1997)CrossRefGoogle Scholar
  18. 18.
    Green, S.A., Chase, J.G., Chen, V., Billinghurst, M.: Evaluating the augmented reality human-robot collaboration system. In: 2008 15th International Conference on Mechatronics and Machine Vision in Practice, M2VIP 2008, pp. 521–526 (2008)Google Scholar
  19. 19.
    Livatino, S., Muscato, G., Sessa, S., Neri, V.: Depth-enhanced mobile robot teleguide based on laser images. Mechatronics 20(7), 739–750 (2010). Special Issue on Design and Control Methodologies in Telerobotics (2010)CrossRefGoogle Scholar
  20. 20.
    de Barros, P.G., Lindeman, R.W.: Performance effects of multi-sensory displays in virtual teleoperation environments. In: 1st Symposium on Spatial User Interaction, SUI 2013, pp. 41–48. ACM, New York (2013)Google Scholar
  21. 21.
    Lee, S., Kim, G.J.: Effects of haptic feedback, stereoscopy, and image resolution on performance and presence in remote navigation. Int. J. Hum Comput Stud. 66(10), 701–717 (2008)CrossRefGoogle Scholar
  22. 22.
    Haas, E., Stachowiak, C.: Multimodal displays to enhance human robot interaction on-the-move. In: 2007 Workshop on Performance Metrics for Intelligent Systems, PerMIS 2007, pp. 135–140. ACM, New York (2007)Google Scholar
  23. 23.
    Sauer, M., Leutert, F., Schilling, K.: An augmented reality supported control system for remote operation and monitoring of an industrial work cell. In: 2nd IFAC Symposium on Telematics Applications, pp. 83–88 (2010)Google Scholar
  24. 24.
    Ryu, H., Lee, W.: Where you point is where the robot is. In: 7th ACM SIGCHI New Zealand Chapter’s International Conference on Computer-Human Interaction: Design Centered HCI, CHINZ 2006, pp. 33–42. ACM, New York (2006)Google Scholar
  25. 25.
    Hart, S.G.: NASA-task load index (NASA-TLX); 20 years later. Hum. Factors Ergonomics Soc. Annu. Meet. 50(9), 904–908 (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Carolina Cani D.L.
    • 1
    Email author
  • Felipe B. Breyer
    • 1
  • Judith Kelner
    • 1
  1. 1.Virtual Reality and Multimedia Research GroupUniversidade Federal de PernambucoRecifeBrazil

Personalised recommendations