Improving Skills and Perception in Robot Navigation by an Augmented Virtuality Assistance System


Successful navigation in a teleoperation scenario requires a good level of situational or environmental awareness. This paper presents the main features and capabilities of a new augmented virtuality-based system aimed at providing users with improved perception of the robot’s remote environment. With this purpose, a mixed-perspective exocentric display (ME3D), and a video centric display (VC2D) are compared. Both interfaces were implemented on a mobile robot and experiments were performed in a real working scenario. To assess this contribution, this works analyzes the teleoperation capability, performance, and human workload of users by means of NASA-TLX (Task Load Index). The results show that participants experienced a reduction in the driving workload and showed high degrees of acceptance for the proposed ME3D interface.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.


  1. 1.

    Gibson, J.J.: Perceiving, Acting, and Knowing: Toward an Ecological Psychology, pp. 67–82. Lawrence Erlbaum, Hillsdale (1977)

  2. 2.

    Mateo Sanguino, T.J., Andújar Márquez, J.M., Carlson, T., Millán, J.d.R.: Interaction and evaluation of an augmented virtuality assistance system for teleoperated robots. In: IEEE Intern. Symposium on Robotic and Sensors Environments (ROSE), Magdeburg (Germany), pp. 19–24 (2012)

  3. 3.

    Hart, S., Staveland, L.: Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock, P., Meshkati, N. (eds.) Human Mental Workload. North-Holland, pp. 139–83 (1988)

  4. 4.

    Carlson, T., Demiris, Y.: Collaborative control for a robotic wheelchair: evaluation of performance, attention and workload. IEEE Trans. Syst Man Cybern. 42(3), 876–888 (2012)

  5. 5.

    Tonin, L., Leeb, R., Tavella, M., Perdikis, S., Millán, J.d.R.: The role of shared-control in BCI-based telepresence. IEEE Conf. Syst. Man Cybern, 1462–1466 (2010)

  6. 6.

    Boessenkool, H., Abbink, D.A., Heemskerk, C.J.M., van der Helm, F.C.T.: Haptic shared control improves tele-operated task performance towards performance in direct control. In: IEEE World Haptics Conference, pp. 433–438 (2011)

  7. 7.

    Chen, I.Y., MacDonald, B., Wunsche, B.: Mixed reality simulation for mobile robots. In: IEEE International Conference Robotics and Automation (ICRA), pp. 232–237 (2009)

  8. 8.

    Nielsen, C.W.: Using Augmented Virtuality to Improve Human-Robot Interactions. Ph.D., Brigham Young University (2006)

  9. 9.

    Milgram, P., Rastogi, A., Grodski, J.: Telerobotic control using augmented reality. In: 4th IEEE International Workshop on Robot and Human Communication, pp. 21–29 (1995)

  10. 10.

    Salzmann, Ch., Gillet, D., Huguenin, P.: Introduction to real-time control using LabVIEW™ with an application to distance learning. Int. J. Eng. Educ. 16(3), 255–272 (2000)

  11. 11.

    Andújar, J.M., Mejias, A., Márquez, M.A.: Augmented reality for the improvement of remote laboratories: an augmented remote laboratory. IEEE Trans. Educ. 54(3), 492–500 (2011)

  12. 12.

    Perrin, X., Chavarriaga, R., Colas, F., et al.: Brain-coupled interaction for semi-autonomous navigation of an assistive robot. Robot. Auton. Syst. 58, 1246–1255 (2010)

  13. 13.

    Carff, J., Johnson, M., El-Sheikh, E.M., Pratt, J.E.: Human-robot team navigation in visually complex environments. In: IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pp. 3043–3050 (2009)

  14. 14.

    Amstutz, P., Fagg, A.: Real time visualization of robot state with mobile virtual reality. IEEE Int. Conf. Robotics Autom. (ICRA) 1, 241–247 (2002)

  15. 15.

    Stoker, C., Zbinden, E., Blackmon, T., et al.: Analyzing pathfinder data using virtual reality and superresolved imaging. J. Geophys. Res. 104(E4), 8889–8906 (1999)

  16. 16.

    Livatino, S., Muscato, G., Sessa, S., et al.: Depth-enhanced mobile robot teleguide based on laser images. Mechatronics 20, 739–750 (2010)

  17. 17.

    Regenbrecht, H., Ott, C., Wagner, M., Lum, T., Kohler, P., Wilke, W., Mueller, E.: An augmented virtuality approach to 3D videoconferencing. In: 2nd IEEE and ACM International Sympposium Mixed and Augmented Reality, pp. 290–291 (2003)

  18. 18.

    Varadarajan, K.M., Vincze, M.: Augmented virtuality based immersive telepresence for control of mining robots. In: 5th International Symposium Computational Intelligence and Intelligent Informatics (ISCIII), pp. 133–138 (2011)

  19. 19.

    Pacis, E.B., Everett, H.R., Farrington, N., Kogut’a, G., Sights, B., Kramera, T.: Transitioning unmanned ground vehicle research technologies. Unmanned Ground Veh. Tech. VII, 29–31 (2005)

  20. 20.

    Nielsen, C.W., Goodrich, M.A., Ricks, R.W.: Ecological interfaces for improving mobile robot teleoperation. IEEE Trans. Robotics 23(5), 927–941 (2007)

  21. 21.

    Michaud, F., Boissy, P., Labonté, D., et al.: Exploratory design and evaluation of a homecare teleassistive mobile robotic system. Mechatronics 20, 751–766 (2010)

  22. 22.

    Vicente, K.J., Rasmussen, J.: Ecological interface design: theoretical foundations. IEEE Trans. Syst. Man Cybern. 22, 589–606 (1992)

  23. 23.

    Chen, J.Y.C., Oden, R.V.N., Drexler, J.M.: Evaluation of stereoscopic displays for indirect-vision driving and robot teleoperation. In: 27th Army Science Conference (2010)

  24. 24.

    Gomer, J.A., Pagano, C.C.: NASA task load index for human-robot interaction workload measurement. ITEA J. 32, 210–214 (2011)

  25. 25.

    Chen, J.Y.C., Haas, E.C., Barnes, M.J.: Human performance issues and user interface design for teleoperated robots. IEEE Trans. Syst. Man Cybern. 37(6), 1231–1245 (2007)

  26. 26.

    Schöner, G., Dose, M., Engels, C.: Dynamics of behavior: theory and applications for autonomous robot architectures. Robot. Auton. Syst. 16, 213–245 (1995)

  27. 27.

    Chellali, R., Baizid, K.: What maps and what displays for remote situation awareness and ROV localization?. In: Lecture Notes in Computer Science, 6772 LNCS (PART 2), pp. 364–372 (2011)

  28. 28.

    Creem-Regehr, S.H., Willemsen, P., Gooch, A.A., Thompson, W.B.: The influence of restricted viewing conditions on egocentric distance perception: Implications for real and virtual environments. Perception 34(2), 191–204 (2005)

  29. 29.

    Yanco, H.A., Drury, J.L.: Where am I? Acquiring situation awareness using a remote robot platform. IEEE Conf. Syst. Man Cybern. 3, 2835–2840 (2004)

  30. 30.

    GStreamer: Open Source Multimedia Framework. Available:

  31. 31.

    Norman, D.: The design of everyday things. Doubleday Business (2002)

Download references

Author information

Correspondence to T. J. Mateo Sanguino.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 90.3 MB)

(MP4 90.3 MB)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Sanguino, T.M., Márquez, J.A., Carlson, T. et al. Improving Skills and Perception in Robot Navigation by an Augmented Virtuality Assistance System. J Intell Robot Syst 76, 255–266 (2014) doi:10.1007/s10846-014-0038-5

Download citation


  • 3D interface
  • Augmented virtuality
  • Mobile robotics
  • Telepresence
  • Assisted navigation