Skip to main content
SpringerLink
Log in

A Review on Teleoperation of Mobile Ground Robots: Architecture and Situation Awareness

International Journal of Control, Automation and Systems volume 19, pages 1384–1407 (2021)Cite this article

Abstract

Currently, the application of mobile ground robots spans a range of fields from surveillance, search and rescue, exploration, agriculture, military among others. In unstructured and dangerous environments such as disaster scene, military fields or chemical spray in agricultural farms, the experience and intelligence of the operator are necessary for making complex decisions beyond the autonomy of the robot. In such cases, teleoperation allow the operator to guide the robot in achieving complex task from a safe location. The effectiveness with which the operator controls the robot depends on, among others, operator’s awareness of the robot’s environment, the quality of communication link, the robustness of robot’s control system and experience of the human operator. Ground mobile robots form the basis of this work since they are applicable in many fields and mostly operate in dynamic environments that require additional guidance from a human operator. This study reviews research work on mobile robot teleoperation systems, and puts more emphasis on the architecture, communication link and situation awareness creation. Moreover, future trend in mobile robot teleoperation is also put forward in this review to give ground for new research work in this field. Based on the sited literature, it is noted that making the operator feel present in the robot’s environment through sufficient visual and force feedback as well as use of good quality network, significantly improve the navigation efficiency and task achievement of mobile ground robots.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. L. Basañez and R. Suárez, “Teleoperation,” Springer Handbook of Automation, Springer, pp. 449–468, 2009.

  2. J. Vertut and P. Coiffet, “Teleoperation and robotics evolution and development,” Robot Technology, vol. 3a, 1985.

  3. R. G. Boboc, H. Moga, and D. Talaba, “A review of current applications in teleoperation of mobile robots,” Bulletin of the Transilvania University of Brasov. Engineering Sciences, vol. 5, no. 2, p. 9, 2012.

    Google Scholar 

  4. N. Kubota and S. Ozawa, “Tele-operated robots for monitoring based on sensor networks,” Proc. of SICE Annual Conference, pp. 3355–3360, 2008.

  5. A. Eliav, T. Lavie, Y. Parmet, H. Stern, and Y. Edan, “Advanced methods for displays and remote control of robots,” Applied Ergonomics, vol. 42, no. 6, pp. 820–829, 2011.

    Google Scholar 

  6. T. Fong and C. Thorpe, “Vehicle teleoperation interfaces,” Autonomous Robots, vol. 11, no. 1, pp. 9–18, 2001.

    MATH  Google Scholar 

  7. T. B. Sheridan, Telerobotics, Automation, and Human Supervisory Control, MIT press, 1992.

  8. J. P. Vasconez, G. A. Kantor, and F. A. Auat Cheein, “Human-robot interaction in agriculture: A survey and current challenges,” Biosystems Engineering, vol. 179, pp 35–48, 2019.

    Google Scholar 

  9. P. M. Kebria, H. Abdi, M. M. Dalvand, A. Khosravi, and S. Nahavandi, “Control methods for internet-based teleoperation systems: A review,” IEEE Transactions on Human-Machine Systems, vol. 49, no. 1, pp. 32–46, 2019.

    Google Scholar 

  10. E. Slawiñski, D. Santiago, and V. Mut, “Control for delayed bilateral teleoperation of a quadrotor,” ISA Transactions, vol. 71, pp 415–425, 2017.

    MATH  Google Scholar 

  11. L. Chan, F. Naghdy, and D. Stirling, “Application of adaptive controllers in teleoperation systems: A survey,” vol. 44, no. 3, pp. 337–352, 2014.

    Google Scholar 

  12. G. Niemeyer and J.-J. Slotine, “Towards force-reflecting teleoperation over the internet,” Proc. of IEEE International Conference on Robotics and Automation (Cat. No. 98CH36146), vol. 3, pp 1909–1915, 1998.

    Google Scholar 

  13. P. F. Hokayem and M. W. Spong, “Bilateral teleoperation: An historical survey,” Automatica, vol. 42, no. 12, pp. 2035–2057, 2006.

    MathSciNet  MATH  Google Scholar 

  14. A. Erfani, S. Rezaei, M. Pourseifi, and H. Derili, “Optimal control in teleoperation systems with time delay: A singular perturbation approach,” Journal of Computational and Applied Mathematics, vol. 338, pp 168–184, 2018.

    MathSciNet  MATH  Google Scholar 

  15. P. Varnell and F. Zhang, “Dissipativity-based teleoperation with time-varying communication delays,” IFAC Proceedings Volumes, vol. 46, no. 27, pp. 369–376, 2013.

    Google Scholar 

  16. C. W. Nielsen, M. A. Goodrich, and R. W. Ricks, “Ecological interfaces for improving mobile robot teleoperation,” IEEE Transactions on Robotics, vol. 23, no. 5, pp. 927–941, 2007.

    Google Scholar 

  17. P. L. Alfano and G. F. Michel, “Restricting the field of view: Perceptual and performance effects,” Perceptual and Motor Skills, vol. 70, no. 1, pp. 35–45, 1990.

    Google Scholar 

  18. R. Meier, T. Fong, C. Thorpe, and C. Baur, “Sensor fusion based user interface for vehicle teleoperation,” Field and Service Robotics, no. LSRO2-CONF-1999-002, 1999.

  19. H. A. Yanco, J. L. Drury, and J. Scholtz, “Beyond usability evaluation: Analysis of human-robot interaction at a major robotics competition,” Human-Computer Interaction, vol. 19, no. 1–2, pp. 117–149, 2004.

    Google Scholar 

  20. J. H. Park and T. B. Sheridan, “Supervisory teleoperation control using computer graphics,” Proc. of 1991 IEEE International Conference on Robotics and Automation, pp. 493–494, 1991.

  21. T. Tang, F. Chucholowski, and M. Lienkamp, “Teleoperated driving basics and system design,” ATZ Worldwide, vol. 116, no. 2, pp. 16–19, 2014.

    Google Scholar 

  22. D. Drascic, “Skill acquisition and task performance in teleoperation using monoscopic and stereoscopic video remote viewing,” Proc. of the Human Factors Society Annual Meeting, vol. 35, no. 19, pp. 1367–1371, 1991.

    Google Scholar 

  23. D. Lee and M. W. Spong, “Passive bilateral teleoperation with constant time delay,” IEEE Transactions on Robotics, vol. 22, no. 2, pp. 269–281, 2006.

    Google Scholar 

  24. Z. Chen, Y.-J. Pan, and J. Gu, “A novel adaptive robust control architecture for bilateral teleoperation systems under time-varying delays,” International Journal of Robust and Nonlinear Control, vol. 25, no. 17, pp. 3349–3366, 2015.

    MathSciNet  MATH  Google Scholar 

  25. T. Imaida, Y. Yokokohji, T. Doi, M. Oda, and T. Yoshikawa, “Ground-space bilateral teleoperation of ETS-VII robot arm by direct bilateral coupling under 7s time delay condition,” IEEE Transactions on Robotics and Automation, vol. 20, no. 3, pp. 499–511, 2004.

    Google Scholar 

  26. D. Sun, F. Naghdy, and H. Du, “Application of wave-variable control to bilateral teleoperation systems: A survey,” Annual Reviews in Control, vol. 38, no. 1, pp. 12–31, 2014.

    Google Scholar 

  27. D. Lee, A. Franchi, H. Il Son, C. Ha, H. H. Bülthoff, and P. R. Giordano, “Semiautonomous haptic teleoperation control architecture of multiple unmanned aerial vehicles,” IEEE/ASME Transactions on Mechatronics, vol. 18, no. 4, pp. 1334–1345, 2013.

    Google Scholar 

  28. Y.-H. Seo, H.-Y. Jung, C.-S. Lee, and T.-K. Yang, “Remote data acquisition and touch-based control of a mobile robot using a smart phone,” Proc. of International Conference on Future Generation Communication and Networking, pp. 219–226, 2011.

  29. N. Guenard, T. Hamel, and L. Eck, “Control laws for the tele operation of an unmanned aerial vehicle known as an x4-flyer,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3249–3254, 2006.

  30. D. Aschenbrenner, M. Fritscher, F. Sittner, M. Krauß, and K. Schilling, “Teleoperation of an industrial robot in an active production line,” IFAC-PapersOnLine, vol. 48, no. 10, pp. 159–164, 2015.

    Google Scholar 

  31. S. Thrun, “Toward a framework for human-robot interaction,” Human-Computer Interaction, vol. 19, no. 1–2, pp. 9–24, 2004.

    Google Scholar 

  32. A. Bechar and C. Vigneault, “Agricultural robots for field operations. Part 2: Operations and systems,” Biosystems Engineering, vol. 153, pp 110–128, 2017.

    Google Scholar 

  33. W. Lee, S. Kang, M. Kim, and M. Park, “ROBHAZDT3: teleoperated mobile platform with passively adaptive double-track for hazardous environment applications,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), vol. 1, pp 33–38, 2004.

    Google Scholar 

  34. A. F. Winfield, “Future directions in tele-operated robotics,” Telerobotic Applications, pp. 147–163, 2000.

  35. E. P. Godoy, R. A. Tabile, R. R. D. Pereira, G. T. Tangerino, A. J. V Porto, and R. Y. Inamasu, “Design and implementation of an electronic architecture for an agricultural mobile robot,” Revista Brasileira de Engenharia Agricola e Ambiental, vol. 14, no. 11, pp. 1240–1247, 2010.

    Google Scholar 

  36. J. R. Moyne and D. M. Tilbury, “The emergence of industrial control networks for manufacturing control, diagnostics, and safety data,” Proceedings of the IEEE, vol. 95, no. 1, pp. 29–47, 2007.

    Google Scholar 

  37. J. Baillieul and P. J. Antsaklis, “Control and communication challenges in networked real-time systems,” Proceedings of the IEEE, vol. 95, no. 1, pp. 9–28, 2007.

    Google Scholar 

  38. K. Tindell, H. Hanssmon, and A. J. Wellings, “Analysing real-time communications: Controller area network (CAN),” RTSS, pp. 259–263, 1994.

  39. S. Han and J. Lee, “Tele-operation of a mobile robot using a force reflection joystick with a single hall sensor,” Proc. of RO-MAN, The 16th IEEE International Symposium on Robot and Human Interactive Communication, pp. 206–211, 2007.

  40. S. H. Kenyon, D. Creary, D. Thi, and J. Maynard, “A small, cheap, and portable reconnaissance robot,” Sensors, and Command, Control, Communications, and Intelligence (C3I) Technologies for Homeland Security and Homeland Defense IV, vol. 5778, pp 434–443, 2005.

    Google Scholar 

  41. H. C. Shin, Y. G. Kim, J. I. Cho, and Y.-J. Cho, “A teleoperated gesture recognition mobile robot using a stereo vision,” IFAC Proceedings Volumes, vol. 41, no. 2, pp. 12661–12666, 2008.

    Google Scholar 

  42. J. Bernshausen, C. Joochim, and H. Roth, “Mobile robot self localization and 3D map building using a 3D PMD-camera for tele-robotic applications,” IFAC Proceedings Volumes, vol. 44, no. 1, pp. 6851–6856, 2011.

    Google Scholar 

  43. J. J. Leonard and H. F. Durrant-Whyte, “Simultaneous map building and localization for an autonomous mobile robot,” Proc. of IROS, vol. 3, pp 1442–1447, 1991.

    Google Scholar 

  44. H. Liu, F. Sun, B. Fang, and X. Zhang, “Robotic room-level localization using multiple sets of sonar measurements,” IEEE Transactions on Instrumentation and Measurement, vol. 66, no. 1, pp. 2–13, 2016.

    Google Scholar 

  45. S. Karakaya, G. Küçükyildiz, and H. Ocak, “Detection of obstacle-free gaps for mobile robot applications using 2-D LIDAR data.,” International Journal of Natural & Engineering Sciences, vol. 10, no. 3, 2016.

  46. G. A. Kumar, A. K. Patil, R. Patil, S. S. Park, and Y. H. Chai, “A LiDAR and IMU integrated indoor navigation system for UAVs and its application in real-time pipeline classification,” Sensors, vol. 17, no. 6, p. 1268, 2017.

    Google Scholar 

  47. N. Shalal, T. Low, C. McCarthy, and N. Hancock, “Orchard mapping and mobile robot localisation using onboard camera and laser scanner data fusion-Part B: Mapping and localisation,” Computers and Electronics in Agriculture, vol. 119, pp 267–278, 2015.

    Google Scholar 

  48. N. Shalal, T. Low, C. McCarthy, and N. Hancock, “Orchard mapping and mobile robot localisation using onboard camera and laser scanner data fusion-part A: Tree detection,” Computers and Electronics in Agriculture, vol. 119, pp 254–266, 2015.

    Google Scholar 

  49. T. Y. Lim, C. F. Yeong, E. L. M. Su, S. F. Chik, P. J. H. Chin, and P. H. Tan, “Performance evaluation of various 2-D laser scanners for mobile robot map building and localization,” Journal of Telecommunication, Electronic and Computer Engineering (JTEC), vol. 8, no. 11, pp. 105–109, 2016.

    Google Scholar 

  50. S. Livatino, G. Muscato, S. Sessa, and V. Neri, “Depth-enhanced mobile robot teleguide based on laser images,” Mechatronics, vol. 20, no. 7, pp. 739–750, 2010.

    Google Scholar 

  51. J. Ware and Y.-J. Pan, “Realisation of a bilaterally teleoperated robotic vehicle platform with passivity control,” IET Control Theory & Applications, vol. 5, no. 8, pp. 952–962, 2011.

    Google Scholar 

  52. F. Kuipers, P. van Mieghem, T. Korkmaz, and M. Krunz, “An overview of constraint-based path selection algorithms for QoS routing,” IEEE Communications Magazine, vol. 40, no. 12, pp. 50–55, 2002.

    Google Scholar 

  53. B. Siciliano and O. Khatib, Springer Handbook of Robotics, Springer, 2016.

  54. A. F. T. Winfield and O. E. Holland, “The application of wireless local area network technology to the control of mobile robots,” Microprocessors and Microsystems, vol. 23, no. 10, pp. 597–607, 2000.

    Google Scholar 

  55. P. Fiorini and R. Oboe, “Internet-based telerobotics: problems and approaches,” Proc. of 8th International Conference on Advanced Robotics, ICAR’97, pp. 765–770, 1997.

  56. S. Munir and W. J. Book, “Internet based teleoperation using wave variables with prediction,” Proc. of IEEE/ASME International Conference on Advanced Intelligent Mechatronics (Cat. No. 01TH8556), vol. 1, pp 43–50, 2001.

    Google Scholar 

  57. K. R. Fall and W. R. Stevens, TCP/IP Illustrated, Volume 1: The Protocols, Addison-Wesley, 2011.

  58. J. Postel, “User datagram protocol,” 1980.

  59. H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, “RTP: A transport protocol for real-time applications.” RFC 1889, January, 1996.

  60. R. Oboe and P. Fiorini, “Issues on Internet-based teleoperation,” IFAC Proceedings Volumes, vol. 30, no. 20, pp. 591–597, 1997.

    Google Scholar 

  61. P. Papcun, I. Zolotova, and K. I. Tafsi, “Control and teleoperation of robot Khepera via Android mobile device through bluetooth and WiFi,” IFAC-PapersOnLine, vol. 49, no. 25, pp. 188–193, 2016.

    Google Scholar 

  62. F. Cui, M. Zhang, G. Cui, and X. Wu, “A robot teleoperation system based on virtual reality and WLAN,” Proc. of 6th World Congress on Intelligent Control and Automation, vol. 2, pp 9193–9197, 2006.

    Google Scholar 

  63. H. Hu, L. Yu, P. Wo Tsui, and Q. Zhou, “Internet-based robotic systems for teleoperation,” Assembly Automation, vol. 21, no. 2, pp. 143–152, 2001.

    Google Scholar 

  64. D. Wang, J. Yi, D. Zhao, and G. Yang, “Teleoperation system of the internet-based omnidirectional mobile robot with a mounted manipulator,” Proc. of International Conference on Mechatronics and Automation, pp. 1799–1804, 2007.

  65. T. Kot, P. Novák, and J. Babjak, “Application of augmented reality in mobile robot teleoperation,” Proc. of International Workshop on Modelling and Simulation for Autonomous Systems, pp. 223–236, 2017.

  66. P. M. Duong, T. T. Hoang, N. T. T. Van, D. A. Viet, and T. Q. Vinh, “A novel platform for internet-based mobile robot systems,” Proc. of 7th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 1972–1977, 2012.

  67. X. Shen, Z. J. Chong, S. Pendleton, G. M. J. Fu, B. Qin, E. Frazzoli, and M. H. Ang Jr., “Teleoperation of on-road vehicles via immersive telepresence using off-the-shelf components,” Intelligent Autonomous Systems, vol. 13, pp. 1419–1433, 2016.

    Google Scholar 

  68. B. Jincun, L. Qi, and L. Yanfei, “The design of the rescue robot long-distance control based on 3G and GPS,” Proc. of International Conference on Intelligent Human-Machine Systems and Cybernetics, vol. 2, pp 170–172, 2009.

    Google Scholar 

  69. M. S. Uddin, M. Gianni, and A. Lab, “Long range robot teleoperation system based on internet of things,” Proc. of 2nd International Conference on Computer and Communication Systems (ICCCS), pp. 163–167, 2017.

  70. K. M. Al-Aubidy, M. M. Ali, A. M. Derbas, and A. W. Al-Mutairi, “GPRS-based remote sensing and teleoperation of a mobile robot,” Proc. of International Multi-Conferences on Systems, Signals & Devices 2013 (SSD13), pp. 1–7, 2013.

  71. R. Montúfar-Chaveznava, C. Paisano, J. M. Cañas, and E. de Jódar, “Simulating telerobotics by cellular telephony,” ICINCO, pp. 329–334, 2005.

  72. J. Ryu, B. Yoo, and T. Nishimura, “Service robot operated by CDMA networks for security guard at home,” Service Robot Applications, IntechOpen, 2008.

  73. H. H. O. Nasereddin and A. A. Abdelkarim, “Smartphone control robots through Bluetooth,” International Journal of Research Sciences, Reviews in Applied, vol. 4, no. 4, pp. 399–404, 2010.

    Google Scholar 

  74. X. Xue, S. X. Yang, and M.-H. Meng, “Remote sensing and teleoperation of a mobile robot via the internet,” Proc. of IEEE International Conference on Information Acquisition, p. 6, 2005.

  75. T. Fong, C. Thorpe, and C. Baur, “Advanced interfaces for vehicle teleoperation: collaborative control, sensor fusion displays, and web-based tools,” Proc. of Vehicle Teleoperation Interfaces Workshop, IEEE International Conference on Robotics and Automation, no. CONF, 2000.

  76. J. Y. C. Chen, E. C. Haas, and M. J. Barnes, “Human performance issues and user interface design for teleoperated robots,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 37, no. 6, pp. 1231–1245, 2007.

    Google Scholar 

  77. N. Maru, H. Kase, S. Yamada, A. Nishikawa, and F. Miyazaki, “Manipulator control by using servoing with the stereo vision,” Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’93), vol. 3, pp 1866–1870, 1993.

    Google Scholar 

  78. T. Fong, C. Thorpe, and C. Baur, “Advanced interfaces for vehicle teleoperation: collaborative control, sensor fusion displays, and remote driving tools,” Autonomous Robots, vol. 11, no. 1, pp. 77–85, 2001.

    MATH  Google Scholar 

  79. A. Amanatiadis, A. Gasteratos, C. Georgoulas, L. Kotoulas, and I. Andreadis, “Development of a stereo vision system for remotely operated robots: A control and video streaming architecture,” Proc. of IEEE Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, pp. 14–19, 2008.

  80. T. Kot and P. Novák, “Utilization of the oculus rift HMD in mobile robot teleoperation,” Applied Mechanics and Materials, vol. 555, pp. 199–208, 2014.

    Google Scholar 

  81. P. Willemsen, M. B. Colton, S. H. Creem-Regehr, and W. B. Thompson, “The effects of head-mounted display mechanics on distance judgments in virtual environments,” Proc. of the 1st Symposium on Applied Perception in Graphics and Visualization, pp. 35–38, 2004.

  82. U. Martinez-Hernandez, M. Szollosy, L. W. Boorman, H. Kerdegari, and T. J. Prescott, “Towards a wearable interface for immersive telepresence in robotics,” Interactivity, Game Creation, Design, Learning, and Innovation, Springer, pp. 65–73, 2016.

  83. R. S. Allison, L. R. Harris, M. Jenkin, U. Jasiobedzka, and J. E. Zacher, “Tolerance of temporal delay in virtual environments,” Proc. of IEEE Virtual Reality, pp. 247–254, 2001.

  84. F. A. Biocca and J. P. Rolland, “Virtual eyes can rearrange your body: Adaptation to visual displacement in see-through, head-mounted displays,” Presence, vol. 7, no. 3, pp. 262–277, 1998.

    Google Scholar 

  85. K. Mania, B. D. Adelstein, S. R. Ellis, and M. I. Hill, “Perceptual sensitivity to head tracking latency in virtual environments with varying degrees of scene complexity,” Proc. of the 1st Symposium on Applied Perception in Graphics and Visualization, pp. 39–47, 2004.

  86. G. Doisy, A. Ronen, and Y. Edan, “Comparison of three different techniques for camera and motion control of a teleoperated robot,” Applied Ergonomics, vol. 58, pp 527–534, 2017.

    Google Scholar 

  87. V. Villani, F. Pini, F. Leali, and C. Secchi, “Survey on human-robot collaboration in industrial settings: Safety, intuitive interfaces and applications,” Mechatronics, vol. 55, pp 248–266, 2018.

    Google Scholar 

  88. H. Liu, T. Fang, T. Zhou, Y. Wang, and L. Wang, “Deep learning–based multimodal control interface for humanrobot collaboration,” Procedia CIRP, vol. 72, pp 3–8, 2018.

    Google Scholar 

  89. G. Du and P. Zhang, “Markerless human-robot interface for dual robot manipulators using Kinect sensor,” Robotics and Computer-Integrated Manufacturing, vol. 30, no. 2, pp. 150–159, 2014.

    Google Scholar 

  90. G. Du and P. Zhang, “A markerless human-robot interface using particle filter and Kalman filter for dual robots,” IEEE Transactions on Industrial Electronics, vol. 62, no. 4, pp. 2257–2264, 2014.

    Google Scholar 

  91. W. Ji, Y. Wang, H. Liu, and L. Wang, “Interface architecture design for minimum programming in humanrobot collaboration,” Procedia CIRP, vol. 72, pp 129–134, 2018.

    Google Scholar 

  92. R. Palmarini, I. F. del Amo, G. Bertolino, G. Dini, J. A. Erkoyuncu, R. Roy, and M. Farnsworth, “Designing an AR interface to improve trust in Human-Robots collaboration,” Procedia CIRP, vol. 70, pp 350–355, 2018.

    Google Scholar 

  93. K. Krückel, F. Nolden, A. Ferrein, and I. Scholl, “Intuitive visual teleoperation for UGVs using free-look augmented reality displays,” Proc. of IEEE International Conference on Robotics and Automation (ICRA), pp. 4412–4417, 2015.

  94. H. Liu and L. Wang, “Gesture recognition for humanrobot collaboration: A review,” International Journal of Industrial Ergonomics, vol. 68, pp 355–367, 2018.

    Google Scholar 

  95. A. Spada, M. Cognetti, and A. de Luca, “Locomotion and telepresence in virtual and real worlds,” Human Friendly Robotics, Springer, pp. 85–98, 2019.

  96. I. Rae, G. Venolia, J. C. Tang, and D. Molnar, “A framework for understanding and designing telepresence,” Proc. of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, pp. 1552–1566, 2015.

  97. A. W. W. Yew, S. K. Ong, and A. Y. C. Nee, “Immersive augmented reality environment for the teleoperation of maintenance robots,” Procedia Cirp, vol. 61, pp 305–310, 2017.

    Google Scholar 

  98. H. Kim, T. di Giacomo, A. Egges, L. Lyard, S. Garchery, and N. Magnenat-Thalmann, “Believable virtual environment: Sensory and perceptual believability,” Utrecht University Repository, 2004.

  99. A. Luciani, “Dynamics as common criterion to enhance the sense of presence in virtual environments,” Proc. of 7th Annual International Workshop on Presence, Valencia, Spain, pp. 96–103, 2004.

  100. L. R. Elliott, C. Jansen, E. S. Redden, and R. A. Pettitt, “Robotic telepresence: Perception, performance, and user experience,” Army Research Lab Aberdeen Proving Ground MD Human Research and Engineering, 2012.

  101. J. Gradecki, The Virtual Reality Programmer’s Kit, John Wiley & Sons, Inc., 1994.

  102. N. Rodriguez, J.-P. Jessel, and P. Torguet, “A virtual reality tool for teleoperation research,” Virtual Reality, vol. 6, no. 2, pp. 57–62, 2002.

    Google Scholar 

  103. J. Zainan, L. Hong, W. Jie, and H. Jianbin, “Virtual reality-based teleoperation with robustness against modeling errors,” Chinese Journal of Aeronautics, vol. 22, no. 3, pp. 325–333, 2009.

    Google Scholar 

  104. P. K. Pook and D. H. Ballard, “Remote teleassistance,” Proc. of IEEE International Conference on Robotics and Automation, vol. 1, pp 944–949, 1995.

    Google Scholar 

  105. G. Terrien, T. Fong, C. Thorpe, and C. Baur, “Remote driving with a multisensor user interface,” SAE Technical Paper, 2000.

  106. M. Mostefa, L. K. El Boudadi, A. Loukil, K. Mohamed, and D. Amine, “Design of mobile robot teleoperation system based on virtual reality,” Proc. of 3rd International Conference on Control, Engineering & Information Technology (CEIT), pp. 1–6, 2015.

  107. D. Cheng-jun, D. Ping, Z. Ming-lu, and Z. Yan-fang, “Design of mobile robot teleoperation system based on virtual reality,” Proc. of IEEE International Conference on Automation and Logistics, pp. 2024–2029, 2009.

  108. B. Ibari, Z. Ahmed-Foitih, and H. E. A. Reda, “Remote control of mobile robot using the virtual reality,” International Journal of Electrical Engineering, Computer, vol. 5, no. 5, 2015.

  109. P. Františ and J. Hodický, “Human machine interface in command and control system,” Proc. of IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, pp. 38–41, 2010.

  110. S. Kim, Y. Kim, J. Ha, and S. Jo, “Mapping system with virtual reality for mobile robot teleoperation,” Proc. of 18th International Conference on Control, Automation and Systems (ICCAS), p. 1541, 2018.

  111. A. Jacoff, E. Messina, B. A. Weiss, S. Tadokoro, and Y. Nakagawa, “Test arenas and performance metrics for urban search and rescue robots,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003)(Cat. No. 03CH37453), vol. 4, pp 3396–3403, 2003.

    Google Scholar 

  112. J. Liu, Y. Wang, B. Li, and S. Ma, “Current research, key performances and future development of search and rescue robots,” Frontiers of Mechanical Engineering in China, vol. 2, no. 4, pp. 404–416, 2007.

    Google Scholar 

  113. H. Martins and R. Ventura, “Immersive 3-d teleoperation of a search and rescue robot using a head-mounted display,” Proc. of IEEE Conference on Emerging Technologies & Factory Automation, pp. 1–8, 2009.

  114. L. Franzen, H. Kessock, D. Hinkle, S. Ravulakollu, and M. Bhakta, WorldToolKit Reference Manual, Engineering Animation inc, 1999.

  115. K. Schilling and M. P. Vernet, “Field vehicle teleoperations support by virtual reality interfaces,” Proc. of of 15th IFAC World Congress, 2002.

  116. R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre, “Recent advances in augmented reality,” Naval Research Lab Washington DC, 2001.

  117. S. Feiner, B. MacIntyre, T. Höllerer, and A. Webster, “A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment,” Personal Technologies, vol. 1, no. 4, pp. 208–217, 1997.

    Google Scholar 

  118. A. Webster, S. Feiner, B. MacIntyre, W. Massie, and T. Krueger, “Augmented reality in architectural construction, inspection and renovation,” Proc. of ASCE Third Congress on Computing in Civil Engineering, vol. 1, p 996, 1996.

    Google Scholar 

  119. Z. Szalavári, E. Eckstein, and M. Gervautz, “Collaborative gaming in augmented reality,” Proc. of the ACM Symposium on Virtual Reality Software and Technology, pp. 195–204, 1998.

  120. V. Le Ligeour, S. Otmane, and M. Mallem, “Augmented reality interface for free teleoperation,” IFAC Proceedings Volumes, vol. 38, no. 1, pp. 523–528, 2005.

    Google Scholar 

  121. S. Livatino, G. Muscato, F. Bannò, D. de Tommaso, and M. Macaluso, “Video and laser based augmented reality stereoscopic viewing for mobile robot teleoperation,” IFAC Proceedings Volumes, vol. 43, no. 23, pp. 161–168, 2010.

    Google Scholar 

  122. F. Cui, M. Zhang, and S. Guo, “Design of the intelligent control system of telerobot based on virtual reality,” Proc. of First International Conference on Innovative Computing, Information and Control-Volume I (ICICIC’06), vol. 1, pp 364–367, 2006.

    Google Scholar 

  123. M. Daily, Y. Cho, K. Martin, and D. Payton, “World embedded interfaces for human-robot interaction,” Proceedings of the 36th Annual Hawaii International Conference on System Sciences, p. 6, 2003.

  124. J. R. G. Pretlove, “Augmented reality as a tool to aid the telerobotic exploration and characterization of remote environments,” Presence, vol. 2, no. 4, pp. 52–3673, 2002.

    Google Scholar 

  125. P. Milgram and F. Kishino, “A taxonomy of mixed reality visual displays,” IEICE Transactions on Information Systems, vol. 77, no. 12, pp. 1321–1329, 1994.

    Google Scholar 

  126. A. Hosseini and M. Lienkamp, “Enhancing telepresence during the teleoperation of road vehicles using HMD-based mixed reality,” Proc. of IEEE Intelligent Vehicles Symposium (IV), pp. 1366–1373, 2016.

  127. F. Driewer, M. Sauer, and K. Schilling, “Mixed reality for teleoperation of mobile robots in search and rescue scenarios,” IFAC Proceedings Volumes, vol. 39, no. 3, pp. 267–272, 2006.

    Google Scholar 

  128. M. Sauer, M. Hess, and K. Schilling, “Towards a predictive mixed reality user interface for mobile robot teleoperation,” IFAC Proceedings Volumes, vol. 42, no. 22, pp. 91–96, 2009.

    Google Scholar 

  129. D. Lee, O. Martinez-Palafox, and M. W. Spong, “Bilateral teleoperation of a wheeled mobile robot over delayed communication network,” Proc. of IEEE International Conference on Robotics and Automation, ICRA 2006, pp. 3298–3303, 2006.

  130. D. A. Lawrence, “Stability and transparency in bilateral teleoperation,” IEEE Transactions on Robotics Automation, vol. 9, no. 5, pp. 624–637, 1993.

    Google Scholar 

  131. M. Kitagawa, A. M. Okamura, B. T. Bethea, V. L. Gott, and W. A. Baumgartner, “Analysis of suture manipulation forces for teleoperation with force feedback,” Proc. of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 155–162, 2002.

  132. K. Hashtrudi-Zaad and S. E. Salcudean, “Transparency in time-delayed systems and the effect of local force feedback for transparent teleoperation,” IEEE Transactions on Robotics and Automation, vol. 18, no. 1, pp. 108–114, 2002.

    Google Scholar 

  133. C. Pacchierotti, L. Meli, F. Chinello, M. Malvezzi, and D. Prattichizzo, “Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems,” The International Journal of Robotics Research, vol. 34, no. 14, pp. 1773–1787, 2015.

    Google Scholar 

  134. W. R. Ferrell, “Remote manipulation with transmission delay,” IEEE Transactions on Human Factors in Electronics, no. 1, pp. 24–32, 1965.

  135. R. Luck and A. Ray, “Experimental verification of a delay compensation algorithm for integrated communication and control systems,” International Journal of Control, vol. 59, no. 6, pp. 1357–1372, 1994.

    MATH  Google Scholar 

  136. G. Adamides, C. Katsanos, Y. Parmet, G. Christou, M. Xenos, T. Hadzilacos, and Y. Edan, “HRI usability evaluation of interaction modes for a teleoperated agricultural robotic sprayer,” Applied ergonomics, vol. 62, pp 237–246, 2017.

    Google Scholar 

  137. S. Livatino, G. Muscato, F. Bannò, D. de Tommaso, and M. Macaluso, “Video and laser based augmented reality stereoscopic viewing for mobile robot teleoperation,” IFAC Proceedings Volumes, vol. 43, no. 23, pp. 161–168, 2010.

    Google Scholar 

  138. A. Wichmann, B. D. Okkalioglu, and T. Korkmaz, “The integration of mobile (tele) robotics and wireless sensor networks: A survey,” Computer Communications, vol. 51, pp 21–35, 2014.

    Google Scholar 

  139. J. Nieto, E. Slawinski, V. Mut, and B. Wagner, “Mobile robot teleoperation augmented with prediction and pathplanning,” IFAC Proceedings Volumes, vol. 43, no. 13, pp. 53–58, 2010.

    Google Scholar 

  140. S. Lee, G. Sukhatme, G. J. Kim, and C.-M. Park, “Haptic teleoperation of a mobile robot: A user study,” Presence: Teleoperators Environments, Virtual, vol. 14, no. 3, pp. 345–365, 2005.

    Google Scholar 

  141. J. Lim, J. Ko, and J. Lee, “Internet-based teleoperation of a mobile robot with force-reflection,” Proc. of IEEE Conference on Control Applications, CCA 2003, vol. 1, pp 680–685, 2003.

    Google Scholar 

  142. A. Shahzad and H. Roth, “Teleoperation of mobile robot using event based controller and real time force feedback,” Proc. of Scientific Cooperations International Workshops on Electrical and Computer Engineering Subfields, Istanbul, Turkey, pp. 7–12, 2014.

  143. T. W. Fong, F. Conti, S. Grange, and C. Baur, “Novel interfaces for remote driving: gesture, haptic, and PDA,” Mobile Robots XV and Telemanipulator and Telepresence Technologies VII, vol. 4195, pp 300–311 2001.

    Google Scholar 

  144. H. Roth, K. Schilling, and O. J. Rösch, “Haptic interfaces for remote control of mobile robots,” IFAC Proceedings Volumes, vol. 35, no. 1, pp. 177–182, 2002.

    Google Scholar 

  145. N. Diolaiti and C. Melchiorri, “Teleoperation of a mobile robot through haptic feedback,” Proc. of IEEE International Workshop on Haptic Virtual Environments and Their Applications (HAVE), pp. 67–72, 2002.

  146. S. Lee, G. S. Sukhatme, G. J. Kim, and C.-M. Park, “Haptic control of a mobile robot: A user study,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 3, pp 2867–2874, 2002.

    Google Scholar 

  147. O. Martinez-Palafox, D. Lee, M. W. Spong, I. Lopez, and C. T. Abdallah, “Bilateral teleoperation of mobile robot over delayed communication network: Implementation,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4193–4198, 2006.

  148. R. J. Anderson and M. W. Spong, “Bilateral control of teleoperators with time delay,” IEEE Transactions on Automatic Control, vol. 34, no. 5, pp. 494–501, 1989.

    MathSciNet  Google Scholar 

  149. M. W. Spong, S. Hutchinson, and M. Vidyasagar, Robot Modeling and Control, 1st ed., John Wiley & Sons, INC., 2006.

  150. Y.-C. Liu and N. Chopra, “Control of semi-autonomous teleoperation system with time delays,” Automatica, vol. 49, no. 6, pp. 1553–1565, 2013.

    MathSciNet  MATH  Google Scholar 

  151. G. Angelopoulos, G. T. Kalampokis, and M. Dasygenis, “An internet of things humanoid robot teleoperated by an open source Android application,” Proc. of Panhellenic Conference on Electronics and Telecommunications (PACET), pp. 1–4, 2017.

  152. G. Paravati, B. Pralio, A. Sanna, and F. Lamberti, “A reconfigurable multi-touch remote control system for teleoperated robots,” Proc. of IEEE International Conference on Consumer Electronics (ICCE), pp. 153–154, 2011.

  153. J. Nádvorník and P. Smutný, “Remote control robot using Android mobile device,” Proc. of the 15th International Carpathian Control Conference (ICCC), pp. 373–378, 2014.

  154. P. P. Ray, L. Chettri, and N. Thapa, “IoRCar: IoT supported autonomic robotic movement and control,” Proc. of International Conference on Computation of Power, Energy, Information and Communication (ICCPEIC), pp. 77–83, 2018.

  155. C.-F. Chien, K. H. Kim, B. Liu, and M. Gen, “Advanced decision and intelligence technologies for manufacturing and logistics,” Journal of Intelligent Manufacturing, vol. 23, no. 6, p. 2133, 2012.

    Google Scholar 

  156. J. Zhao, J. Zhang, Y. Feng, and J. Guo, “The study and application of the IoT technology in agriculture,” Proc. of 3rd International Conference on Computer Science and Information Technology, vol. 2, pp 462–465, 2010.

    Google Scholar 

  157. X. Chen, J. Liu, X. Li, L. Sun, and Y. Zhen, “Integration of IoT with smart grid,” Proc. of IET International Conference on Communication Technology and Application (ICCTA 2011), pp. 723–726, 2011.

  158. H. Yujie and Z. Xihuang, “Research and application of PV monitoring system based on ZigBee and GPRS,” Proc. of 10th International Symposium on Distributed Computing and Applications to Business, Engineering and Science, pp. 338–342, 2011.

  159. P. P. Ray, “Internet of robotic things: Concept, technologies, and challenges,” IEEE Access, vol. 4, pp 9489–9500, 2016.

    Google Scholar 

  160. R. Toris, C. Shue, and S. Chernova, “Message authentication codes for secure remote non-native client connections to ROS enabled robots,” Proc. of IEEE International Conference on Technologies for Practical Robot Applications (TePRA), pp. 1–6, 2014.

  161. C. Crick, G. Jay, S. Osentoski, B. Pitzer, and O. C. Jenkins, “Rosbridge: ROS for Non-ROS users,” Robotics Research, Springer, pp. 493–504, 2017.

  162. C. Turcu, C. Turcu, and V. Gaitan, “Merging the internet of things and robotics,” Recent Researches in Circuits Systems, pp. 499–504, 2012.

  163. C. Yang, J. Luo, Y. Pan, Z. Liu, and C.-Y. Su Man, “Personalized variable gain control with tremor attenuation for robot teleoperation,” IEEE Transactions on Systems Systems, Cybernetics:, vol. 48, no. 10, pp. 1759–1770, 2017.

    Google Scholar 

  164. D. B. Kaber, E. Onal, and M. R. Endsley, “Design of automation for telerobots and the effect on performance, operator situation awareness, and subjective workload,” Human factors Manufacturing, Ergonomics in Industries, Service, vol. 10, no. 4, pp. 409–430, 2000.

    Google Scholar 

  165. J. Scholtz, B. Antonishek, and J. Young, “Evaluation of a human-robot interface: Development of a situational awareness methodology,” Proc. of 37th Annual Hawaii International Conference on System Sciences, p. 9, 2004.

  166. M. R. Endsley, “Measurement of situation awareness in dynamic systems,” Human Factors, vol. 37, no. 1, pp. 65–84, 1995.

    Google Scholar 

  167. J. Scholtz, B. Antonishek, and J. Young, “Evaluation of operator interventions in autonomous off-road driving,” National Inst. oF Standards and Technology Gaithersburg MD, 2003.

  168. A. Steinfeld, T. Fong, D. Kaber, M. Lewis, J. Scholtz, A. Schultz, and M. Goodrich, “Common metrics for humanrobot interaction,” Proc. of the 1st ACM SIGCHI/SIGART Conference on Human-robot Interaction, pp. 33–40, 2006.

  169. M. S. Sanders and E. J. McCormick, Human Factors in Engineering and Design, McGraw-Hill, 1998.

  170. J. A. Macedo, D. B. Kaber, M. R. Endsley, P. Powanusorn, and S. Myung, “The effect of automated compensation for incongruent axes on teleoperator performance,” Human Factors, vol. 40, no. 4, pp. 541–553, 1998.

    Google Scholar 

  171. W. M. Newman, M. G. Lamming, and M. Lamming, Interactive System Design, Addison-Wesley Reading, 1995.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Engineering, Nanjing Agricultural University, Nanjing, 210031, China

    Samwel Opiyo

  2. the School of Science, Catholic University of Eastern Africa, 62157-00200, Nairobi, Kenya

    Samwel Opiyo

  3. the College of Engineering, Nanjing Agricultural University, Nanjing, 210031, China

    Jun Zhou, Emmy Mwangi, Wang Kai & Idris Sunusi

Authors
  1. Samwel Opiyo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emmy Mwangi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wang Kai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Idris Sunusi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Zhou.

Additional information

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Recommended by Associate Editor Seok Chang Ryu under the direction of Editor Won-jong Kim. The authors of this work would like to acknowledge the funding support given by the Jiangsu Provincial Key Research & Development Plan (no. BE2017370).

Samwel Opiyo was born and raised in Kisumu, Kenya. He received a B.E. degree (physics, majoring in electronics) at Moi University-Kenya and graduated with second class upper division in 2008. He then enrolled for an M.S. degree (electronics and instrumentation) at Kenyatta University, Kenya, where he graduated in 2015. He has taught digital electronics, computer programing and robotics-related courses in Catholic University-Kenya, Mount Kenya University and Moi University-Kenya. He is currently a Ph.D. candidate majoring in electrification and automation at Nanjing Agricultural University specifically dealing with teleoperation of robots. His areas of interest include human-robot collaboration, digital image processing, teleoperation of robots, and IoT.

Jun Zhou is a full professor at Nanjing Agricultural University. He graduated, in 1998, with a B.E. degree from the Agricultral Mechanization Department, Nanjing Agricultural University, China. In 2003, he received a Ph.D. degree in agricultral engineering, Nanjing Agricultural University, China. In the year 2005, he completed his Postdoctoral Research Fellow, in mechatronics engineering, Shanghai Jiaotong University, China. He was a Visiting Scholar, in agicultural engineering, Iowa State University, USA, in 2013. His research interests are agricultural robot, digital image processing and pattern recognition, automation for agriculture, equipments for precision agriculture.

Emmy Mwangi graduated with the second upper B.Eng. degree (control and instrumentation) from Egerton University Kenya in 2017. She is currently an M.Sc. candidate majoring in agricultural electrification and automation at Nanjing Agriculural University. Her research interests include agricultural robots, digital image processing and control systems.

Wang Kai obtained his Master’s degree in mechatronics engineering from Changzhou University in 2012. He is curretly a Ph.D. candidate at Nanjing Agricultural University. His research directions are agricultural robots and robot navigation technology.

Idris Sunusi received his B.Eng. degree in agricultural engineering from Bayero University, Kano-Nigeria in 2009 and an M.Sc. degree in engineering (farm machinery) from Ahmadu Bello University, Zaria-Nigeria in 2016. He later joined the services of National agricultural extension and research liaison services, Ahmadu Bello University, Zaria in 2011 as an assistant lecturer/extension specialist. He has worked on the development of machinery for harvesting and processing of crop and is currently a Ph.D. research student in Nanjing Agricultural University, China. His research directions span dynamic control of electric vehicles, application of artificial intelligence in agricultural systems and automation of agricultural machinery.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Opiyo, S., Zhou, J., Mwangi, E. et al. A Review on Teleoperation of Mobile Ground Robots: Architecture and Situation Awareness. Int. J. Control Autom. Syst. 19, 1384–1407 (2021). https://doi.org/10.1007/s12555-019-0999-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12555-019-0999-z

Keywords

search

Navigation