Currently, the application of mobile ground robots spans a range of fields from surveillance, search and rescue, exploration, agriculture, military among others. In unstructured and dangerous environments such as disaster scene, military fields or chemical spray in agricultural farms, the experience and intelligence of the operator are necessary for making complex decisions beyond the autonomy of the robot. In such cases, teleoperation allow the operator to guide the robot in achieving complex task from a safe location. The effectiveness with which the operator controls the robot depends on, among others, operator’s awareness of the robot’s environment, the quality of communication link, the robustness of robot’s control system and experience of the human operator. Ground mobile robots form the basis of this work since they are applicable in many fields and mostly operate in dynamic environments that require additional guidance from a human operator. This study reviews research work on mobile robot teleoperation systems, and puts more emphasis on the architecture, communication link and situation awareness creation. Moreover, future trend in mobile robot teleoperation is also put forward in this review to give ground for new research work in this field. Based on the sited literature, it is noted that making the operator feel present in the robot’s environment through sufficient visual and force feedback as well as use of good quality network, significantly improve the navigation efficiency and task achievement of mobile ground robots.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
L. Basañez and R. Suárez, “Teleoperation,” Springer Handbook of Automation, Springer, pp. 449–468, 2009.
J. Vertut and P. Coiffet, “Teleoperation and robotics evolution and development,” Robot Technology, vol. 3a, 1985.
R. G. Boboc, H. Moga, and D. Talaba, “A review of current applications in teleoperation of mobile robots,” Bulletin of the Transilvania University of Brasov. Engineering Sciences, vol. 5, no. 2, p. 9, 2012.
N. Kubota and S. Ozawa, “Tele-operated robots for monitoring based on sensor networks,” Proc. of SICE Annual Conference, pp. 3355–3360, 2008.
A. Eliav, T. Lavie, Y. Parmet, H. Stern, and Y. Edan, “Advanced methods for displays and remote control of robots,” Applied Ergonomics, vol. 42, no. 6, pp. 820–829, 2011.
T. Fong and C. Thorpe, “Vehicle teleoperation interfaces,” Autonomous Robots, vol. 11, no. 1, pp. 9–18, 2001.
T. B. Sheridan, Telerobotics, Automation, and Human Supervisory Control, MIT press, 1992.
J. P. Vasconez, G. A. Kantor, and F. A. Auat Cheein, “Human-robot interaction in agriculture: A survey and current challenges,” Biosystems Engineering, vol. 179, pp 35–48, 2019.
P. M. Kebria, H. Abdi, M. M. Dalvand, A. Khosravi, and S. Nahavandi, “Control methods for internet-based teleoperation systems: A review,” IEEE Transactions on Human-Machine Systems, vol. 49, no. 1, pp. 32–46, 2019.
E. Slawiñski, D. Santiago, and V. Mut, “Control for delayed bilateral teleoperation of a quadrotor,” ISA Transactions, vol. 71, pp 415–425, 2017.
L. Chan, F. Naghdy, and D. Stirling, “Application of adaptive controllers in teleoperation systems: A survey,” vol. 44, no. 3, pp. 337–352, 2014.
G. Niemeyer and J.-J. Slotine, “Towards force-reflecting teleoperation over the internet,” Proc. of IEEE International Conference on Robotics and Automation (Cat. No. 98CH36146), vol. 3, pp 1909–1915, 1998.
P. F. Hokayem and M. W. Spong, “Bilateral teleoperation: An historical survey,” Automatica, vol. 42, no. 12, pp. 2035–2057, 2006.
A. Erfani, S. Rezaei, M. Pourseifi, and H. Derili, “Optimal control in teleoperation systems with time delay: A singular perturbation approach,” Journal of Computational and Applied Mathematics, vol. 338, pp 168–184, 2018.
P. Varnell and F. Zhang, “Dissipativity-based teleoperation with time-varying communication delays,” IFAC Proceedings Volumes, vol. 46, no. 27, pp. 369–376, 2013.
C. W. Nielsen, M. A. Goodrich, and R. W. Ricks, “Ecological interfaces for improving mobile robot teleoperation,” IEEE Transactions on Robotics, vol. 23, no. 5, pp. 927–941, 2007.
P. L. Alfano and G. F. Michel, “Restricting the field of view: Perceptual and performance effects,” Perceptual and Motor Skills, vol. 70, no. 1, pp. 35–45, 1990.
R. Meier, T. Fong, C. Thorpe, and C. Baur, “Sensor fusion based user interface for vehicle teleoperation,” Field and Service Robotics, no. LSRO2-CONF-1999-002, 1999.
H. A. Yanco, J. L. Drury, and J. Scholtz, “Beyond usability evaluation: Analysis of human-robot interaction at a major robotics competition,” Human-Computer Interaction, vol. 19, no. 1–2, pp. 117–149, 2004.
J. H. Park and T. B. Sheridan, “Supervisory teleoperation control using computer graphics,” Proc. of 1991 IEEE International Conference on Robotics and Automation, pp. 493–494, 1991.
T. Tang, F. Chucholowski, and M. Lienkamp, “Teleoperated driving basics and system design,” ATZ Worldwide, vol. 116, no. 2, pp. 16–19, 2014.
D. Drascic, “Skill acquisition and task performance in teleoperation using monoscopic and stereoscopic video remote viewing,” Proc. of the Human Factors Society Annual Meeting, vol. 35, no. 19, pp. 1367–1371, 1991.
D. Lee and M. W. Spong, “Passive bilateral teleoperation with constant time delay,” IEEE Transactions on Robotics, vol. 22, no. 2, pp. 269–281, 2006.
Z. Chen, Y.-J. Pan, and J. Gu, “A novel adaptive robust control architecture for bilateral teleoperation systems under time-varying delays,” International Journal of Robust and Nonlinear Control, vol. 25, no. 17, pp. 3349–3366, 2015.
T. Imaida, Y. Yokokohji, T. Doi, M. Oda, and T. Yoshikawa, “Ground-space bilateral teleoperation of ETS-VII robot arm by direct bilateral coupling under 7s time delay condition,” IEEE Transactions on Robotics and Automation, vol. 20, no. 3, pp. 499–511, 2004.
D. Sun, F. Naghdy, and H. Du, “Application of wave-variable control to bilateral teleoperation systems: A survey,” Annual Reviews in Control, vol. 38, no. 1, pp. 12–31, 2014.
D. Lee, A. Franchi, H. Il Son, C. Ha, H. H. Bülthoff, and P. R. Giordano, “Semiautonomous haptic teleoperation control architecture of multiple unmanned aerial vehicles,” IEEE/ASME Transactions on Mechatronics, vol. 18, no. 4, pp. 1334–1345, 2013.
Y.-H. Seo, H.-Y. Jung, C.-S. Lee, and T.-K. Yang, “Remote data acquisition and touch-based control of a mobile robot using a smart phone,” Proc. of International Conference on Future Generation Communication and Networking, pp. 219–226, 2011.
N. Guenard, T. Hamel, and L. Eck, “Control laws for the tele operation of an unmanned aerial vehicle known as an x4-flyer,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3249–3254, 2006.
D. Aschenbrenner, M. Fritscher, F. Sittner, M. Krauß, and K. Schilling, “Teleoperation of an industrial robot in an active production line,” IFAC-PapersOnLine, vol. 48, no. 10, pp. 159–164, 2015.
S. Thrun, “Toward a framework for human-robot interaction,” Human-Computer Interaction, vol. 19, no. 1–2, pp. 9–24, 2004.
A. Bechar and C. Vigneault, “Agricultural robots for field operations. Part 2: Operations and systems,” Biosystems Engineering, vol. 153, pp 110–128, 2017.
W. Lee, S. Kang, M. Kim, and M. Park, “ROBHAZDT3: teleoperated mobile platform with passively adaptive double-track for hazardous environment applications,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), vol. 1, pp 33–38, 2004.
A. F. Winfield, “Future directions in tele-operated robotics,” Telerobotic Applications, pp. 147–163, 2000.
E. P. Godoy, R. A. Tabile, R. R. D. Pereira, G. T. Tangerino, A. J. V Porto, and R. Y. Inamasu, “Design and implementation of an electronic architecture for an agricultural mobile robot,” Revista Brasileira de Engenharia Agricola e Ambiental, vol. 14, no. 11, pp. 1240–1247, 2010.
J. R. Moyne and D. M. Tilbury, “The emergence of industrial control networks for manufacturing control, diagnostics, and safety data,” Proceedings of the IEEE, vol. 95, no. 1, pp. 29–47, 2007.
J. Baillieul and P. J. Antsaklis, “Control and communication challenges in networked real-time systems,” Proceedings of the IEEE, vol. 95, no. 1, pp. 9–28, 2007.
K. Tindell, H. Hanssmon, and A. J. Wellings, “Analysing real-time communications: Controller area network (CAN),” RTSS, pp. 259–263, 1994.
S. Han and J. Lee, “Tele-operation of a mobile robot using a force reflection joystick with a single hall sensor,” Proc. of RO-MAN, The 16th IEEE International Symposium on Robot and Human Interactive Communication, pp. 206–211, 2007.
S. H. Kenyon, D. Creary, D. Thi, and J. Maynard, “A small, cheap, and portable reconnaissance robot,” Sensors, and Command, Control, Communications, and Intelligence (C3I) Technologies for Homeland Security and Homeland Defense IV, vol. 5778, pp 434–443, 2005.
H. C. Shin, Y. G. Kim, J. I. Cho, and Y.-J. Cho, “A teleoperated gesture recognition mobile robot using a stereo vision,” IFAC Proceedings Volumes, vol. 41, no. 2, pp. 12661–12666, 2008.
J. Bernshausen, C. Joochim, and H. Roth, “Mobile robot self localization and 3D map building using a 3D PMD-camera for tele-robotic applications,” IFAC Proceedings Volumes, vol. 44, no. 1, pp. 6851–6856, 2011.
J. J. Leonard and H. F. Durrant-Whyte, “Simultaneous map building and localization for an autonomous mobile robot,” Proc. of IROS, vol. 3, pp 1442–1447, 1991.
H. Liu, F. Sun, B. Fang, and X. Zhang, “Robotic room-level localization using multiple sets of sonar measurements,” IEEE Transactions on Instrumentation and Measurement, vol. 66, no. 1, pp. 2–13, 2016.
S. Karakaya, G. Küçükyildiz, and H. Ocak, “Detection of obstacle-free gaps for mobile robot applications using 2-D LIDAR data.,” International Journal of Natural & Engineering Sciences, vol. 10, no. 3, 2016.
G. A. Kumar, A. K. Patil, R. Patil, S. S. Park, and Y. H. Chai, “A LiDAR and IMU integrated indoor navigation system for UAVs and its application in real-time pipeline classification,” Sensors, vol. 17, no. 6, p. 1268, 2017.
N. Shalal, T. Low, C. McCarthy, and N. Hancock, “Orchard mapping and mobile robot localisation using onboard camera and laser scanner data fusion-Part B: Mapping and localisation,” Computers and Electronics in Agriculture, vol. 119, pp 267–278, 2015.
N. Shalal, T. Low, C. McCarthy, and N. Hancock, “Orchard mapping and mobile robot localisation using onboard camera and laser scanner data fusion-part A: Tree detection,” Computers and Electronics in Agriculture, vol. 119, pp 254–266, 2015.
T. Y. Lim, C. F. Yeong, E. L. M. Su, S. F. Chik, P. J. H. Chin, and P. H. Tan, “Performance evaluation of various 2-D laser scanners for mobile robot map building and localization,” Journal of Telecommunication, Electronic and Computer Engineering (JTEC), vol. 8, no. 11, pp. 105–109, 2016.
S. Livatino, G. Muscato, S. Sessa, and V. Neri, “Depth-enhanced mobile robot teleguide based on laser images,” Mechatronics, vol. 20, no. 7, pp. 739–750, 2010.
J. Ware and Y.-J. Pan, “Realisation of a bilaterally teleoperated robotic vehicle platform with passivity control,” IET Control Theory & Applications, vol. 5, no. 8, pp. 952–962, 2011.
F. Kuipers, P. van Mieghem, T. Korkmaz, and M. Krunz, “An overview of constraint-based path selection algorithms for QoS routing,” IEEE Communications Magazine, vol. 40, no. 12, pp. 50–55, 2002.
B. Siciliano and O. Khatib, Springer Handbook of Robotics, Springer, 2016.
A. F. T. Winfield and O. E. Holland, “The application of wireless local area network technology to the control of mobile robots,” Microprocessors and Microsystems, vol. 23, no. 10, pp. 597–607, 2000.
P. Fiorini and R. Oboe, “Internet-based telerobotics: problems and approaches,” Proc. of 8th International Conference on Advanced Robotics, ICAR’97, pp. 765–770, 1997.
S. Munir and W. J. Book, “Internet based teleoperation using wave variables with prediction,” Proc. of IEEE/ASME International Conference on Advanced Intelligent Mechatronics (Cat. No. 01TH8556), vol. 1, pp 43–50, 2001.
K. R. Fall and W. R. Stevens, TCP/IP Illustrated, Volume 1: The Protocols, Addison-Wesley, 2011.
J. Postel, “User datagram protocol,” 1980.
H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, “RTP: A transport protocol for real-time applications.” RFC 1889, January, 1996.
R. Oboe and P. Fiorini, “Issues on Internet-based teleoperation,” IFAC Proceedings Volumes, vol. 30, no. 20, pp. 591–597, 1997.
P. Papcun, I. Zolotova, and K. I. Tafsi, “Control and teleoperation of robot Khepera via Android mobile device through bluetooth and WiFi,” IFAC-PapersOnLine, vol. 49, no. 25, pp. 188–193, 2016.
F. Cui, M. Zhang, G. Cui, and X. Wu, “A robot teleoperation system based on virtual reality and WLAN,” Proc. of 6th World Congress on Intelligent Control and Automation, vol. 2, pp 9193–9197, 2006.
H. Hu, L. Yu, P. Wo Tsui, and Q. Zhou, “Internet-based robotic systems for teleoperation,” Assembly Automation, vol. 21, no. 2, pp. 143–152, 2001.
D. Wang, J. Yi, D. Zhao, and G. Yang, “Teleoperation system of the internet-based omnidirectional mobile robot with a mounted manipulator,” Proc. of International Conference on Mechatronics and Automation, pp. 1799–1804, 2007.
T. Kot, P. Novák, and J. Babjak, “Application of augmented reality in mobile robot teleoperation,” Proc. of International Workshop on Modelling and Simulation for Autonomous Systems, pp. 223–236, 2017.
P. M. Duong, T. T. Hoang, N. T. T. Van, D. A. Viet, and T. Q. Vinh, “A novel platform for internet-based mobile robot systems,” Proc. of 7th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 1972–1977, 2012.
X. Shen, Z. J. Chong, S. Pendleton, G. M. J. Fu, B. Qin, E. Frazzoli, and M. H. Ang Jr., “Teleoperation of on-road vehicles via immersive telepresence using off-the-shelf components,” Intelligent Autonomous Systems, vol. 13, pp. 1419–1433, 2016.
B. Jincun, L. Qi, and L. Yanfei, “The design of the rescue robot long-distance control based on 3G and GPS,” Proc. of International Conference on Intelligent Human-Machine Systems and Cybernetics, vol. 2, pp 170–172, 2009.
M. S. Uddin, M. Gianni, and A. Lab, “Long range robot teleoperation system based on internet of things,” Proc. of 2nd International Conference on Computer and Communication Systems (ICCCS), pp. 163–167, 2017.
K. M. Al-Aubidy, M. M. Ali, A. M. Derbas, and A. W. Al-Mutairi, “GPRS-based remote sensing and teleoperation of a mobile robot,” Proc. of International Multi-Conferences on Systems, Signals & Devices 2013 (SSD13), pp. 1–7, 2013.
R. Montúfar-Chaveznava, C. Paisano, J. M. Cañas, and E. de Jódar, “Simulating telerobotics by cellular telephony,” ICINCO, pp. 329–334, 2005.
J. Ryu, B. Yoo, and T. Nishimura, “Service robot operated by CDMA networks for security guard at home,” Service Robot Applications, IntechOpen, 2008.
H. H. O. Nasereddin and A. A. Abdelkarim, “Smartphone control robots through Bluetooth,” International Journal of Research Sciences, Reviews in Applied, vol. 4, no. 4, pp. 399–404, 2010.
X. Xue, S. X. Yang, and M.-H. Meng, “Remote sensing and teleoperation of a mobile robot via the internet,” Proc. of IEEE International Conference on Information Acquisition, p. 6, 2005.
T. Fong, C. Thorpe, and C. Baur, “Advanced interfaces for vehicle teleoperation: collaborative control, sensor fusion displays, and web-based tools,” Proc. of Vehicle Teleoperation Interfaces Workshop, IEEE International Conference on Robotics and Automation, no. CONF, 2000.
J. Y. C. Chen, E. C. Haas, and M. J. Barnes, “Human performance issues and user interface design for teleoperated robots,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 37, no. 6, pp. 1231–1245, 2007.
N. Maru, H. Kase, S. Yamada, A. Nishikawa, and F. Miyazaki, “Manipulator control by using servoing with the stereo vision,” Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’93), vol. 3, pp 1866–1870, 1993.
T. Fong, C. Thorpe, and C. Baur, “Advanced interfaces for vehicle teleoperation: collaborative control, sensor fusion displays, and remote driving tools,” Autonomous Robots, vol. 11, no. 1, pp. 77–85, 2001.
A. Amanatiadis, A. Gasteratos, C. Georgoulas, L. Kotoulas, and I. Andreadis, “Development of a stereo vision system for remotely operated robots: A control and video streaming architecture,” Proc. of IEEE Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, pp. 14–19, 2008.
T. Kot and P. Novák, “Utilization of the oculus rift HMD in mobile robot teleoperation,” Applied Mechanics and Materials, vol. 555, pp. 199–208, 2014.
P. Willemsen, M. B. Colton, S. H. Creem-Regehr, and W. B. Thompson, “The effects of head-mounted display mechanics on distance judgments in virtual environments,” Proc. of the 1st Symposium on Applied Perception in Graphics and Visualization, pp. 35–38, 2004.
U. Martinez-Hernandez, M. Szollosy, L. W. Boorman, H. Kerdegari, and T. J. Prescott, “Towards a wearable interface for immersive telepresence in robotics,” Interactivity, Game Creation, Design, Learning, and Innovation, Springer, pp. 65–73, 2016.
R. S. Allison, L. R. Harris, M. Jenkin, U. Jasiobedzka, and J. E. Zacher, “Tolerance of temporal delay in virtual environments,” Proc. of IEEE Virtual Reality, pp. 247–254, 2001.
F. A. Biocca and J. P. Rolland, “Virtual eyes can rearrange your body: Adaptation to visual displacement in see-through, head-mounted displays,” Presence, vol. 7, no. 3, pp. 262–277, 1998.
K. Mania, B. D. Adelstein, S. R. Ellis, and M. I. Hill, “Perceptual sensitivity to head tracking latency in virtual environments with varying degrees of scene complexity,” Proc. of the 1st Symposium on Applied Perception in Graphics and Visualization, pp. 39–47, 2004.
G. Doisy, A. Ronen, and Y. Edan, “Comparison of three different techniques for camera and motion control of a teleoperated robot,” Applied Ergonomics, vol. 58, pp 527–534, 2017.
V. Villani, F. Pini, F. Leali, and C. Secchi, “Survey on human-robot collaboration in industrial settings: Safety, intuitive interfaces and applications,” Mechatronics, vol. 55, pp 248–266, 2018.
H. Liu, T. Fang, T. Zhou, Y. Wang, and L. Wang, “Deep learning–based multimodal control interface for humanrobot collaboration,” Procedia CIRP, vol. 72, pp 3–8, 2018.
G. Du and P. Zhang, “Markerless human-robot interface for dual robot manipulators using Kinect sensor,” Robotics and Computer-Integrated Manufacturing, vol. 30, no. 2, pp. 150–159, 2014.
G. Du and P. Zhang, “A markerless human-robot interface using particle filter and Kalman filter for dual robots,” IEEE Transactions on Industrial Electronics, vol. 62, no. 4, pp. 2257–2264, 2014.
W. Ji, Y. Wang, H. Liu, and L. Wang, “Interface architecture design for minimum programming in humanrobot collaboration,” Procedia CIRP, vol. 72, pp 129–134, 2018.
R. Palmarini, I. F. del Amo, G. Bertolino, G. Dini, J. A. Erkoyuncu, R. Roy, and M. Farnsworth, “Designing an AR interface to improve trust in Human-Robots collaboration,” Procedia CIRP, vol. 70, pp 350–355, 2018.
K. Krückel, F. Nolden, A. Ferrein, and I. Scholl, “Intuitive visual teleoperation for UGVs using free-look augmented reality displays,” Proc. of IEEE International Conference on Robotics and Automation (ICRA), pp. 4412–4417, 2015.
H. Liu and L. Wang, “Gesture recognition for humanrobot collaboration: A review,” International Journal of Industrial Ergonomics, vol. 68, pp 355–367, 2018.
A. Spada, M. Cognetti, and A. de Luca, “Locomotion and telepresence in virtual and real worlds,” Human Friendly Robotics, Springer, pp. 85–98, 2019.
I. Rae, G. Venolia, J. C. Tang, and D. Molnar, “A framework for understanding and designing telepresence,” Proc. of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, pp. 1552–1566, 2015.
A. W. W. Yew, S. K. Ong, and A. Y. C. Nee, “Immersive augmented reality environment for the teleoperation of maintenance robots,” Procedia Cirp, vol. 61, pp 305–310, 2017.
H. Kim, T. di Giacomo, A. Egges, L. Lyard, S. Garchery, and N. Magnenat-Thalmann, “Believable virtual environment: Sensory and perceptual believability,” Utrecht University Repository, 2004.
A. Luciani, “Dynamics as common criterion to enhance the sense of presence in virtual environments,” Proc. of 7th Annual International Workshop on Presence, Valencia, Spain, pp. 96–103, 2004.
L. R. Elliott, C. Jansen, E. S. Redden, and R. A. Pettitt, “Robotic telepresence: Perception, performance, and user experience,” Army Research Lab Aberdeen Proving Ground MD Human Research and Engineering, 2012.
J. Gradecki, The Virtual Reality Programmer’s Kit, John Wiley & Sons, Inc., 1994.
N. Rodriguez, J.-P. Jessel, and P. Torguet, “A virtual reality tool for teleoperation research,” Virtual Reality, vol. 6, no. 2, pp. 57–62, 2002.
J. Zainan, L. Hong, W. Jie, and H. Jianbin, “Virtual reality-based teleoperation with robustness against modeling errors,” Chinese Journal of Aeronautics, vol. 22, no. 3, pp. 325–333, 2009.
P. K. Pook and D. H. Ballard, “Remote teleassistance,” Proc. of IEEE International Conference on Robotics and Automation, vol. 1, pp 944–949, 1995.
G. Terrien, T. Fong, C. Thorpe, and C. Baur, “Remote driving with a multisensor user interface,” SAE Technical Paper, 2000.
M. Mostefa, L. K. El Boudadi, A. Loukil, K. Mohamed, and D. Amine, “Design of mobile robot teleoperation system based on virtual reality,” Proc. of 3rd International Conference on Control, Engineering & Information Technology (CEIT), pp. 1–6, 2015.
D. Cheng-jun, D. Ping, Z. Ming-lu, and Z. Yan-fang, “Design of mobile robot teleoperation system based on virtual reality,” Proc. of IEEE International Conference on Automation and Logistics, pp. 2024–2029, 2009.
B. Ibari, Z. Ahmed-Foitih, and H. E. A. Reda, “Remote control of mobile robot using the virtual reality,” International Journal of Electrical Engineering, Computer, vol. 5, no. 5, 2015.
P. Františ and J. Hodický, “Human machine interface in command and control system,” Proc. of IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, pp. 38–41, 2010.
S. Kim, Y. Kim, J. Ha, and S. Jo, “Mapping system with virtual reality for mobile robot teleoperation,” Proc. of 18th International Conference on Control, Automation and Systems (ICCAS), p. 1541, 2018.
A. Jacoff, E. Messina, B. A. Weiss, S. Tadokoro, and Y. Nakagawa, “Test arenas and performance metrics for urban search and rescue robots,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003)(Cat. No. 03CH37453), vol. 4, pp 3396–3403, 2003.
J. Liu, Y. Wang, B. Li, and S. Ma, “Current research, key performances and future development of search and rescue robots,” Frontiers of Mechanical Engineering in China, vol. 2, no. 4, pp. 404–416, 2007.
H. Martins and R. Ventura, “Immersive 3-d teleoperation of a search and rescue robot using a head-mounted display,” Proc. of IEEE Conference on Emerging Technologies & Factory Automation, pp. 1–8, 2009.
L. Franzen, H. Kessock, D. Hinkle, S. Ravulakollu, and M. Bhakta, WorldToolKit Reference Manual, Engineering Animation inc, 1999.
K. Schilling and M. P. Vernet, “Field vehicle teleoperations support by virtual reality interfaces,” Proc. of of 15th IFAC World Congress, 2002.
R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre, “Recent advances in augmented reality,” Naval Research Lab Washington DC, 2001.
S. Feiner, B. MacIntyre, T. Höllerer, and A. Webster, “A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment,” Personal Technologies, vol. 1, no. 4, pp. 208–217, 1997.
A. Webster, S. Feiner, B. MacIntyre, W. Massie, and T. Krueger, “Augmented reality in architectural construction, inspection and renovation,” Proc. of ASCE Third Congress on Computing in Civil Engineering, vol. 1, p 996, 1996.
Z. Szalavári, E. Eckstein, and M. Gervautz, “Collaborative gaming in augmented reality,” Proc. of the ACM Symposium on Virtual Reality Software and Technology, pp. 195–204, 1998.
V. Le Ligeour, S. Otmane, and M. Mallem, “Augmented reality interface for free teleoperation,” IFAC Proceedings Volumes, vol. 38, no. 1, pp. 523–528, 2005.
S. Livatino, G. Muscato, F. Bannò, D. de Tommaso, and M. Macaluso, “Video and laser based augmented reality stereoscopic viewing for mobile robot teleoperation,” IFAC Proceedings Volumes, vol. 43, no. 23, pp. 161–168, 2010.
F. Cui, M. Zhang, and S. Guo, “Design of the intelligent control system of telerobot based on virtual reality,” Proc. of First International Conference on Innovative Computing, Information and Control-Volume I (ICICIC’06), vol. 1, pp 364–367, 2006.
M. Daily, Y. Cho, K. Martin, and D. Payton, “World embedded interfaces for human-robot interaction,” Proceedings of the 36th Annual Hawaii International Conference on System Sciences, p. 6, 2003.
J. R. G. Pretlove, “Augmented reality as a tool to aid the telerobotic exploration and characterization of remote environments,” Presence, vol. 2, no. 4, pp. 52–3673, 2002.
P. Milgram and F. Kishino, “A taxonomy of mixed reality visual displays,” IEICE Transactions on Information Systems, vol. 77, no. 12, pp. 1321–1329, 1994.
A. Hosseini and M. Lienkamp, “Enhancing telepresence during the teleoperation of road vehicles using HMD-based mixed reality,” Proc. of IEEE Intelligent Vehicles Symposium (IV), pp. 1366–1373, 2016.
F. Driewer, M. Sauer, and K. Schilling, “Mixed reality for teleoperation of mobile robots in search and rescue scenarios,” IFAC Proceedings Volumes, vol. 39, no. 3, pp. 267–272, 2006.
M. Sauer, M. Hess, and K. Schilling, “Towards a predictive mixed reality user interface for mobile robot teleoperation,” IFAC Proceedings Volumes, vol. 42, no. 22, pp. 91–96, 2009.
D. Lee, O. Martinez-Palafox, and M. W. Spong, “Bilateral teleoperation of a wheeled mobile robot over delayed communication network,” Proc. of IEEE International Conference on Robotics and Automation, ICRA 2006, pp. 3298–3303, 2006.
D. A. Lawrence, “Stability and transparency in bilateral teleoperation,” IEEE Transactions on Robotics Automation, vol. 9, no. 5, pp. 624–637, 1993.
M. Kitagawa, A. M. Okamura, B. T. Bethea, V. L. Gott, and W. A. Baumgartner, “Analysis of suture manipulation forces for teleoperation with force feedback,” Proc. of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 155–162, 2002.
K. Hashtrudi-Zaad and S. E. Salcudean, “Transparency in time-delayed systems and the effect of local force feedback for transparent teleoperation,” IEEE Transactions on Robotics and Automation, vol. 18, no. 1, pp. 108–114, 2002.
C. Pacchierotti, L. Meli, F. Chinello, M. Malvezzi, and D. Prattichizzo, “Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems,” The International Journal of Robotics Research, vol. 34, no. 14, pp. 1773–1787, 2015.
W. R. Ferrell, “Remote manipulation with transmission delay,” IEEE Transactions on Human Factors in Electronics, no. 1, pp. 24–32, 1965.
R. Luck and A. Ray, “Experimental verification of a delay compensation algorithm for integrated communication and control systems,” International Journal of Control, vol. 59, no. 6, pp. 1357–1372, 1994.
G. Adamides, C. Katsanos, Y. Parmet, G. Christou, M. Xenos, T. Hadzilacos, and Y. Edan, “HRI usability evaluation of interaction modes for a teleoperated agricultural robotic sprayer,” Applied ergonomics, vol. 62, pp 237–246, 2017.
S. Livatino, G. Muscato, F. Bannò, D. de Tommaso, and M. Macaluso, “Video and laser based augmented reality stereoscopic viewing for mobile robot teleoperation,” IFAC Proceedings Volumes, vol. 43, no. 23, pp. 161–168, 2010.
A. Wichmann, B. D. Okkalioglu, and T. Korkmaz, “The integration of mobile (tele) robotics and wireless sensor networks: A survey,” Computer Communications, vol. 51, pp 21–35, 2014.
J. Nieto, E. Slawinski, V. Mut, and B. Wagner, “Mobile robot teleoperation augmented with prediction and pathplanning,” IFAC Proceedings Volumes, vol. 43, no. 13, pp. 53–58, 2010.
S. Lee, G. Sukhatme, G. J. Kim, and C.-M. Park, “Haptic teleoperation of a mobile robot: A user study,” Presence: Teleoperators Environments, Virtual, vol. 14, no. 3, pp. 345–365, 2005.
J. Lim, J. Ko, and J. Lee, “Internet-based teleoperation of a mobile robot with force-reflection,” Proc. of IEEE Conference on Control Applications, CCA 2003, vol. 1, pp 680–685, 2003.
A. Shahzad and H. Roth, “Teleoperation of mobile robot using event based controller and real time force feedback,” Proc. of Scientific Cooperations International Workshops on Electrical and Computer Engineering Subfields, Istanbul, Turkey, pp. 7–12, 2014.
T. W. Fong, F. Conti, S. Grange, and C. Baur, “Novel interfaces for remote driving: gesture, haptic, and PDA,” Mobile Robots XV and Telemanipulator and Telepresence Technologies VII, vol. 4195, pp 300–311 2001.
H. Roth, K. Schilling, and O. J. Rösch, “Haptic interfaces for remote control of mobile robots,” IFAC Proceedings Volumes, vol. 35, no. 1, pp. 177–182, 2002.
N. Diolaiti and C. Melchiorri, “Teleoperation of a mobile robot through haptic feedback,” Proc. of IEEE International Workshop on Haptic Virtual Environments and Their Applications (HAVE), pp. 67–72, 2002.
S. Lee, G. S. Sukhatme, G. J. Kim, and C.-M. Park, “Haptic control of a mobile robot: A user study,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 3, pp 2867–2874, 2002.
O. Martinez-Palafox, D. Lee, M. W. Spong, I. Lopez, and C. T. Abdallah, “Bilateral teleoperation of mobile robot over delayed communication network: Implementation,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4193–4198, 2006.
R. J. Anderson and M. W. Spong, “Bilateral control of teleoperators with time delay,” IEEE Transactions on Automatic Control, vol. 34, no. 5, pp. 494–501, 1989.
M. W. Spong, S. Hutchinson, and M. Vidyasagar, Robot Modeling and Control, 1st ed., John Wiley & Sons, INC., 2006.
Y.-C. Liu and N. Chopra, “Control of semi-autonomous teleoperation system with time delays,” Automatica, vol. 49, no. 6, pp. 1553–1565, 2013.
G. Angelopoulos, G. T. Kalampokis, and M. Dasygenis, “An internet of things humanoid robot teleoperated by an open source Android application,” Proc. of Panhellenic Conference on Electronics and Telecommunications (PACET), pp. 1–4, 2017.
G. Paravati, B. Pralio, A. Sanna, and F. Lamberti, “A reconfigurable multi-touch remote control system for teleoperated robots,” Proc. of IEEE International Conference on Consumer Electronics (ICCE), pp. 153–154, 2011.
J. Nádvorník and P. Smutný, “Remote control robot using Android mobile device,” Proc. of the 15th International Carpathian Control Conference (ICCC), pp. 373–378, 2014.
P. P. Ray, L. Chettri, and N. Thapa, “IoRCar: IoT supported autonomic robotic movement and control,” Proc. of International Conference on Computation of Power, Energy, Information and Communication (ICCPEIC), pp. 77–83, 2018.
C.-F. Chien, K. H. Kim, B. Liu, and M. Gen, “Advanced decision and intelligence technologies for manufacturing and logistics,” Journal of Intelligent Manufacturing, vol. 23, no. 6, p. 2133, 2012.
J. Zhao, J. Zhang, Y. Feng, and J. Guo, “The study and application of the IoT technology in agriculture,” Proc. of 3rd International Conference on Computer Science and Information Technology, vol. 2, pp 462–465, 2010.
X. Chen, J. Liu, X. Li, L. Sun, and Y. Zhen, “Integration of IoT with smart grid,” Proc. of IET International Conference on Communication Technology and Application (ICCTA 2011), pp. 723–726, 2011.
H. Yujie and Z. Xihuang, “Research and application of PV monitoring system based on ZigBee and GPRS,” Proc. of 10th International Symposium on Distributed Computing and Applications to Business, Engineering and Science, pp. 338–342, 2011.
P. P. Ray, “Internet of robotic things: Concept, technologies, and challenges,” IEEE Access, vol. 4, pp 9489–9500, 2016.
R. Toris, C. Shue, and S. Chernova, “Message authentication codes for secure remote non-native client connections to ROS enabled robots,” Proc. of IEEE International Conference on Technologies for Practical Robot Applications (TePRA), pp. 1–6, 2014.
C. Crick, G. Jay, S. Osentoski, B. Pitzer, and O. C. Jenkins, “Rosbridge: ROS for Non-ROS users,” Robotics Research, Springer, pp. 493–504, 2017.
C. Turcu, C. Turcu, and V. Gaitan, “Merging the internet of things and robotics,” Recent Researches in Circuits Systems, pp. 499–504, 2012.
C. Yang, J. Luo, Y. Pan, Z. Liu, and C.-Y. Su Man, “Personalized variable gain control with tremor attenuation for robot teleoperation,” IEEE Transactions on Systems Systems, Cybernetics:, vol. 48, no. 10, pp. 1759–1770, 2017.
D. B. Kaber, E. Onal, and M. R. Endsley, “Design of automation for telerobots and the effect on performance, operator situation awareness, and subjective workload,” Human factors Manufacturing, Ergonomics in Industries, Service, vol. 10, no. 4, pp. 409–430, 2000.
J. Scholtz, B. Antonishek, and J. Young, “Evaluation of a human-robot interface: Development of a situational awareness methodology,” Proc. of 37th Annual Hawaii International Conference on System Sciences, p. 9, 2004.
M. R. Endsley, “Measurement of situation awareness in dynamic systems,” Human Factors, vol. 37, no. 1, pp. 65–84, 1995.
J. Scholtz, B. Antonishek, and J. Young, “Evaluation of operator interventions in autonomous off-road driving,” National Inst. oF Standards and Technology Gaithersburg MD, 2003.
A. Steinfeld, T. Fong, D. Kaber, M. Lewis, J. Scholtz, A. Schultz, and M. Goodrich, “Common metrics for humanrobot interaction,” Proc. of the 1st ACM SIGCHI/SIGART Conference on Human-robot Interaction, pp. 33–40, 2006.
M. S. Sanders and E. J. McCormick, Human Factors in Engineering and Design, McGraw-Hill, 1998.
J. A. Macedo, D. B. Kaber, M. R. Endsley, P. Powanusorn, and S. Myung, “The effect of automated compensation for incongruent axes on teleoperator performance,” Human Factors, vol. 40, no. 4, pp. 541–553, 1998.
W. M. Newman, M. G. Lamming, and M. Lamming, Interactive System Design, Addison-Wesley Reading, 1995.
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Recommended by Associate Editor Seok Chang Ryu under the direction of Editor Won-jong Kim. The authors of this work would like to acknowledge the funding support given by the Jiangsu Provincial Key Research & Development Plan (no. BE2017370).
Samwel Opiyo was born and raised in Kisumu, Kenya. He received a B.E. degree (physics, majoring in electronics) at Moi University-Kenya and graduated with second class upper division in 2008. He then enrolled for an M.S. degree (electronics and instrumentation) at Kenyatta University, Kenya, where he graduated in 2015. He has taught digital electronics, computer programing and robotics-related courses in Catholic University-Kenya, Mount Kenya University and Moi University-Kenya. He is currently a Ph.D. candidate majoring in electrification and automation at Nanjing Agricultural University specifically dealing with teleoperation of robots. His areas of interest include human-robot collaboration, digital image processing, teleoperation of robots, and IoT.
Jun Zhou is a full professor at Nanjing Agricultural University. He graduated, in 1998, with a B.E. degree from the Agricultral Mechanization Department, Nanjing Agricultural University, China. In 2003, he received a Ph.D. degree in agricultral engineering, Nanjing Agricultural University, China. In the year 2005, he completed his Postdoctoral Research Fellow, in mechatronics engineering, Shanghai Jiaotong University, China. He was a Visiting Scholar, in agicultural engineering, Iowa State University, USA, in 2013. His research interests are agricultural robot, digital image processing and pattern recognition, automation for agriculture, equipments for precision agriculture.
Emmy Mwangi graduated with the second upper B.Eng. degree (control and instrumentation) from Egerton University Kenya in 2017. She is currently an M.Sc. candidate majoring in agricultural electrification and automation at Nanjing Agriculural University. Her research interests include agricultural robots, digital image processing and control systems.
Wang Kai obtained his Master’s degree in mechatronics engineering from Changzhou University in 2012. He is curretly a Ph.D. candidate at Nanjing Agricultural University. His research directions are agricultural robots and robot navigation technology.
Idris Sunusi received his B.Eng. degree in agricultural engineering from Bayero University, Kano-Nigeria in 2009 and an M.Sc. degree in engineering (farm machinery) from Ahmadu Bello University, Zaria-Nigeria in 2016. He later joined the services of National agricultural extension and research liaison services, Ahmadu Bello University, Zaria in 2011 as an assistant lecturer/extension specialist. He has worked on the development of machinery for harvesting and processing of crop and is currently a Ph.D. research student in Nanjing Agricultural University, China. His research directions span dynamic control of electric vehicles, application of artificial intelligence in agricultural systems and automation of agricultural machinery.
About this article
Cite this article
Opiyo, S., Zhou, J., Mwangi, E. et al. A Review on Teleoperation of Mobile Ground Robots: Architecture and Situation Awareness. Int. J. Control Autom. Syst. 19, 1384–1407 (2021). https://doi.org/10.1007/s12555-019-0999-z