Teleoperation of On-Road Vehicles via Immersive Telepresence Using Off-the-shelf Components

  • Xiaotong Shen
  • Zhuang Jie Chong
  • Scott Pendleton
  • Guo Ming James Fu
  • Baoxing Qin
  • Emilio Frazzoli
  • Marcelo H. AngJr.
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 302)


The quality of visual information and response time are crucial aspects of any modern teleoperation system. This is especially true for operation of on-road vehicles, which must function in highly dynamic, unforgiving environments. In this work we demonstrate that a suitable teleoperation system can be exclusively composed of low-cost off-the-shelf components yet still meet the high performance demands of remotely driving a car on the road. The user is given immersive situational awareness through an on-board head-mounted display linked to an actuated stereoscopic camera, thereby maintaining depth perception and intuitive camera control. Communication speeds are evaluated over various wireless connection types, and a usability study shows that the system allows for advanced driving maneuvers while remotely controlled. 3G and 4G data networks are demonstrated to provide adequate bandwidth for the task given proper data compression, thus expanding the potential range for teleoperation. Applications for such a system are further discussed, extending to fleet management and autonomous vehicle safety measures.


Teleoperation Telepresence Stereoscopic Head-mounted display Head-aimed vision Latency On-road vehicle Unmanned ground vehicle 



This research was supported by the National Research Foundation (NRF) Singapore through the Singapore-MIT Alliance for Research and Technology’s (FM IRG) research programme, in addition to the partnership with the Defence Science Organization (DSO). We are grateful for their support.


  1. 1.
    Yamauchi, B., Massey, K.: Stingray: High-Speed Teleoperation Of Ugvs In Urban Terrain Using Driver-Assist Behaviors And Immersive Telepresence. Proceedings of the Army Science Conference (26th), December (2008).Google Scholar
  2. 2.
    Tyczka, D.R., Wright R., Janiszewski, B., Chatten, M.J., Bowen, T.A., Skibba, B.: Study of high-definition and stereoscopic head-aimed vision for improved teleoperation of an unmanned ground vehicle. Proc. SPIE 8387, Unmanned Systems Technology XIV, 83870L May 1, (2012).Google Scholar
  3. 3.
    Brady, K., Tarn, T.: Internet-based remote teleoperation. Robotics and Automation, 1998. Proceedings. 1998 IEEE International Conference on, vol. 1, no., pp. 65,70 vol.1, 16–20 May (1998).Google Scholar
  4. 4.
    Mohajerpoor, R., Dezfuli, S.S., Bahadori, B.: Teleoperation of an unmanned car via robust adaptive backstepping control approach. Advanced Intelligent Mechatronics (AIM), 2013 IEEE/ASME International Conference on, vol., no., pp. 1540–1545, 9–12 July (2013).Google Scholar
  5. 5.
    Suomela, J.: Tele-presence Aided Teleoperation of Semi-autonomous Work Vehicles. PhD diss., Helsinki University of Technology (2001)Google Scholar
  6. 6.
    Tang, T., Vetter, P., Finkl, S., Figel, K., Leinkamp, M.: Teleoperated Road Vehicles - The “Free Corridor” as a Safety Strategy Approach. Mechanics and Materials, 490–491, 1399 (2014)Google Scholar
  7. 7.
    Kim, J., Ryu, J.: Shared teleoperation of a vehicle with a virtual driving interface. Control, Automation and Systems (ICCAS), 2013 13th International Conference on, vol., no., pp. 851–857, 20–23 Oct. (2013).Google Scholar
  8. 8.
    Chong, Z.J., Qin, B., Bandyopadhyay, T., Wongpiromsarn, T., Rankin, E.S., Ang, M.H., Frazzoli, E., Rus, D., Hsu, D., Low, K.H.: Autonomous personal vehicle for the first- and last-mile transportation services. Cybernetics and Intelligent Systems (CIS), 2011 IEEE 5th International Conference on, pp. 253–260, 17–19 Sept. (2011).Google Scholar
  9. 9.
    Chong, Z.J., Qin, B., Bandyopadhyay, T., Wongpiromsarn, T., Rebsamen, B., Dai, P.; Kim, S., Ang, M.H., Hsu, D., Rus, D., Frazzoli, E.: Autonomy for mobility on demand. Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pp. 4235–4236, 7–12 Oct. (2012).Google Scholar
  10. 10.
    Amanatiadis, A., Gasteratos, A.: Stereo Vision System for Remotely Operated Robots. Remote and Telerobotics, Nicolas Mollet (Ed.), ISBN: 978-953-307-081-0, InTech (2010).Google Scholar
  11. 11.
    Nielsen, C.W., Goodrich, M.A., Ricks, R.W.: Ecological Interfaces for Improving Mobile Robot Teleoperation. Robotics, IEEE Transactions on, vol. 23, no. 5, pp. 927–941, Oct. (2007).Google Scholar
  12. 12.
    Tang, T., Chucholowski, F., Lienkamp, M.: Teleoperated Driving Basics and System Design. ATZ worldwide, vol. 116, no. 2, pp. 16–19. Feb. (2014).Google Scholar
  13. 13.
    Nguyen, L.A., Bualat, M., Ewards, L.J., Flueckiger, L., Neveu, C., Schweher, K., Wagner, M.D., Zbinden, E.: Virtual Reality Interfaces for Visualization and Control of Remote Vehicles. Autonmous Robots, vol. 11, no. 1, pp. 59–68 July (2001).Google Scholar
  14. 14.
    Drascic, D.: Skill Acquisition and Task Performance in Teleoperation Using Monoscopic and Stereoscopic Video Remote Viewing. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 1991, 35: 1367 (1991).Google Scholar
  15. 15.
    Chucholowski, F., Tang, T., Lienkamp, M.: Teleoperated Driving Robust and Secure Data Connections. ATZelektronik worldwide, vol. 9, no. 1, pp. 42–45, Feb (2014).Google Scholar
  16. 16.
    Kim, S., Chong, Z.J., Qin, B., Shen, X., Cheng, Z., Liu, W., Ang, M.H.: Cooperative perception for autonomous vehicle control on the road: Motivation and experimental results. Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, pp. 5059–5066, 3–7 Nov. (2013).Google Scholar
  17. 17.
    Quigley, M., Conley, K., Gerkey, B.P., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: ROS: an open-source Robot Operating System. ICRA Workshop on Open Source Software (2009).Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Xiaotong Shen
    • 1
  • Zhuang Jie Chong
    • 1
  • Scott Pendleton
    • 1
  • Guo Ming James Fu
    • 2
  • Baoxing Qin
    • 1
  • Emilio Frazzoli
    • 3
  • Marcelo H. AngJr.
    • 1
  1. 1.National University of SingaporeSingaporeSingapore
  2. 2.Singapore-MIT Alliance for Research and TechnologySingaporeSingapore
  3. 3.Massachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations