Skip to main content

Towards Intercontinental Teleoperation: A Cloud-Based Framework for Ultra-Remote Human-Robot Dual-Arm Motion Mapping

  • Conference paper
  • First Online:
Intelligent Robotics and Applications (ICIRA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14273))

Included in the following conference series:

  • 620 Accesses

Abstract

Teleoperated robots offer a unique combination of robotic strength and precision, coupled with human judgment and ingenuity, to enhance productivity and safety in complex, unstructured, and hazardous industrial environments. However, teleoperation over ultra-remote distances has posed a significant challenge, with control performance being compromised by the inherent trade-off between distance and latency. This study presents a novel cloud-based human-robot motion mapping framework for ultra-remote teleoperation. An intuitive dual-arm human-robot motion mapping framework based on capturing human motion has been implemented, and private network connectivity is established using the Google Cloud platform for the efficient transfer of control data required for human-robot motion mapping. Moreover, a novel feedforward control framework is proposed to mitigate the negative impact of high network latency on teleoperation control performance. To demonstrate the feasibility of the proposed approach, an intercontinental teleoperation system between China and Sweden was developed, and a series of practical applications were demonstrated by capturing the motion of an operator in Sweden to control the dual-arm robot located over 7800 km away in China. The proposed feedforward control framework for intercontinental teleoperation has the potential utility for improving human-robot interaction performance in ultra-remote teleoperation scenarios with high network latency. More demonstration videos can be found at https://fsie-robotics.com/intercontinental-teleoperation/.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hirche, S., Buss, M.: Human-oriented control for haptic teleoperation. Proc. IEEE 100(3), 623–647 (2012)

    Article  Google Scholar 

  2. Yang, G., Lv, H., Zhang, Z., Yang, L., Deng, J., You, S., et al.: Keep healthcare workers safe: Application of teleoperated robot in isolation ward for COVID-19 prevention and control. Chin. J. Mech. Eng. 33(1), 47 (2020)

    Article  Google Scholar 

  3. Mahler, J., Matl, M., Satish, V., Danielczuk, M., DeRose, B., McKinley, S., et al.: Learning ambidextrous robot grasping policies. Sci. Robot. 4(26) (2019)

    Google Scholar 

  4. Lv, H., Yang, G., Zhou, H., Huang, X., Yang, H., Pang, Z.: Teleoperation of collaborative robot for remote dementia care in home environments. IEEE J. Transl. Eng. Health. Med. 8, 1400510 (2020)

    Article  Google Scholar 

  5. Lv, H., Kong, D., Pang, G., Wang, B., Yu, Z., Pang, Z., et al.: GuLiM: a hybrid motion mapping technique for teleoperation of medical assistive robot in combating the COVID-19 pandemic. IEEE Trans. Med. Robot. Bio. 4(1), 106–117 (2022)

    Article  Google Scholar 

  6. Parsaei, M.R., Boveiri, H.R., Javidan, R., Khayami, R.: Telesurgery QoS improvement over SDN based on a Type-2 fuzzy system and enhanced cuckoo optimization algorithm. Int. J. Commun. Syst. 33(11) (2020)

    Google Scholar 

  7. Moya, V., Slawiñski, E., Mut, V., Chávez, D., Wagner, B.: Stable bilateral teleoperation control method for biped robots with time-varying delays. J. Rob. 2023, 3197743 (2023)

    Google Scholar 

  8. Lv, H., Pang, Z., Xiao, M., Yang, G.: Hardware-in-the-loop simulation for evaluating communication impacts on the wireless-network-controlled robots. In: Proc. Annu. Conf. IEEE Ind. Electron. Soc., Brussels, Belgium (2022)

    Google Scholar 

  9. Aleksy, M., Dai, F., Enayati, N., Rost, P., Pocovi, G.: Utilizing 5G in industrial robotic applications. In: Proceeding of FiCloud (2019)

    Google Scholar 

  10. Lyu, H., Bengtsson, A., Nilsson, S., Pang, Z., Isaksson, A., Yang, G.: Latency-aware control for wireless cloud fog automation: framework and case study. TechRxiv (2023). https://doi.org/10.36227/techrxiv.22586044.v2

    Article  Google Scholar 

  11. Lv, H., Pang, Z., Bhimavarapu, K., Yang, G.: Impacts of wireless on robot control: the network hardware-in-the-loop simulation framework and real-life comparisons. IEEE Trans. Industr. Inform. 19(9), 9255–9265 (2023)

    Article  Google Scholar 

  12. Chen, Z., Huang, F., Sun, W., Gu, J., Yao, B.: RBF-neural-network-based adaptive robust control for nonlinear bilateral teleoperation manipulators with uncertainty and time delay. IEEE/ASME Trans. Mechatron. 25(2), 906–918 (2020)

    Article  Google Scholar 

  13. Annamraju, S., Pediredla, V.K., Thondiyath, A.: Lyapunov stable teleoperation controllers using passivity analysis. IFAC-PapersOnLine. 53(1), 435–440 (2020)

    Article  Google Scholar 

  14. Baranitha, R., Mohajerpoor, R., Rakkiyappan, R.: Bilateral teleoperation of single-master multislave systems with semi-markovian jump stochastic interval time-varying delayed communication channels. IEEE Trans. Cybern. 51(1), 247–257 (2021)

    Article  Google Scholar 

  15. Handa, A., Wyk, K.V., Yang, W., Liang, J., Chao, Y.W., Wan, Q., et al.: DexPilot: vision-based teleoperation of dexterous robotic hand-arm system. In: Proc. DexPilot: Vision-Based Teleoperation of Dexterous Robotic Hand-Arm System (2020)

    Google Scholar 

  16. Mathew, S., Varia, J.: Overview of amazon web services. Amazon Whitepapers 105, 1–22 (2014)

    Google Scholar 

  17. Wilder, B.: Cloud Architecture Patterns: Using Microsoft Azure. O’Reilly Media Inc, US (2012)

    Google Scholar 

  18. Bisong, E.: An Overview of Google Cloud Platform Services. Apress, Berkeley, CA (2019)

    Book  Google Scholar 

  19. Noitom-Ltd.: Perception neuron. https://noitom.com/perception-neuron-series. Accessed 21 May 2023

  20. Beeson, P., Ames, B.: TRAC-IK: an open-source library for improved solving of generic inverse kinematics. In: Proc. IEEE-RAS Int. Conf. Hum. Robot., Seoul, Korea (2015)

    Google Scholar 

  21. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., et al.: ROS: an open-source Robot Operating System. In: Proc. IEEE Int. Conf. Robot. Autom. workshop, Kobe, Japan (2009)

    Google Scholar 

  22. Bhimavarapu, K., Pang, Z., Dobrijevic, O., Wiatr, P.: Unobtrusive, accurate, and live measurements of network latency and reliability for time-critical internet of things. IEEE Internet Things Mag. 5(3), 38–43 (2022)

    Article  Google Scholar 

Download references

Acknowledgment

This work was supported in part by the National Natural Science Foundation of China (No. 51975513), the Natural Science Foundation of Zhejiang Province, China (No. LR20E050003), the Major Research Plan of National Natural Science Foundation of China (No. 51890884), the Major Research Plan of Ningbo Innovation 2025. (Grant No. 2020Z022), the Bellwethers Research and Development Plan of Zhejiang Province (No. 2023C01045), and the Swedish Foundation for Strategic Research (SSF) through the project APR20–0023. This work was conducted by Honghao Lv during his tenure as a visiting doctoral student at the Royal Institute of Technology (KTH) in Sweden. Honghao Lv gratefully acknowledges financial support from China Scholarship Council.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Geng Yang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lv, H., Zhou, H., Wang, R., Wu, H., Pang, Z., Yang, G. (2023). Towards Intercontinental Teleoperation: A Cloud-Based Framework for Ultra-Remote Human-Robot Dual-Arm Motion Mapping. In: Yang, H., et al. Intelligent Robotics and Applications. ICIRA 2023. Lecture Notes in Computer Science(), vol 14273. Springer, Singapore. https://doi.org/10.1007/978-981-99-6498-7_12

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-6498-7_12

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-6497-0

  • Online ISBN: 978-981-99-6498-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics