Skip to main content
SpringerLink
Log in

Adaptive online prediction of operator position in teleoperation with unknown time-varying delay: simulation and experiments

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

One of the most important problems in teleoperation systems is time delay and packet loss in the communication channel, which can affect transparency and stability. One way to overcome the time delay effects in a teleoperation system is to predict the master-side motion. In this way, when data is received in the slave side, it will be considered as the current position of the master robot and, thus, complete transparency could be achieved. The majority of the previous works regarding operator position prediction have considered known and constant time delay in the system; however, in the real applications, time delay is unknown and variable. In this paper, an adaptive online prediction approach based on artificial neural network (NN) is proposed. The time delay of the communication channel is estimated using an observer based on the dynamics of the master and slave sides. Then an artificial NN predicts the master-side motion based on the current available data of the master robot and the variable time delay estimated by the observer. This adaptive prediction approach is utilized in simulations and experiments on Phantom Omni haptic devices. The simulation results indicate the feasibility of this approach. It is revealed that this approach can predict an alternative human’s hand motion in a teleoperation system with unknown and variable time delay. Finally, the simulation results would be supported by experimental results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31

Similar content being viewed by others

References

  1. Uddin R, Ryu J (2016) Predictive control approaches for bilateral teleoperation. Ann Rev Control 42:82–99

    Article  Google Scholar 

  2. Passenberg C, Peer A, Buss M (2010) A survey of environment-, operator-, and task-adapted controllers for teleoperation systems. Mechatronics 20(7):787–801

    Article  Google Scholar 

  3. Choi H, Jung S (2020) Teleoperation control of a position-based impedance force controlled mobile robot by neural network learning: experimental studies. Asian J Control 22(1):92–103

    Article  Google Scholar 

  4. Xu X, Cizmeci B, Schuwerk C, Steinbach E (2016) Model-mediated teleoperation: toward stable and transparent teleoperation systems. IEEE Access 4:425–449

    Article  Google Scholar 

  5. Mitra P, Niemeyer G (2008) Model-mediated telemanipulation. Int J Robot Res 27(2):253–262

    Article  Google Scholar 

  6. Yazdankhoo B, Beigzadeh B (2019) Increasing stability in model-mediated teleoperation approach by reducing model jump effect. Sci Iran 26:3–14. https://doi.org/10.24200/sci.2017.20007(Special Issue on: Socio-Cognitive Engineering)

    Article  Google Scholar 

  7. Li S, Bowman M, Nobarani H, Zhang X (2020) Inference of manipulation intent in teleoperation for robotic assistance. J Intell Rob Syst. https://doi.org/10.1007/s10846-019-01125-8

    Article  Google Scholar 

  8. Jarrassé N, Paik J, Pasqui V, Morel G (2008) How can human motion prediction increase transparency? In: 2008 IEEE international conference on robotics and automation (ICRA), Pasadena, CA, USA. IEEE, pp 2134–2139

  9. Uddin R, Park S, Ryu J (2016) A predictive energy-bounding approach for Haptic teleoperation. Mechatronics 35:148–161

    Article  Google Scholar 

  10. Engelbrecht SE (2001) Minimum principles in motor control. J Math Psychol 45(3):497–542

    Article  MathSciNet  Google Scholar 

  11. Smith C, Jensfelt P (2010) A predictor for operator input for time-delayed teleoperation. Mechatronics 20(7):778–786

    Article  Google Scholar 

  12. Feth D, Peer A, Buss M (2014) Enhancement of multi-user teleoperation systems by prediction of dyadic haptic interaction. In: Experimental robotics. Springer, pp 855–869

  13. Prekopiou P, Tzafestas SG, Harwin WS (1999) Towards variable-time-delays-robust telemanipulation through master state prediction. In: IEEE/ASME international conference on advanced intelligent mechatronics, Atlanta, USA. IEEE, pp 305–310

  14. Clarke S, Schillhuber G, Zaeh MF, Ulbrich H (2008) Prediction-based methods for teleoperation across delayed networks. Multimedia Syst 13(4):253–261

    Article  Google Scholar 

  15. Stakem F, AlRegib G (2009) An adaptive approach to exponential smoothing for CVE state prediction. In: 2nd International conference on immersive telecommunications, Berkley, USA. Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering (ICST), pp 141–146

  16. Sakr N, Georganas ND, Zhao J, Shen X (2007) Motion and force prediction in haptic media. In: 2007 IEEE International Conference on Multimedia and Expo, Beijing, China. IEEE, pp 2242–2245

  17. Pentland A, Liu A (1999) Modeling and prediction of human behavior. Neural Comput 11(1):229–242

    Article  Google Scholar 

  18. Hua C-C, Yang Y, Guan X (2013) Neural network-based adaptive position tracking control for bilateral teleoperation under constant time delay. Neurocomputing 113:204–212

    Article  Google Scholar 

  19. Nava J, Kreinovich V (2014) Why a model produced by training a neural network is often more computationally efficient than a nonlinear regression model: a theoretical explanation. J Uncertain Syst 8(3):193–204

    Google Scholar 

  20. Nicolau V, Palade V, Aiordachioaie D, Miholca C (2007) Neural network prediction of the roll motion of a ship for intelligent course control. In: International conference on knowledge-based and intelligent information and engineering systems, Vietri sul Mare, Italy. Springer, pp 284–291

  21. Alvarez-Aguirre A, Van De Wouw N, Oguchi T, Nijmeijer H (2014) Predictor-based remote tracking control of a mobile robot. IEEE Trans Control Syst Technol 22(6):2087–2102

    Article  Google Scholar 

  22. Alt GH, Lages WF (2003) Networked robot control with delay compensation. In: 5th Real-Time Linux Workshop

  23. Lian F-L, Moyne J, Tilbury D (2001) Time delay modeling and sample time selection for networked control systems. In: Proceedings of ASME-DSC. DCS, New York, pp 5–6

  24. Jiang W, Schulzrinne H (2000) Modeling of packet loss and delay and their effect on real-time multimedia service quality. In: Proceedings of NOSSDAV’2000. Citeseer

  25. Liu T, Boumaiza S, Ghannouchi F (2004) Dynamic behavioral modeling of 3G power amplifiers using real-valued time-delay neural networks. IEEE Trans Microw Theory Tech 52(3):1025–1033

    Article  Google Scholar 

  26. Han K-H, Kim S, Kim Y-J, Kim J-H (2001) Internet control architecture for internet-based personal robot. Auton Robot 10(2):135–147

    Article  Google Scholar 

  27. Juanping Z, Xianwen G (2009) Time-delay analysis and estimation of Internet-based robot teleoperation system. In: Control and decision conference, CCDC’09. Chinese. IEEE, pp 4643–4646

  28. Çetin K, Bayrak A, Tatlicioglu E (2016) Online time delay estimation in networked control systems with application to bilateral teleoperation. In: Control conference (ECC), European. IEEE, pp 1007–1012

  29. Teklemariam HG, Das A (2017) A case study of Phantom Omni force feedback device for virtual product design. Int J Interact Des Manuf (IJIDeM) 11(4):881–892

    Article  Google Scholar 

  30. Chen H, Huang P, Liu Z, Ma Z (2020) Time delay prediction for space telerobot system with a modified sparse multivariate linear regression method. Acta Astronaut 166:330–341

    Article  Google Scholar 

  31. Yazdankhoo B, Nikpour M, Beigzadeh B, Meghdari A (2019) Improvement of operator position prediction in teleoperation systems with time delay: simulation and experimental studies on Phantom Omni devices. Jordan J Mech Ind Eng 13(3):197–205

    Google Scholar 

  32. Mehrotra K, Mohan CK, Ranka S (2000) Elements of artificial neural networks. MIT Press, Cambridge

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Biomechatronics and Cognitive Engineering Research Lab, School of Mechanical Engineering, Iran University of Science and Technology, P.O.B. 16765163, Tehran, Iran

    Moein Nikpour, Behnam Yazdankhoo & Borhan Beigzadeh

  2. Social and Cognitive Robotics Laboratory, Center of Excellence in Design, Robotics and Automation (CEDRA), Sharif University of Technology, P.O.B. 11365-9567, Tehran, Iran

    Ali Meghdari

Authors
  1. Moein Nikpour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Behnam Yazdankhoo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Borhan Beigzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ali Meghdari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Behnam Yazdankhoo or Borhan Beigzadeh.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nikpour, M., Yazdankhoo, B., Beigzadeh, B. et al. Adaptive online prediction of operator position in teleoperation with unknown time-varying delay: simulation and experiments. Neural Comput & Applic 33, 7575–7592 (2021). https://doi.org/10.1007/s00521-020-05502-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05502-5

Keywords

Search

Navigation