Skip to main content
SpringerLink
Log in

Human stochastic closed-loop behavior for master-slave teleoperation using multi-leap-motion sensor

  • Article
  • Published:
Science China Technological Sciences Aims and scope Submit manuscript

Abstract

Teleoperation has a wide range of applications that have been under development over the past two decades. Previous researches have focused on the control design of teleoperation machine systems to deal with obstacles such as time-delayed stability and transparency. Recent researches have shown that the inclusion of human closed-loop dynamics in control design can improve the performance of robot telemanipulation. The complexity of human behavior arises from the uncertainty of both human physiology and psychology; hence, the investigation can benefit from empirical studies. This study develops a type of statistical learning method to model and evaluate human stochastic closed-loop behavior, which is considered as a hand motion during the direct incremental control process of master-slave teleoperation. The hand trajectory is empirically considered as having a binary linear regression relationship with the error and error rate between the demanded and simulated teleoperator trajectories, while random movements with zero error and error rate are discovered. Hand movement tracking is achieved using a multi-leap-motion sensor (MLM), which is a markerless and natural infrared vision-based manner for motion capture. The established behavior model and statistical learning results reveal certain human properties of operational activities including visual perception, decision making, and robot telemanipulation. The properties indicate some probable system enhancements for future work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Iborra A, Pastor J A, Alvarez B, et al. Robots in radioactive environments. IEEE Robot Autom Mag, 2003, 10: 12–22

    Article  Google Scholar 

  2. El-Fakdi A, Cufí X, Hurtós N, et al. Team-based building of a remotely operated underwater robot, an innovative method of teaching engineering. J Intell Robot Syst, 2016, 81: 51–61

    Article  Google Scholar 

  3. Imaida T, Yokokohji Y, Doi T, et al. Ground-space bilateral teleoperation of ETS-VII robot arm by direct bilateral coupling under 7-s time delay condition. IEEE T Robotic Autom, 2004, 20: 499–511

    Article  Google Scholar 

  4. Norris J S, Powell M W, Vona M A, et al. Mars exploration rover operations with the science activity planner. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005. Barcelona: IEEE, 2005. 4618–4623

    Google Scholar 

  5. Ding L, Gao H B, Deng Z Q, et al. Three-layer intelligence of planetary exploration wheeled mobile robots: Robint, virtint, and humint. Sci China Tech Sci, 2015, 58: 1299–1317

    Article  Google Scholar 

  6. Rembold U, Burghart C R. Surgical robotics: An introduction. J Intell Robot Syst, 2001, 30: 1–28

    Article  MATH  Google Scholar 

  7. Koizumi N, Warisawa S, Nagoshi M, et al. Construction methodology for a remote ultrasound diagnostic system. IEEE Trans Robot, 2009, 25: 522–538

    Article  Google Scholar 

  8. Enloe C L, Pakula W A, Finney G A, et al. Teleoperation in the undergraduate physics laboratory-teaching an old dog new tricks. IEEE Trans Educ, 1999, 42: 174–179

    Article  Google Scholar 

  9. Baier H, Schmidt G. Transparency and stability of bilateral kinesthetic teleoperation with time-delayed communication. J Intell Robot Syst, 2004, 40: 1–22

    Article  Google Scholar 

  10. Kron A, Schmidt G. Stability and performance analysis of kinesthetic control architectures for bimanual telepresence systems. J Intell Robot Syst, 2006, 46: 1–26

    Article  Google Scholar 

  11. Lii N Y, Chen Z P, Pleintinger B, et al. Toward understanding the effects of visual- and force-feedback on robotic hand grasping performance for space teleoperation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2010. Taipei: IEEE, 2010. 3745–3752

    Chapter  Google Scholar 

  12. Thompson R L, Daniel R W, Murray D W. Experiments on operator matching in visual teleoperation. In: 2000 IEEE International Conference on Systems, Man, and Cybernetics, 2000. Nashville: IEEE, 2000. 931–936

    Google Scholar 

  13. Cong S, Wang J N. Internet-based and visual feedback networked robot arm teleoperation system. In: International Conference on Networking, Sensing and Control. Chicago: IEEE, 2010. 452–457

    Google Scholar 

  14. Hu T, Huang X, Tan Q. Active disturbance rejection controller for space teleoperation. In: International Conference on Automatic Control and Artificial Intelligence (ACAI 2000). Wuhan: IEEE, 2012. 334–337

    Chapter  Google Scholar 

  15. Anderson R J, Spong M W. Bilateral control of teleoperators with time delay. IEEE T Automat Contr, 1989, 34: 494–501

    Article  MathSciNet  Google Scholar 

  16. Salcudean S E, Hashtrudi-Zaad K, Tafazoli S, et al. Bilateral matched impedance teleoperation with application to excavator control. IEEE Contr Syst Mag, 1999, 19: 29–37

    Article  Google Scholar 

  17. Prokopiou P A, Tzafestas S G, Harwin W S. A novel scheme for human-friendly and time-delays robust neuropredictive teleoperation. J Intell Robot Syst, 1999, 25: 311–340

    Article  Google Scholar 

  18. Hirche S, Buss M. Human-oriented control for haptic teleoperation. Proc IEEE, 2012, 100: 623–647

    Article  Google Scholar 

  19. Ou Y, Qian H, Xu Y. Support vector machine based approach for abstracting human control strategy in controlling dynamically stable robots. J Intell Robot Syst, 2009, 55: 39–54

    Article  MATH  Google Scholar 

  20. Medina J R, Lorenz T, Hirche S. Synthesizing anticipatory haptic assistance considering human behavior uncertainty. IEEE Trans Robot, 2015, 31: 180–190

    Article  Google Scholar 

  21. Du G, Zhang P. A markerless human-robot interface using particle filter and kalman filter for dual robots. IEEE Trans Ind Electron, 2015, 62: 2257–2264

    Article  Google Scholar 

  22. Lawrence D A. Stability and transparency in bilateral teleoperation. IEEE Trans Rob Aut, 1993, 9: 624–637

    Article  Google Scholar 

  23. Lee D, Spong M W. Passive bilateral teleoperation with constant time delay. IEEE Trans Robot, 2006, 22: 269–281

    Article  Google Scholar 

  24. Chopra N, Spong M W, Hirche S, et al. Bilateral teleoperation over the internet: the time varying delay problem. In: Proceedings of the 2003 American Control Conference. Denver, Colorado, USA. 2003. 155–160

    Google Scholar 

  25. Guo S P, Li D X, Meng Y H, et al. Task space control of free-floating space robots using constrained adaptive RBF-NTSM. Sci China Tech Sci, 2014, 57: 828–837

    Article  Google Scholar 

  26. Li Z, Xia Y. Adaptive neural network control of bilateral teleoperation with unsymmetrical stochastic delays and unmodeled dynamics. Int J Robust Nonlin, 2014, 24: 1628–1652

    Article  MathSciNet  MATH  Google Scholar 

  27. Li Z, Xia Y, Wang D, et al. Neural network-based control of networked trilateral teleoperation with geometrically unknown constraints. IEEE Trans Cybern, 2016, 46: 1051–1064

    Article  Google Scholar 

  28. Ando T, Tsukahara R, Seki M, et al. A haptic interface “force blinker 2” for navigation of the visually impaired. IEEE Trans Ind Electron, 2012, 59: 4112–4119

    Article  Google Scholar 

  29. Du G, Zhang P, Li D. Human-manipulator interface based on multisensory process via Kalman filters. IEEE Trans Ind Electron, 2014, 61: 5411–5418

    Article  Google Scholar 

  30. Bachmann E R, McGhee R B, Yun Z, et al. Inertial and magnetic posture tracking for inserting humans into networked virtual environments. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology. Banff, Canada, 2001. 9–16

    Google Scholar 

  31. Rebelo J, Sednaoui T, den Exter E B, et al. Bilateral robot teleoperation: A wearable arm exoskeleton featuring an intuitive user interface. IEEE Robot Autom Mag, 2014, 21: 62–69

    Article  Google Scholar 

  32. Ma Z, Ben-Tzvi P. RML glove—An exoskeleton glove mechanism with haptics feedback. IEEE/ASME Trans Mech, 2015, 20: 641–652

    Article  Google Scholar 

  33. Kofman J, Verma S, Wu X H. Robot-manipulator teleoperation by markerless vision-based hand-arm tracking. Int J Optomechatron, 2007, 1: 331–357

    Article  Google Scholar 

  34. Kofman J, Wu X, Luu T J, et al. Teleoperation of a robot manipulator using a vision-based human-robot interface. IEEE Trans Ind Electron, 2005, 52: 1206–1219

    Article  Google Scholar 

  35. Du G, Zhang P. Markerless human-robot interface for dual robot manipulators using Kinect sensor. Robot Comput Integr Manuf, 2014, 30: 150–159

    Article  Google Scholar 

  36. Weichert F, Bachmann D, Rudak B, et al. Analysis of the accuracy and robustness of the leap motion controller. Sensors, 2013, 13: 6380–6393

    Article  Google Scholar 

  37. James G, Witten D, Hastie T, et al. An introduction to statistical learning. 2014, http://www.springer.com/series/417

    MATH  Google Scholar 

  38. Xu W, Liang B, Xu Y, et al. A ground experiment system of freefloating robot for capturing space target. J Intell Robot Syst, 2007, 48: 187–208

    Article  Google Scholar 

  39. Matlin M W. Cognitive psychology. 2013, http://www.islib.com/ecommerce/resources/online-read-resource/276403/

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Beijing Institute of Tracking and Telecommunication Technology, Beijing, 100094, China

    TianJian Hu & WeiPing Qian

  2. School of Aerospace Engineering, Tsinghua University, Beijing, 100084, China

    TianJian Hu, TianShu Wang & JunFeng Li

  3. Shenzhen Laboratory of Space Robotics and Telescience, Shenzhen, 518055, China

    XiaoJun Zhu & XueQian Wang

  4. The Graduate School of Shenzhen, Tsinghua University, Shenzhen, 518055, China

    XueQian Wang

Authors
  1. TianJian Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. XiaoJun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. XueQian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. TianShu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. JunFeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. WeiPing Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to XueQian Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, T., Zhu, X., Wang, X. et al. Human stochastic closed-loop behavior for master-slave teleoperation using multi-leap-motion sensor. Sci. China Technol. Sci. 60, 374–384 (2017). https://doi.org/10.1007/s11431-016-0434-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11431-016-0434-x

Keywords

Search

Navigation