Skip to main content
SpringerLink
Log in

A human–robot interface for mobile manipulator

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

This paper presents a remote manipulation method for mobile manipulator through operator’s gesture. In particular, a track mobile robot is equipped with a 4-DOF robot arm to grasp objects. Operator uses one hand to control both the motion of mobile robot and the posture of robot arm via scheme of gesture polysemy method which is put forward in this paper. A sensor called leap motion (LM), which can obtain the position and posture data of hand, is employed in this system. Two filters were employed to estimate the position and posture of human hand so as to reduce the inherent noise of the sensor. Kalman filter was used to estimate the position, and particle filter was used to estimate the orientation. The advantage of the proposed method is that it is feasible to control a mobile manipulator through just one hand using a LM sensor. The effectiveness of the proposed human–robot interface was verified in laboratory with a series of experiments. And the results indicate that the proposed human–robot interface is able to track the movements of operator’s hand with high accuracy. It is found that the system can be employed by a non-professional operator for robot teleoperation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Maria M, Cristina S, Anselmo F, Ramón C (2014) Real time control of the ASBGo walker through a physical human–robot interface. Measurement 48(2):77–86

    Google Scholar 

  2. João S, Alberto V, Rodrigo V (2015) A multi-purpose rescue vehicle and a human–robot interface architecture for remote assistance in ITER. Fus Eng Des 98–99:1656–1659

    Google Scholar 

  3. Ana CP, Maria M, Miguel AS (2014) Signage system for the navigation of autonomous robots in indoor environments. IEEE Trans Ind Inform 10(1):680–688

    Article  Google Scholar 

  4. Guanglong D, Ping Z (2014) Online serial manipulator calibration based on multisensory process via extended Kalman and Particle filters. IEEE Trans Ind Electron 61(12):6852–6859

    Article  Google Scholar 

  5. Jingying C, Dan C, Xiaoli L, Kun Z (2014) Towards improving social communication skills with multimodal sensory information. IEEE Trans Ind Inform 10(1):323–330

    Article  Google Scholar 

  6. Jian H, Weiguang H, Wenxia X, Samer M, Yacine A (2015) Control of upper-limb power-assist exoskeleton using a human–robot interface based on motion intention recognition. IEEE Trans Autom Sci Eng 12(4):1257–1270

    Article  Google Scholar 

  7. Petr M, Michal R (2014) Speech recognition via STT API for autonomous mobile robot. In: International conference on mechatronics-mechatronika IEEE, pp 594–599

  8. Björn B et al (2014) Active in-hand object recognition on a humanoid robot. IEEE Trans Robot 30(5):1260–1269

    Article  Google Scholar 

  9. Cuong T, Mohan Manubhai T (2012) 3-D posture and gesture recognition for interactivity in smart spaces. IEEE Trans Ind Inform 8(1):178–187

    Article  Google Scholar 

  10. Mohammad K, Wen-Fang X, Abolfazl M (2014) Augmented image-based visual servoing of a manipulator using acceleration command. IEEE Trans Ind Electron 61(10):5444–5452

    Article  Google Scholar 

  11. Guanglong D, Ping Z (2014) Markerless human–robot interface for dual robot manipulators using kinect sensor. Robot Comput Integr Manuf 30(2):150–159

    Article  Google Scholar 

  12. Takahiro N et al (2013) Recognition of grasping motion based on modal space haptic information using DP pattern-matching algorithm. IEEE Trans Ind Inform 9(4):2043–2051

    Article  Google Scholar 

  13. Eshed O, Mohan Manubhai T (2014) Hand gesture recognition in real time for automotive interfaces: a multimodal vision-based approach and evaluations. IEEE Trans Intell Transport Syst 15(6):2368–2377

    Article  Google Scholar 

  14. Guanglong D et al (2014) Human-manipulator interface based on multisensory process via Kalman filters. IEEE Trans Ind Electron 61(10):5411–5418

    Article  Google Scholar 

  15. Ping Z, Xin L, Guanglong D, Bin L, Xueqian W (2015) A markerless human-manipulators interface using multi-sensors. Ind Robot Int J 42(6):544–553

    Article  Google Scholar 

  16. Marion P et al (2017) Director: a user interface designed for robot operation with shared autonomy. J Field Robot 34(2):262–280

    Article  Google Scholar 

  17. Leap Motion. https://www.leapmotion.com. Aug 2014

  18. Frank W et al (2013) Analysis of the accuracy and robustness of the leap motion controller. Sensors 2013(13):6380–6393

    Google Scholar 

  19. Ho Kim M, Suk L, Kyung Chang L (2010) Kalman predictive redundancy system for fault tolerance of safety-critical systems. IEEE Trans Ind Inform 6(1):46–53

    Article  Google Scholar 

  20. Seong-Hoon W, Wael William M, Farid G (2009) A fastening tool trackingsystem using an IMU and a position sensor with Kalman filters and a fuzzy expert system. IEEE Trans Ind Electron 56(5):1782–1792

    Article  Google Scholar 

  21. Yun X, Bachmann ER, McGhee RB (2008) A simplified quaternion-based algorithm for orientation estimation from earth gravity and magnetic field measurements. IEEE Trans Instrum Meas 57(3):638–650

    Article  Google Scholar 

  22. Guanglong D, Ping Z (2014) Human-manipulator interface using hybrid sensors with kalman filters and adaptive multi-space transformation. Measurement 55:413–422

    Article  Google Scholar 

  23. Seong-Hoon Peter W, Wael William M, Farid G (2010) A Kalman/Particle filter-based position and orientation estimation method using a position sensor/inertial measurement unit hybrid system. IEEE Trans Ind Electron 57(5):1787–1798

    Article  Google Scholar 

  24. Seong-Hoon Peter W, Wael William M, Farid GM (2011) Fastening tool tracking system using a Kalman filter and particle filter combination. Meas Sci Technol 22(12):1–12

    Google Scholar 

  25. Antonelli G, Chiaverini S, Fusco G (2003) A new on-line algorithm for inverse kinematics of robot manipulators ensuring path tracking capability under joint limits. IEEE Trans Robot Autom 19(1):162–167

    Article  Google Scholar 

  26. Jinling W (2000) Stochastic modeling for real-time kinematic GPS/GLONASS position. Navigation 46(4):297–305

    Google Scholar 

  27. Weidong D, Jinling W, Chris R (2007) Improving adaptive kalman estimation in GPS/INS integration. J Navig 60(3):517–529

    Article  Google Scholar 

  28. Glas DF, Kanda T, Ishiguro H, Hagita N (2012) Teleoperation of multiple social robots. IEEE Trans Syst Man Cybern A Syst Hum 42(3):530–544

    Article  Google Scholar 

Download references

Acknowledgements

The project was funded by “Guangdong Natural Science Funds for Distinguished Young Scholar (2017A030306015),” “Pearl River S&T Nova Program of Guangzhou (201710010059),” “Guangdong special projects (2016TQ03X824),” “the Fundamental Research Funds for the Central Universities (No: 2017JQ009),” “National Natural Science Foundation of China (Nos: 61403145, 61602182).”

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, South China University of Technology, Guangzhou, People’s Republic of China

    Mingxuan Chen, Caibing Liu & Guanglong Du

Authors
  1. Mingxuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Caibing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guanglong Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guanglong Du.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, M., Liu, C. & Du, G. A human–robot interface for mobile manipulator. Intel Serv Robotics 11, 269–278 (2018). https://doi.org/10.1007/s11370-018-0251-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-018-0251-3

Keywords

Search

Navigation