Skip to main content
SpringerLink
Log in

Gestures for Manually Controlling a Helping Hand Robot

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

Helping hand robots have been the focus of a number of studies and have high potential in modern manufacturing processes and for use in daily living. As helping hand robots interact closely with users, it is important to find natural and intuitive user interfaces for interacting with the robots in various situations. This study describes a set of gestures for interacting with and controlling helping hand robots in situations in which users need to manually control the robot but both hands are not available, for example, when users are holding tools or objects in their hands. The gestures are derived from an experimental study that asked participants for gestures suitable for controlling primitive robot motions. The selected gestures can be used to control translation and orientation of an end effector of a helping hand robot when one or both hands are engaged with tasks. As an example for validating the proposed gestures, we implemented a helping hand robot system to perform a soldering task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. 3D visualization tool for ROS. http://wiki.ros.org/rviz. Accessed 15 July 2013

  2. HiveGround ROS Packages. https://github.com/hiveground-ros-package. Accessed July 2013

  3. Kinect SDK. http://www.microsoft.com/en-us/kinectforwindows. Accessed 31 July 2013

  4. MoveIt!. http://moveit.ros.org/wiki/MoveIt!. Accessed 8 July 2013

  5. Spacenavigator. http://www.3dconnexion.com/. Accessed 18 July 2013

  6. Bauer A, Wollherr D, Buss M (2008) Human–robot collaboration: a survey. Hum Robot 05(01):47–66

    Article  Google Scholar 

  7. Bradski G, Kaehler A (2008) Learning OpenCV. O’Reilly Media, Cambridge

    Google Scholar 

  8. Burger B, Ferrane I, Lerasle F, Infantes G (2012) Two-handed gesture recognition and fusion with speech to command a robot. Auton Robot 32(2):129–147

    Article  Google Scholar 

  9. Ende T, Haddadin S, Parusel S, Wusthoff T, Hassenzahl M, Albu-Schaffer A (2011) A human-centered approach to robot gesture based communication within collaborative working processes. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, pp. 3367–3374

  10. Faul F, Erdfelder E, Lang AG, Buchner A (2007) G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 39(2):175–191

    Article  Google Scholar 

  11. Gleeson B, MacLean K, Haddadi A, Croft E, Alcazar J (2013) Gestures for industry: intuitive human–robot communication from human observation. In: Proceedings of the 8th ACM/IEEE international conference on human–robot interaction, pp. 349–356

  12. Goodrich MA, Schultz AC (2007) Human–robot interaction: a survey. Found Trends Human–Comput Interact 1:203–275

    Article  MATH  Google Scholar 

  13. Gossow D, Leeper A, Hershberger D, Ciocarlie M (2011) Interactive markers: 3-D user interfaces for ROS applications [ROS topics]. IEEE Robot Autom Mag 18(4):14–15

    Article  Google Scholar 

  14. Groothuis S, Stramigioli S, Carloni R (2013) Lending a helping hand: toward novel assistive robotic arms. IEEE Robot Autom Mag 20(1):20–29

    Article  Google Scholar 

  15. Hammel J, Hall K, Lees D, Leifer L, der Loos MV, Perkash I, Crigler R (1989) Clinical evaluation of a desktop robotic assistant. Rehabil Res Dev 26:1–16

    Google Scholar 

  16. Hato Y, Satake S, Kanda T, Imai M, Hagita N (2010) Pointing to space: modeling of deictic interaction referring to regions. In: Proceedings of the 5th ACM/IEEE international conference on human–robot interaction, pp. 301–308

  17. Iossifidis I, Theis C, Grote C, Faubel C, Schoner G (2003) Anthropomorphism as a pervasive design concept for a robotic assistant. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, vol. 3, pp. 3465–3472

  18. Kuniyoshi Y, Inaba M, Inoue H (1994) Learning by watching: extracting reusable task knowledge from visual observation of human performance. IEEE Trans Robot Autom 10(6):799–822

    Article  Google Scholar 

  19. Neto P, Pires J, Moreira A (2009) Accelerometer-based control of an industrial robotic arm. In: Proceedings of IEEE international symposium on robot and human interactive communication, pp. 1192–1197

  20. Nof SY (ed) (1999) Handbook of industrial robotics, 2nd edn. Wiley, Chicago

    Google Scholar 

  21. Oikonomidis I, Kyriazis N, Argyros A (2012) Tracking the articulated motion of two strongly interacting hands. In: Proceedings of IEEE international conference on computer vision and pattern recognition, pp. 1862–1869

  22. Oviatt S (1999) Ten myths of multimodal interaction. Commun ACM 42(11):74–81

    Article  Google Scholar 

  23. Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) ROS: an open-source robot operating system. In: ICRA workshop on open source software

  24. Rogalla O, Ehrenmann M, Zollner R, Becher R, Dillmann R (2002) Using gesture and speech control for commanding a robot assistant. In: Proceeding of IEEE international workshop on robot and human interactive communication, pp. 454–459

  25. Rusu RB, Cousins S (2011) 3D is here: point cloud library (PCL). In: Proceedings IEEE international conference robotics and automation, pp. 1–4

  26. Stiefelhagen R, Fugen C, Gieselmann R, Holzapfel H, Nickel K, Waibel A (2004) Natural human–robot interaction using speech, head pose and gestures. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, vol. 3, pp. 2422–2427

  27. Tran N, Phan H, Dinh V, Ellen J, Berg B, Lum J, Alcantara E, Bruch M, Ceruti M, Kao C, Garcia D, Fugate S, Duffy L (2009) Wireless data glove for gesture-based robotic control. In: Jacko J (ed) Human–computer interaction. Novel interaction methods and techniques, lecture notes in computer science, vol. 5611. Springer, Berlin, Heidelberg, pp. 271–280

  28. Wachs J, Stern H, Edan Y (2005) Cluster labeling and parameter estimation for the automated setup of a hand-gesture recognition system. IEEE Trans Syst Man Cybern Part A 35(6):932–944

    Article  Google Scholar 

  29. Wachs JP, Kölsch M, Stern H, Edan Y (2011) Vision-based hand-gesture applications. Commun ACM 54:60–71

    Article  Google Scholar 

  30. Wallhoff F, Blume J, Bannat A, Rsel W, Lenz C, Knoll A (2010) A skill-based approach towards hybrid assembly. Adv Eng Inf 24(3):329–339

    Article  Google Scholar 

  31. Wang R, Paris S, Popović J (2011) 6D hands: markerless hand-tracking for computer aided design. In: Proceedings of the \(24{{\rm th}}\) annual ACM symposium on user interface software and technology, UIST ’11. ACM, pp. 549–558

  32. Welch G, Bishop G (2006) An introduction to the Kalman filter. Tech Rep, Department of Computer Science University of North Carolina at Chapel Hill Chapel Hill

  33. Wobbrock JO, Morris MR, Wilson AD (2009) User-defined gestures for surface computing. In: Proceedings of the ACM international conference on human factors in computing systems, pp. 1083–1092

  34. Wongphati M, Kanai Y, Osawa H, Imai M (2012) Give me a hand how users ask a robotic arm for help with gestures. In: Proceedings of IEEE international conference on cyber technology in automation, control, and intelligent systems, pp. 64–68

Download references

Author information

Authors and Affiliations

  1. Graduate School of Science and Technology, Keio University, Yokohama, Japan

    Mahisorn Wongphati

  2. Faculty of Intelligent Interaction Technologies, Tsukuba University, Tsukuba, Japan

    Hirotaka Osawa

  3. Faculty of Science and Technology, Keio University, Yokohama, Japan

    Michita Imai

Authors
  1. Mahisorn Wongphati
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hirotaka Osawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michita Imai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mahisorn Wongphati.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wongphati, M., Osawa, H. & Imai, M. Gestures for Manually Controlling a Helping Hand Robot. Int J of Soc Robotics 7, 731–742 (2015). https://doi.org/10.1007/s12369-015-0302-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-015-0302-2

Keywords

Search

Navigation