Gesture-Based Robot Programming Using Microsoft Kinect

  • Sebastian BlankemeyerEmail author
  • Joshua Göke
  • Tobias Grimm
  • Benedikt Meier
  • Annika Raatz
Conference paper
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 530)


Companies have to face constantly changing framework conditions. Product lifecycles are reduced and the number of variants increase as a result of increasing product individualization. To ensure that production systems such as robots can achieve this flexibility, more and more programming procedures are being developed that do not require a high level of qualification. This thesis deals with an intuitive approach that can be used for programming adhesive joints. With the help of the simple pointing to surfaces, a path corresponding to the intersection line of the two planes is calculated. The required algorithm for fingertip recognition is implemented with the help of Voronoi diagrams. The recognition of the planes is based on a region growing algorithm. The accuracy of the whole process is limited by the current technical state of the art and does not yet meet the necessary requirements.


Robot programming Intuitive human machine interface Gesture recognition 


  1. 1.
    Akgun, B., et al.: Trajectories for kinesthetic teaching: a human-robot interaction perspective. In: Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction, vol. 12, pp. 391–398 (2012)Google Scholar
  2. 2.
    Lun, R., Zhao, W.: A survey of applications and human motion recognition with microsoft kinect. Int. J. Pattern Recogn. Artif. Intell. 29(5), 1555008 (2015)CrossRefGoogle Scholar
  3. 3.
    Yang, L., et al.: Evaluating and improving the depth accuracy of kinect for windows v2. IEEE Sens. J. 15(8), 4275–4285 (2015)CrossRefGoogle Scholar
  4. 4.
    Windows Dev Center. Accessed 08 Nov 2017
  5. 5.
    Landa-Huartado, L., et al.: Kinect-based trajectory teaching for industrial robots. In: Pan-American Congress of Applied Mechanics, vol. 14 (2016)Google Scholar
  6. 6.
    Canal, G.: A real-time human-robot interaction system based on gestures for assistive scenarios. Comput. Vis. Image Underst. 149, 65–77 (2016)CrossRefGoogle Scholar
  7. 7.
    Taylor, J., et al.: Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences. ACM Trans. Graph. (TOG) 35(4), 143 (2016)CrossRefGoogle Scholar
  8. 8.
    Yeo, H., et al.: Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware. Multimed. Tools Appl. 74(8), 2687–2715 (2015)CrossRefGoogle Scholar
  9. 9.
    Gesing, S.: Approximation von Punktmengen durch Kreise (2005)Google Scholar
  10. 10.
    Holz, D., Behnke, S.: Fast range image segmentation and smoothing using approximate surface reconstruction and region growing. Intell. Auton. Syst. 12, 61–73 (2012)Google Scholar
  11. 11.
    Stueckler, J., et al.: Real-time 3D perception and efficient grasp planning for everyday manipulation tasks. In: European Conference on Mobile Robots, vol. 5 (2011)Google Scholar
  12. 12.
    Zhang, L., et al.: Fast plane segmentation with line primitives for RGB-D sensor. Int. J. Adv. Robot. Syst. 6, 1–8 (2016)CrossRefGoogle Scholar
  13. 13.
    Vosselmann, G., et al.: Recognising structure in laser scanner point clouds. Inf. Sci. 46(8), 1–6 (2004)Google Scholar
  14. 14.
    Borrmann, D., et al.: The 3D Hough transform for plane detection in point clouds: a review and a new accumulator design. 3D. Research 2, 1–13 (2011)Google Scholar
  15. 15.
    Lachat, E., et al.: First experiences with kinect V2 sensor for close range 3D modelling. In: International Workshop 3D-ARCH, vol. 6, pp. 93–100 (2015)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2019

Authors and Affiliations

  • Sebastian Blankemeyer
    • 1
    Email author
  • Joshua Göke
    • 1
  • Tobias Grimm
    • 1
  • Benedikt Meier
    • 1
  • Annika Raatz
    • 1
  1. 1.Institute of Assembly TechnologyLeibniz Universität HannoverGarbsenGermany

Personalised recommendations