A Deep Learning Approach for Hand Posture Recognition from Depth Data

  • Thomas KopinskiEmail author
  • Fabian Sachara
  • Alexander Gepperth
  • Uwe Handmann
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9887)


Given the success of convolutional neural networks (CNNs) during recent years in numerous object recognition tasks, it seems logical to further extend their applicability to the treatment of three-dimensional data such as point clouds provided by depth sensors. To this end, we present an approach exploiting the CNN’s ability of automated feature generation and combine it with a novel 3D feature computation technique, preserving local information contained in the data. Experiments are conducted on a large data set of 600.000 samples of hand postures obtained via ToF (time-of-flight) sensors from 20 different persons, after an extensive parameter search in order to optimize network structure. Generalization performance, measured by a leave-one-person-out scheme, exceeds that of any other method presented for this specific task, bringing the error for some persons down to 1.5 %.


Deep learning Hand posture recognition 3D data 


  1. 1.
    Thrun, S.: Learning occupancy grid maps with forward sensor models. Auton. Robot. 15(2), 111–127 (2003). ISO 690CrossRefGoogle Scholar
  2. 2.
    Maturana, D., Scherer, S.: VoxNet: a 3D convolutional neural network for real-time object recognition. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922–928. IEEE, September 2015Google Scholar
  3. 3.
    Wu, D., Shao, L.: Deep dynamic neural networks for gesture segmentation and recognition. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014 Workshops. LNCS, vol. 8925, pp. 552–571. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  4. 4.
    Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bengio, Y.: Theano: new features and speed improvements. arXiv preprint arXiv:1211.5590 (2012)
  5. 5.
    REHAP, Large-scale data set for Recognition of Hand Postures.
  6. 6.
    Glatt, R.: Deep learning architecture for gesture recognition (2014)Google Scholar
  7. 7.
    Barros, P., Parisi, G. I., Jirak, D., Wermter, S.: Real-time gesture recognition using a humanoid robot with a deep neural architecture. In: 2014 IEEE-RAS International Conference on Humanoid Robots, pp. 646–651. IEEE, November 2014Google Scholar
  8. 8.
    Tang, A., Lu, K., Wang, Y., Huang, J., Li, H.: A real-time hand posture recognition system using deep neural networks. ACM Trans. Intell. Syst. Technol. (TIST) 6(2), 21 (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Thomas Kopinski
    • 1
    • 2
    Email author
  • Fabian Sachara
    • 1
    • 2
  • Alexander Gepperth
    • 1
    • 2
  • Uwe Handmann
    • 1
    • 2
  1. 1.Hochschule Ruhr West, Computer Science InstituteBottropGermany
  2. 2.UIIS Lab and FLOWERS Team, Inria, Université Paris-SaclayPalaiseauFrance

Personalised recommendations