Sign Language Recognition Using Kinect

  • Simon Lang
  • Marco Block
  • Raúl Rojas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7267)


An open source framework for general gesture recognition is presented and tested with isolated signs of sign language. Other than common systems for sign language recognition, this framework makes use of Kinect, a depth camera which makes real-time 3D-reconstruction easily applicable. Recognition is done using hidden Markov models with a continuous observation density. The framework also offers an easy way of initializing and training new gestures or signs by performing them several times in front of the camera. First results with a recognition rate of 97% show that depth cameras are well-suited for sign language recognition.


Hide Markov Model Recognition Rate Training Sequence Sign Language Recognition Observation Sequence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kelly, D., McDonald, J., Markham, C.: Recognizing Spatiotemporal Gestures and Movement Epenthesis in Sign Language. In: 13th International Machine Vision and Image Processing Conference (IMVIP 2009). IEEE Computer Society, Washington, DC (2009)Google Scholar
  2. 2.
    Dreuw, P., Rybach, D., Deselaers, T., Zahedi, M., Ney, H.: Speech Recognition Techniques for a Sign Language Recognition System. In: INTERSPEECH 2007, 8th Annual Conference of the International Speech Communication Association (ISCA 2007), pp. 2513–2516 (2007)Google Scholar
  3. 3.
    Li, X., Parizeau, M., Plamondon, R.: Training Hidden Markov Models with Multiple Observations – A combinatorial Method. IEEE Transactions on PAMI PAMI-22(4), 371–377 (2000)Google Scholar
  4. 4.
    Vogler, C., Metaxas, D.: ASL Recognition Based on a Coupling Between HMMs and 3D Motion Analysis. In: Proceedings of the Sixth International Conference on Computer Vision, pp. 363–369. Narosa Publishing House (1998) ISBN: 978-8-17319-221-0Google Scholar
  5. 5.
    Starner, T., Pentland, A.: Real-Time American Sign Language Recognition from Video Using Hidden Markov Models. In: Proceedings of the International Symposium on Computer Vision, ISCV 1995, pp. 265–270. IEEE Publications, U.S (1995) ISBN: 978-0-81867-190-6CrossRefGoogle Scholar
  6. 6.
    Boyes Braem, P.: Einführung in die Gebärdensprache und ihre Erforschung. In: Internationale Arbeiten zur Gebärdensprache und Kommunikation Gehörloser, 1st edn., vol. 11. SIGNUM-Verlag (1990) ISBN: 978-3-92773-110-3Google Scholar
  7. 7.
    Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  8. 8.
    Rahimi, A.: An Erratum for “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition”, website of Ali Rahimi at MIT Media Laboratory,
  9. 9.
    Official Microsoft Xbox website, introduction of Kinect,
  10. 10.
    Countdown to Kinect: 17 Controller-Free Games Launch in November, Microsoft News Center,
  11. 11.
    Kinect Downgraded To Save Money, Can’t Read Sign Language, News at Kotaku,
  12. 12.
    CopyCat and Kinect, overview of the CopyCat Kinect demo on the website of the Center for Accessible Technology in Sign (CATS),
  13. 13.
    Integrating Speech and Hearing Challenge Individuals, YouTube channel of Dr. Natheer Khasawneh,

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Simon Lang
    • 1
  • Marco Block
    • 1
  • Raúl Rojas
    • 1
  1. 1.Institut für Informatik und MathematikFreie Universität BerlinBerlinGermany

Personalised recommendations