An Open Source Framework for Real-Time, Incremental, Static and Dynamic Hand Gesture Learning and Recognition

  • Todd C. Alexander
  • Hassan S. Ahmed
  • Georgios C. Anagnostopoulos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5611)


Real-time, static and dynamic hand gesture learning and recognition makes it possible to have computers recognize hand gestures naturally. This creates endless possibilities in the way humans can interact with computers, allowing a human hand to be a peripheral by itself. The software framework developed provides a lightweight, robust, and practical application programming interface that helps further research in the area of human-computer interaction. Approaches that have proven in analogous areas such as speech and handwriting recognition were applied to static and dynamic hand gestures. A semi-supervised Fuzzy ARTMAP neural network was used for incremental online learning and recognition of static gestures; and, Hidden Markov models for online recognition of dynamic gestures. A simple anticipatory method was implemented for determining when to update key frames allowing the framework to work with dynamic backgrounds.


Motion detection hand tracking real-time gesture recognition software framework FAST corner detection ART Neural Networks 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vogler, C., Metaxas, D.: A framework for recognizing the simultaneous aspects of american sign language. Computer Vision and Image Understanding 81, 358–384 (2001)CrossRefzbMATHGoogle Scholar
  2. 2.
    Fang, G., Gao, W., Ma, J.: Signer-independent sign language recognition based on SOFM/HMM, Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, 2001. In: Proceedings of the IEEE ICCV Workshop on RATFG in RTS, p. 90 (2001)Google Scholar
  3. 3.
    von Hardenberg, C., Berard, F.: Bare-hand human-computer interaction. In: Proceedings of the 2001 workshop on Perceptive user interfaces, pp. 1–8 (2001)Google Scholar
  4. 4.
    Rainer, J.M.: Fast hand gesture recognition for real-time teleconferencing applications. In: International Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-time Systems, pp. 133–140 (2001)Google Scholar
  5. 5.
    McAllister, G., McKenna, S.J., Ricketts, I.W.: Hand tracking for behaviour understanding. Image Vision Computing 20, 827–840 (2002)CrossRefGoogle Scholar
  6. 6.
    Keskin, C., Erkan, A., Akarun, L.: Real time hand tracking and 3d gesture recognition for interactive interfaces using hmm. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714. Springer, Heidelberg (2003)Google Scholar
  7. 7.
    Breuer, P., Eckes, C., Muller, S.: Hand Gesture Recognition with a Novel IR Time-of-Flight Range Camera—A Pilot Study. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2007. LNCS, vol. 4418, pp. 247–260. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Lyons, K., Brashear, H., Westeyn, T., Kim, J., Starner, T.: GART: The gesture and activity recognition toolkit. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 718–727. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  9. 9.
    Westeyn, T., Brashear, H., Atrash, A., Starner, T.: Georgia tech gesture toolkit: supporting experiments in gesture recognition. In: Proceedings of the 5th international Conference on Multimodal interfaces. ICMI 2003, pp. 85–92. ACM, New York (2003)Google Scholar
  10. 10.
    Hejlsberg, A., Wiltamuth, S., Golde, P.: C# Language Specifcation. Addison-Wesley Longman Publishing Co., Inc., Boston (2003)Google Scholar
  11. 11.
  12. 12. Library,
  13. 13.
    Alkaabi, S., Deravi, F.: Candidate pruning for fast corner detection. Electronics Letters 40, 18–19 (2004)CrossRefGoogle Scholar
  14. 14.
    Angel, E., Morrison, D.: Speeding up bresenham’s algorithm. IEEE Computer Graphics and Applications 11, 16–17 (1991)CrossRefGoogle Scholar
  15. 15.
    Jerde, T., Soechting, J., Flanders, M.: Biological constraints simplify the recognition of hand shapes. IEEE Transactions on Biomedical Engineering 50, 265–269 (2003)CrossRefGoogle Scholar
  16. 16.
    Grossberg, S.: Adaptive pattern classification and universal recoding, II: feedback, expectation, olfaction and illusions. Biological Cybernetics 23, 187–202 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Anagnostopoulos, G., Georgiopoulos, M., Verzi, S., Heileman, G.: Reducing generalization error and category proliferation in ellipsoid ARTMAP via tunable misclassification error tolerance: Boosted ellipsoid ARTMAP. In: Proc. IEEE-INNS-ENNS Int’l Joint Conf. Neural Networks (IJCNN 2002), vol. 3, pp. 2650–2655 (2002)Google Scholar
  18. 18.
    Anagnostopoulos, G., Bharadwaj, M., Georgiopoulos, M., Verzi, S., Heileman, G.: Exemplar-based pattern recognition via semi-supervised learning. In: Proceedings of the International Joint Conference on Neural Networks, vol. 4, pp. 2782–2787 (2003)Google Scholar
  19. 19.
    Makhoul, J., Starner, T., Schwartz, R., Chou, G.: On-line cursive handwriting recognition using hidden Markov models and statistical grammars. In: Proceedings of the Workshop on Human Language Technology, Human Language Technology Conference, Plainsboro, NJ, March 08 - 11, 1994, pp. 432–436. Association for Computational Linguistics, Morristown (1994)Google Scholar
  20. 20.
    Zen, H., Tokuda, K., Kitamura, T.: A viterbi algorithm for trajectory model derived from HMM with explicity relationship between static and dynamic features. In: Proceedings of ICASSP 2004, pp. 837–840 (2004)Google Scholar
  21. 21.
    Cavalin, P., Sabourin, R., Suen, C., Britto, A.: Evaluation of incremental learning algorithms for HMM in the recognition of alphanumeric characters. Pattern Recognition (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Todd C. Alexander
    • 1
  • Hassan S. Ahmed
    • 2
  • Georgios C. Anagnostopoulos
    • 1
  1. 1.Electrical and Computer EngineeringFlorida Institute of TechnologyMelbourneUSA
  2. 2.Electrical EngineeringUniversity of MiamiMiamiUSA

Personalised recommendations