Advertisement

Recognition of Finger Spelling of American Sign Language with Artificial Neural Network Using Position/Orientation Sensors and Data Glove

  • Cemil Oz
  • Ming C. Leu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3497)

Abstract

An American Sign Language (ASL) finger spelling and an alphabet gesture recognition system was designed with ANN and constructed in order to translate the ASL alphabet into the corresponding printed and sounded English letters. The system uses a sensory Cyberglove and a Flock of Birds 3-D motion tracker to extract the gestures. The finger joint angle data obtained from strain gauges in the sensory glove define the hand shape while the data from the tracker describes the trajectory and orientation. The data flow from these devices is controlled by a motion trigger. Then, data is processed by an alphabet recognition network to generate the words and names. Our goal is to establish an ASL finger spelling system using these devices in real time. We trained and tested our system for ASL alphabet, names and word spelling. Our test results show that the accuracy of recognition is 96%.

Keywords

Artificial Neural Network Hide Markov Model Motion Tracker American Sign Language Deaf People 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    National Center of Birth Defects and Developmental Disabilities home page, http://www.cdc.gov/ncbddd/bd/bdres.htm
  2. 2.
    Chapin, W., Kramer, J., Haas, C., Leifer, L., Macken, E.: TeleSign: A Sign Language Telecomm-unication System. IEEE, Los Alamitos (1992)Google Scholar
  3. 3.
    Wysoski, S.G., Lamar, M.V., Kuroyanagi, S., Iwata, A.: A Rotation Invariant Approach on Static Gesture Recognition Using Boundary Histograms and Neural Networks. In: Proceeding of the 9th International Conference on Neural Information Processing, ICONIP 2002, vol. 4, pp. 2137–2141 (2002)Google Scholar
  4. 4.
    Vogler, C., Metaxas, D.: ASL Recognition Based on a Coupling Between HMMs and 3D Motion Analysis. In: Proceedings of Sixth IEEE International Computer Vision Conference, pp. 363–369 (1998)Google Scholar
  5. 5.
    Lalit, G., Suei, M.: Gesture-based Interaction and Communication: Automated Classification of Hand Gesture Contours. IEEE Transactions on Systems, Man, and Cybernetics part c: Application and Reviews 31, 114–120 (2001)CrossRefGoogle Scholar
  6. 6.
    Yang, M.H., Ahuja, N.: Recognizing Hand Gestures Using Motion Trajectories, pp. 466–472. IEEE, Los Alamitos (1999)Google Scholar
  7. 7.
    Stanner, T., Pentland, A.: Real-time American Sign Language Recognition from Video Using Hidden Markov Models. In: Proceeding of Computer Vision International Symposium, pp. 265–270 (1995)Google Scholar
  8. 8.
    Vogler, C., Metaxas, D.: Adapting Hidden Markov Models for ASL Recognition by Using Three-dimensional Computer Vision Methods. In: Proceeding of the IEEE International Conference on Systems, Man, Cybernetics, Orlando, pp. 12–15 (1997)Google Scholar
  9. 9.
    Cui, Y., Weng, J.: A Learning-based Prediction and Verification Segmentation Scheme for Hand Sign Image Sequence. IEEE Transactions on Pattern Analysis and Machine Intelligence 21, 798–804 (1999)CrossRefGoogle Scholar
  10. 10.
    Vogler, C., Metaxas, D.: Parallel Hidden Markov Models for American Sign Language Recognition. In: IEEE Proceeding of International Computer Vision Conference, pp. 116–122 (1999)Google Scholar
  11. 11.
    Allen, M.J., Asselin, P.K., Foulds, R.: American Sign Language Finger Spelling Recognition System. In: 29th Bioengineering Conference Proceeding, pp. 22–23. IEEE, Los Alamitos (2003)Google Scholar
  12. 12.
    De Marco, R.M., Foulds, R.A.: Data Recording and Analysis of American Sign Language. In: IEEE 29th Proceeding of Bioengineering Conference, pp. 49–50 (2003)Google Scholar
  13. 13.
    Wang, H.G., Sarawate, N.N., Leu, M.C.: Recognition of American Sign Language Gestures with a Sensory Glove. In: Japan USA Symposium on Flexible Automation, Denver, CO (2004)Google Scholar
  14. 14.
    Lee, L.K., Kim, S., Choi, Y.K., Lee, M.H.: Recognition of Hand Gesture to Humancomputer Interaction, pp. 2177–2122. IEEE, Los Alamitos (2000)Google Scholar
  15. 15.
    Sternberg, M.L.A.: American Sign Language Dictionary (Revised Edition). Harper Perennial (1994) ISBN: 0-06-273275-7 Google Scholar
  16. 16.
    Sturman, D.J., Zelter, D.: A Survey of Glove-based Input. IEEE Computer and Applications, 30–39 (1994)Google Scholar
  17. 17.
    Lippman, R.P.: An Introduction to Computing with Neural Nets. IEEE ASSP, 2–22 (1987)Google Scholar
  18. 18.
    Abulafya, N.: Neural Networks for System Identification and Control. MSc Thesis, University of London (1995)Google Scholar
  19. 19.
    Narendra, K.S.: Adaptive Control Using Neural Networks. In: Miller, W.T., Sutton, R.S., Werbos, P.J. (eds.) Neural Networks for Control, MIT, Cambridge (1992) (3rd printing)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Cemil Oz
    • 1
  • Ming C. Leu
    • 1
  1. 1.Department of Mechanical and Aerospace EngineeringUniversity of Missouri-RollaRollaUSA

Personalised recommendations