Efficient Generation of Large Amounts of Training Data for Sign Language Recognition: A Semi-automatic Tool

  • Ruiduo Yang
  • Sudeep Sarkar
  • Barbara Loeding
  • Arthur Karshmer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4061)


We have developed a video hand segmentation tool which can help with generating hands ground truth from sign language image sequences. This tool may greatly facilitate research in the area of sign language recognition. In this tool, we offer a semi automatic scheme to assist with the localization of hand pixels, which is important for the purpose of recognition. A candidate hand generator is applied by using the mean shift image segmentation algorithm and a greedy seeds growing algorithm. After a number of hand candidates is generated, the user can reduce the candidates by simple mouse clicks. The tool also provides a hand tracking function for faster processing and a face detection function for groundtruthing non manual signals. In addition, we provided a two-passes groundtruthing scheme unlike other tools that only does one-pass. Our first pass processing is automatic and does not need user interaction. The experiment results demonstrate that based on the first pass’s result, one can groundtruth 10,000+ frames of sign language within 8 hours.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Loeding, B.L., Sarkar, S., Parashar, A., Karshmer, A.I.: Progress in Automated Computer Recognition of Sign Language. In: Miesenberger, K., Klaus, J., Zagler, W., Burger, D. (eds.) ICCHP 2004. LNCS, vol. 3118, pp. 1079–1087. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Starner, T., Weaver, J., Pentland, A.P.: Real-time American Sign Language Recognition using Desk and Wearable Computer Based Video. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 1371–1375 (1998)CrossRefGoogle Scholar
  3. 3.
    Tsap, L.V., Shin, M.C.: Dynamic Disparity Adjustment and Histogram-Based Filtering of Range Data for Fast 3-D Hand Tracking. Journal of Digital Signal Processing 14, 550–565 (2004)CrossRefGoogle Scholar
  4. 4.
    Just, A., Marcel, S.: Two-Handed Gesture Recognition, IDIAP Research Institute CH-1920, Martigny, Switzerland (2005)Google Scholar
  5. 5.
    Yang, M.H., Ahuja, N., Tabb, M.: Extraction of 2D Motion Trajectories and its Application to Hand Gesture Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 168–185 (2002)Google Scholar
  6. 6.
    Volkmer, T., Smith, J.R., Natsev, A.: A Web-based System for Collaborative Annotation of Large Image and Video Collections: An Evaluation and User Study. In: MULTIMEDIA 2005. Proceedings of the 13th annual ACM international conference on Multimedia, pp. 892–901 (2005)Google Scholar
  7. 7.
    Ahn, L.V., Dabbish, L.: Labeling Images with a Computer Game. In: Conference on Human Factors in Computing Systems (CHI), pp. 319–326 (2004)Google Scholar
  8. 8.
    Pfund, T., Maillet, S.M.: A Dynamic Multimedia Annotation tool. In: Beretta, G., Schettini, R. (eds.) Proceedings of SPIE Photonics West, Electronic Imaging 2002, Internet Imaging III, vol. 4672, pp. 216–224 (2002)Google Scholar
  9. 9.
    Doermann, D., Mihalcik, D.: Tools and Techniques for Video Performance Evaluation. In: Proceedings of the IEEE International Conference on Pattern Recognistion (ICPR 2000), vol. 4, pp. 167–170 (2000)Google Scholar
  10. 10.
    Marcotegui, B., Correia, P.: A Video Object Generation Tool Allowing Friendly User Interaction. In: International Conference on Image Processing, vol. 4, pp. 391–395 (1999)Google Scholar
  11. 11.
    Comanicu, D., Meer, P.: Mean shift: A Robust Approach Toward Feature Space Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 603–619 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ruiduo Yang
    • 1
  • Sudeep Sarkar
    • 1
  • Barbara Loeding
    • 2
  • Arthur Karshmer
    • 2
  1. 1.University of South FloridaTampaUSA
  2. 2.University of South FloridaLakelandUSA

Personalised recommendations