Computers Helping People with Special Needs

Volume 4061 of the series Lecture Notes in Computer Science pp 635-642

Efficient Generation of Large Amounts of Training Data for Sign Language Recognition: A Semi-automatic Tool

  • Ruiduo YangAffiliated withUniversity of South Florida
  • , Sudeep SarkarAffiliated withUniversity of South Florida
  • , Barbara LoedingAffiliated withUniversity of South Florida
  • , Arthur KarshmerAffiliated withUniversity of South Florida

* Final gross prices may vary according to local VAT.

Get Access


We have developed a video hand segmentation tool which can help with generating hands ground truth from sign language image sequences. This tool may greatly facilitate research in the area of sign language recognition. In this tool, we offer a semi automatic scheme to assist with the localization of hand pixels, which is important for the purpose of recognition. A candidate hand generator is applied by using the mean shift image segmentation algorithm and a greedy seeds growing algorithm. After a number of hand candidates is generated, the user can reduce the candidates by simple mouse clicks. The tool also provides a hand tracking function for faster processing and a face detection function for groundtruthing non manual signals. In addition, we provided a two-passes groundtruthing scheme unlike other tools that only does one-pass. Our first pass processing is automatic and does not need user interaction. The experiment results demonstrate that based on the first pass’s result, one can groundtruth 10,000+ frames of sign language within 8 hours.