Using Convolutional Neural Networks to Distinguish Different Sign Language Alphanumerics
Using Convolutional Neural Networks (CNN)’s to create Deep Learning systems that turns Sign Language into text has been a vital tool in breaking communication barriers between deaf-mute people. Conventional research on this subject concerns training networks to recognize alphanumerical gestures and produce their textual equivalents.
A problem with current methods is that images are scarce, with little variation in available gestures, often skewed towards skin tones and hand sizes that makes a significant subset of gestures hard to detect. Current identification programs are only trained in a single language despite there being over two-hundred known variants so far. This presents a limitation for traditional exploitation for the state of current technologies such as CNN’s, due to their large number of required parameters.
This work presents a technology that aims to resolve this issue by combining a pretrained legacy AI system for a generic object recognition task with a corrector method to uptrain the legacy network. As a result, a program is created that can receive finger spelling from multiple tactile languages and deduct the corresponding alphanumeric and its language which no other neural network has been able to replicate.
KeywordsConvolutional Neural Networks Sign language Deep Learning Legacy AI
- 1.World Health Organization: Deafness and hearing loss. http://www.who.int/mediacentre/factsheets/fs300/en/
- 4.British Deaf Association. https://bda.org.uk/help-resources/
- 7.Mohandes, M., Aliyu, S., Deriche, M.: Prototype Arabic Sign language recognition using multi-sensor data fusion of two leap motion controllers. In: 2015 IEEE 12th International Multi-conference on Systems, Signals & Devices (SSD15), Mahdia, pp. 1–6 (2015)Google Scholar
- 8.Abhishek, K.S., Qubeley, L.C.F., Ho, D.: Glove-based hand gesture recognition sign language translator using capacitive touch sensor. In: 2016 IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC). IEEE (2016)Google Scholar
- 9.Yang, H.-D.: Sign language recognition with the Kinect sensor based on conditional random fields. Sensors (2014)Google Scholar
- 12.Efficiency of Shallow Cascades for Improving Deep Learning AI Systems. https://doi.org/10.1109/IJCNN.2018.8489266
- 13.[cs.CV] arXiv:1512.00567
- 17.[cs.CV] arXiv:1409.4842
- 18.[cs.CV] arXiv:1805.06618
- 19.Tyukin, I.Y., Gorban, A.N., Green, S., Prokhorov, D.: Fast construction of correcting ensembles for legacy artificial intelligence systems: algorithms and a case studyGoogle Scholar