Abstract
As the media of communication for people with hearing and speech disabilities, the importance to bridge the communication gap between them and normal people using sign language has become significance. This research proposed a model for the development of sign language recognition technology using Microsoft Kinect and convolutional neural network (CNNs). The proposed model succeeds in recognizing 10 dynamic Indonesian sign language words on complex background. There are total of 100 gesture image sequences containing color and depth data, perform by different users. The classifier consists of two CNNs and one ANN. The first CNN is to extract hand feature from color data, while the other is to extract hand feature from depth data. The training consists of three modes by applying drop-out and data augmentation and achieves the highest validation rate on 81.60 % and test result on 73.00 %.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
World Health Organization.: Media Centre Fact Sheets on Deafness and Hearing Loss (2015)
Kyatanavar, R.D., Futane, P.R.: Comparative study of sign language recognition systems. Int. J. Sci. Res. Publ. 2(16) (2012)
Cooper, H., Ong, E.J., Pugeault, N., Bowden, R.: Sign language recognition using sub-units. J. Mach. Learn. Res. 13(1), 2205–2231 (2012)
Memis, A., Albaryak. S.: A kinect based sign language recognition system using spatio-temporal features. In: Proceedings of SPIE (The International Society for Optical Engineering) (2013)
Pigou, L., Dieleman, S., Kindermans, P., Schrauwen, B.: Sign language recognition using convolutional neural networks. Lect. Notes Comput. Sci. 8925, 572–578 (2015)
Molchanov, P., Gupta, S., Kim, K., Kautz. J.: Hand gesture recognition with 3D convolutional neural network. IEEE Comput. Vis. Pattern Recognit. (CVPR) (2015)
Vedaldi, A., Lenc, K.: MatConvNet—Convolutional Neural Networks for MATLAB. CoRR, vol. abs/1412.4564 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Science+Business Media Singapore
About this paper
Cite this paper
Sugianto, N., Yuwono, E.I. (2016). Indonesian Dynamic Sign Language Recognition at Complex Background with 2D Convolutional Neural Networks. In: Pasila, F., Tanoto, Y., Lim, R., Santoso, M., Pah, N. (eds) Proceedings of Second International Conference on Electrical Systems, Technology and Information 2015 (ICESTI 2015). Lecture Notes in Electrical Engineering, vol 365. Springer, Singapore. https://doi.org/10.1007/978-981-287-988-2_9
Download citation
DOI: https://doi.org/10.1007/978-981-287-988-2_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-287-986-8
Online ISBN: 978-981-287-988-2
eBook Packages: EngineeringEngineering (R0)