Abstract
Sign language recognition especially finger language recognition facilitates the life of deaf people in China. It overcomes many difficulties and provides convenience for deaf people’s life. In this paper, we used the advanced convolutional neural network to extract the different characteristics of the input. We created an optimized seven-layer CNN, including five convolution layers for feature extraction and two fully connected layers for classification to enhance the original signal function and reduce noise after operation. Some advanced techniques such as batch normalization, ReLu and dropout were employed to optimize the neural network. Meanwhile, we adopted data augmentation technology, which not only expanded the data set and improve the performance of machine learning algorithm, but also avoided the over-fitting problem. The experimental results show that the average recognition accuracy reaches 91.99 ± 1.21%, which indicate an excellent property.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jiang, X., Satapathy, S.C., Yang, L., Wang, S.-H., Zhang, Y.-D.: A survey on artificial intelligence in Chinese sign language recognition. Arab. J. Sci. Eng. 45(12), 9859–9894 (2020). https://doi.org/10.1007/s13369-020-04758-2
Premaratne, P.: Historical Development of Hand Gesture Recognition. In: Premaratne, P. (ed.) Human Computer Interaction Using Hand Gestures, pp. 5–29. Springer Singapore, Singapore (2014). https://doi.org/10.1007/978-981-4585-69-9_2
Liang, R.H., Ming, O.: A real-time continuous gesture recognition system for sign language. In: IEEE International Conference on Automatic Face & Gesture Recognition (1998)
Wu, J., Wen, G., Cheng, X.: A system recognizing Chinese finger-spelling alphabets based on data-glove input. Pattern Recogn. Artif. Intell. (1999)
Liang, R.-H.: A real-time continuous gesture recognition system for sign language. In: Proceedings of The Third IEEE International Conference on Automatic Face and Gesture Recognition (1998)
Weng, C., Li, Y., Zhang, M., Guo, K., Tang, X., Pan, Z.: Robust Hand Posture Recognition Integrating Multi-cue Hand Tracking. In: Zhang, X., Zhong, S., Pan, Z., Wong, K., Yun, R. (eds.) Edutainment 2010. LNCS, vol. 6249, pp. 497–508. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14533-9_51
Ameen, S., Vadera, S.: A convolutional neural network to classify American sign language fingerspelling from depth and colour images. Expert Syst. 34(3), e12197 (2017). https://doi.org/10.1111/exsy.12197
Mohanty, A., Rambhatla, S., Sahay, R.: Deep Gesture: Static Hand Gesture Recognition Using CNN. In: Raman, B., Kumar, S., Roy, P.P., Sen, D. (eds.) Proceedings of International Conference on Computer Vision and Image Processing, pp. 449–461. Springer Singapore, Singapore (2017). https://doi.org/10.1007/978-981-10-2107-7_41
Sun, J., He, X., Tan, W., Wu, X., Lu, H.: Recognition of crop seedling and weed recognition based on dilated convolution and global pooling in CNN. Trans. Chin. Soc. Agric. Eng. 34(11), 159–165 (2018)
Yu, H., Ding, L., Shi, H., Hanchao, Y., Huang, T.S.: Computed tomography super-resolution using convolutional neural networks. In: IEEE International Conference on Image Processing (ICIP) (2017)
Wang, Shui-Hua., Hong, J., Yang, M.: Sensorineural hearing loss identification via nine-layer convolutional neural network with batch normalization and dropout. Multimedia Tools Appl. 79(21–22), 15135–15150 (2018). https://doi.org/10.1007/s11042-018-6798-3
Banerjee, C., Mukherjee, T., Pasiliao, E.: An empirical study on generalizations of the ReLU activation function. In: The 2019 ACM Southeast Conference (2019)
Wang, Y., Liu, Z., Mu, X., Gao, S.: Modeling and verification of contact line transient temperature difference based on lifting or lowering the pantograph electric contacts. Chin. J. Sci. Instrum. 35(12), 2663–2672 (2014)
Ying-bing, L.: Research on computer technology of remote supervisory and management system. In: Conference and Technology of West China (2010)
Li, D., Deng, L., Cai, Z.: Research on image classification method based on convolutional neural network. Neural Comput. Appl. 33, 8157–8167 (2020). https://doi.org/10.1007/s00521-020-04930-7
Wang, S.-H., Lv, Y.-D., Sui, Y., Liu, S., Wang, S.-J., Zhang, Y.-D.: Alcoholism detection by data augmentation and convolutional neural network with stochastic pooling. J. Med. Syst. 42(1), 1–11 (2017). https://doi.org/10.1007/s10916-017-0845-x
Wei, Z., Yang, J., Min, S.: A method of underwater acoustic signal classification based on deep neural network. In: 2018 5th International Conference on Information Science and Control Engineering (ICISCE) (2019)
Shen, X., Tian, X., Liu, T., Xu, F. Tao, D.: Continuous dropout. IEEE Trans. Neural Netw. Learn. Syst. 1–12 (2017)
Yu, X., Kang, C., Guttery, D., Kadry, S., Chen, Y., Zhang, Yu-Dong.: ResNet-SCDA-50 for breast abnormality classification. IEEE/ACM Trans. Comput. Biol. Bioinform. 18(1), 94–102 (2021). https://doi.org/10.1109/TCBB.2020.2986544
Motamed, S., Rogalla, P., Khalvati, F.: Data Augmentation using Generative Adversarial Networks (GANs) for GAN-based Detection of Pneumonia and COVID-19 in Chest X-ray Images (2020)
Eckert, D., Vesal, S., Ritschl, L., Kappler, S., Maier, A.: Deep Learning-based Denoising of Mammographic Images using Physics-driven Data Augmentation. Presented at the (2020). https://doi.org/10.1007/978-3-658-29267-6_21
Vasconcelos, C.N., Vasconcelos, B.N.: Convolutional Neural Network Committees for Melanoma Classification with Classical and Expert Knowledge Based Image Transforms Data Augmentation (2017)
Singh, P., Yadav, A.K., Singh, K.: Color image encryption using affine transform in fractional Hartley domain. Opt. Appl. 47(3) (2017)
Igl, M., Ciosek, K., Li, Y., Tschiatschek, S., Hofmann, K.: Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck (2019)
Wang, Shui-Hua., et al.: Multiple sclerosis identification by 14-layer convolutional neural network with batch normalization, dropout, and stochastic pooling. Front. Neurosci. 12, 818 (2018)
Jiang, X.: Isolated Chinese sign language recognition using gray-level co-occurrence matrix and parameter-optimized medium Gaussian support vector machine. Front. Intell. Comput.: Theory Appl. 1014, 182–193 (2019)
Ya, G., et al.: Chinese Fingerspelling Recognition via Hu Moment Invariant and RBF Support Vector Machine. In: Zhang, Y.-D., Wang, S.-H., Liu, S. (eds.) ICMTEL 2020. LNICSSITE, vol. 327, pp. 382–392. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-51103-6_34
Jiang, X., Zhang, Y.-D.: Chinese sign language fingerspelling via six-layer convolutional neural network with leaky rectified linear units for therapy and rehabilitation. J. Med. Imag. Health Inform. 9(9), 2031–2090 (2019)
Zhu, Z., Zhang, M., Jiang, X.: Fingerspelling identification for Chinese sign language via wavelet entropy and kernel support vector machine. Intell. Data Eng. Anal. 1177, 539–549 (2020)
Jiang, X., Hu, B., Chandra Satapathy, S., Wang, S.-H., Zhang, Y.-D.: Fingerspelling identification for Chinese sign language via AlexNet-based transfer learning and Adam optimizer. Sci. Program. 2020, 1–10 (2020)
Acknowledgements
This work was supported by National Philosophy and Social Sciences Foundation (20BTQ065), Natural Science Foundation of Jiangsu Higher Education Institutions of China (19KJA310002), The Philosophy and Social Science Research Foundation Project of Universities of Jiangsu Province (2017SJB0668).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Gao, Y., Zhu, R., Gao, R., Weng, Y., Jiang, X. (2021). An Optimized Seven-Layer Convolutional Neural Network with Data Augmentation for Classification of Chinese Fingerspelling Sign Language. In: Fu, W., Xu, Y., Wang, SH., Zhang, Y. (eds) Multimedia Technology and Enhanced Learning. ICMTEL 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 388. Springer, Cham. https://doi.org/10.1007/978-3-030-82565-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-82565-2_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82564-5
Online ISBN: 978-3-030-82565-2
eBook Packages: Computer ScienceComputer Science (R0)