Skip to main content
View expanded cover

International Conference on Multimedia Technology and Enhanced Learning

ICMTEL 2021: Multimedia Technology and Enhanced Learning pp 21–42Cite as

An Optimized Seven-Layer Convolutional Neural Network with Data Augmentation for Classification of Chinese Fingerspelling Sign Language

  • 229 Accesses

Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST,volume 388)

Abstract

Sign language recognition especially finger language recognition facilitates the life of deaf people in China. It overcomes many difficulties and provides convenience for deaf people’s life. In this paper, we used the advanced convolutional neural network to extract the different characteristics of the input. We created an optimized seven-layer CNN, including five convolution layers for feature extraction and two fully connected layers for classification to enhance the original signal function and reduce noise after operation. Some advanced techniques such as batch normalization, ReLu and dropout were employed to optimize the neural network. Meanwhile, we adopted data augmentation technology, which not only expanded the data set and improve the performance of machine learning algorithm, but also avoided the over-fitting problem. The experimental results show that the average recognition accuracy reaches 91.99 ± 1.21%, which indicate an excellent property.

Keywords

  • Convolutional neural network
  • Data augmentation
  • Chinese fingerspelling sing language
  • Batch normalization
  • ReLU
  • Maximum pooling
  • Dropout

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-82565-2_3
  • Chapter length: 22 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-82565-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.
Fig. 14.
Fig. 15.
Fig. 16.
Fig. 17.
Fig. 18.
Fig. 19.
Fig. 20.
Fig. 21.

References

  1. Jiang, X., Satapathy, S.C., Yang, L., Wang, S.-H., Zhang, Y.-D.: A survey on artificial intelligence in Chinese sign language recognition. Arab. J. Sci. Eng. 45(12), 9859–9894 (2020). https://doi.org/10.1007/s13369-020-04758-2

    CrossRef  Google Scholar 

  2. Premaratne, P.: Historical Development of Hand Gesture Recognition. In: Premaratne, P. (ed.) Human Computer Interaction Using Hand Gestures, pp. 5–29. Springer Singapore, Singapore (2014). https://doi.org/10.1007/978-981-4585-69-9_2

    CrossRef  Google Scholar 

  3. Liang, R.H., Ming, O.: A real-time continuous gesture recognition system for sign language. In: IEEE International Conference on Automatic Face & Gesture Recognition (1998)

    Google Scholar 

  4. Wu, J., Wen, G., Cheng, X.: A system recognizing Chinese finger-spelling alphabets based on data-glove input. Pattern Recogn. Artif. Intell. (1999)

    Google Scholar 

  5. Liang, R.-H.: A real-time continuous gesture recognition system for sign language. In: Proceedings of The Third IEEE International Conference on Automatic Face and Gesture Recognition (1998)

    Google Scholar 

  6. Weng, C., Li, Y., Zhang, M., Guo, K., Tang, X., Pan, Z.: Robust Hand Posture Recognition Integrating Multi-cue Hand Tracking. In: Zhang, X., Zhong, S., Pan, Z., Wong, K., Yun, R. (eds.) Edutainment 2010. LNCS, vol. 6249, pp. 497–508. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14533-9_51

    CrossRef  Google Scholar 

  7. Ameen, S., Vadera, S.: A convolutional neural network to classify American sign language fingerspelling from depth and colour images. Expert Syst. 34(3), e12197 (2017). https://doi.org/10.1111/exsy.12197

    CrossRef  Google Scholar 

  8. Mohanty, A., Rambhatla, S., Sahay, R.: Deep Gesture: Static Hand Gesture Recognition Using CNN. In: Raman, B., Kumar, S., Roy, P.P., Sen, D. (eds.) Proceedings of International Conference on Computer Vision and Image Processing, pp. 449–461. Springer Singapore, Singapore (2017). https://doi.org/10.1007/978-981-10-2107-7_41

    CrossRef  Google Scholar 

  9. Sun, J., He, X., Tan, W., Wu, X., Lu, H.: Recognition of crop seedling and weed recognition based on dilated convolution and global pooling in CNN. Trans. Chin. Soc. Agric. Eng. 34(11), 159–165 (2018)

    Google Scholar 

  10. Yu, H., Ding, L., Shi, H., Hanchao, Y., Huang, T.S.: Computed tomography super-resolution using convolutional neural networks. In: IEEE International Conference on Image Processing (ICIP) (2017)

    Google Scholar 

  11. Wang, Shui-Hua., Hong, J., Yang, M.: Sensorineural hearing loss identification via nine-layer convolutional neural network with batch normalization and dropout. Multimedia Tools Appl. 79(21–22), 15135–15150 (2018). https://doi.org/10.1007/s11042-018-6798-3

    CrossRef  Google Scholar 

  12. Banerjee, C., Mukherjee, T., Pasiliao, E.: An empirical study on generalizations of the ReLU activation function. In: The 2019 ACM Southeast Conference (2019)

    Google Scholar 

  13. Wang, Y., Liu, Z., Mu, X., Gao, S.: Modeling and verification of contact line transient temperature difference based on lifting or lowering the pantograph electric contacts. Chin. J. Sci. Instrum. 35(12), 2663–2672 (2014)

    Google Scholar 

  14. Ying-bing, L.: Research on computer technology of remote supervisory and management system. In: Conference and Technology of West China (2010)

    Google Scholar 

  15. Li, D., Deng, L., Cai, Z.: Research on image classification method based on convolutional neural network. Neural Comput. Appl. 33, 8157–8167 (2020). https://doi.org/10.1007/s00521-020-04930-7

    CrossRef  Google Scholar 

  16. Wang, S.-H., Lv, Y.-D., Sui, Y., Liu, S., Wang, S.-J., Zhang, Y.-D.: Alcoholism detection by data augmentation and convolutional neural network with stochastic pooling. J. Med. Syst. 42(1), 1–11 (2017). https://doi.org/10.1007/s10916-017-0845-x

    CrossRef  Google Scholar 

  17. Wei, Z., Yang, J., Min, S.: A method of underwater acoustic signal classification based on deep neural network. In: 2018 5th International Conference on Information Science and Control Engineering (ICISCE) (2019)

    Google Scholar 

  18. Shen, X., Tian, X., Liu, T., Xu, F. Tao, D.: Continuous dropout. IEEE Trans. Neural Netw. Learn. Syst. 1–12 (2017)

    Google Scholar 

  19. Yu, X., Kang, C., Guttery, D., Kadry, S., Chen, Y., Zhang, Yu-Dong.: ResNet-SCDA-50 for breast abnormality classification. IEEE/ACM Trans. Comput. Biol. Bioinform. 18(1), 94–102 (2021). https://doi.org/10.1109/TCBB.2020.2986544

    CrossRef  Google Scholar 

  20. Motamed, S., Rogalla, P., Khalvati, F.: Data Augmentation using Generative Adversarial Networks (GANs) for GAN-based Detection of Pneumonia and COVID-19 in Chest X-ray Images (2020)

    Google Scholar 

  21. Eckert, D., Vesal, S., Ritschl, L., Kappler, S., Maier, A.: Deep Learning-based Denoising of Mammographic Images using Physics-driven Data Augmentation. Presented at the (2020). https://doi.org/10.1007/978-3-658-29267-6_21

    CrossRef  Google Scholar 

  22. Vasconcelos, C.N., Vasconcelos, B.N.: Convolutional Neural Network Committees for Melanoma Classification with Classical and Expert Knowledge Based Image Transforms Data Augmentation (2017)

    Google Scholar 

  23. Singh, P., Yadav, A.K., Singh, K.: Color image encryption using affine transform in fractional Hartley domain. Opt. Appl. 47(3) (2017)

    Google Scholar 

  24. Igl, M., Ciosek, K., Li, Y., Tschiatschek, S., Hofmann, K.: Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck (2019)

    Google Scholar 

  25. Wang, Shui-Hua., et al.: Multiple sclerosis identification by 14-layer convolutional neural network with batch normalization, dropout, and stochastic pooling. Front. Neurosci. 12, 818 (2018)

    CrossRef  Google Scholar 

  26. Jiang, X.: Isolated Chinese sign language recognition using gray-level co-occurrence matrix and parameter-optimized medium Gaussian support vector machine. Front. Intell. Comput.: Theory Appl. 1014, 182–193 (2019)

    Google Scholar 

  27. Ya, G., et al.: Chinese Fingerspelling Recognition via Hu Moment Invariant and RBF Support Vector Machine. In: Zhang, Y.-D., Wang, S.-H., Liu, S. (eds.) ICMTEL 2020. LNICSSITE, vol. 327, pp. 382–392. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-51103-6_34

    CrossRef  Google Scholar 

  28. Jiang, X., Zhang, Y.-D.: Chinese sign language fingerspelling via six-layer convolutional neural network with leaky rectified linear units for therapy and rehabilitation. J. Med. Imag. Health Inform. 9(9), 2031–2090 (2019)

    CrossRef  Google Scholar 

  29. Zhu, Z., Zhang, M., Jiang, X.: Fingerspelling identification for Chinese sign language via wavelet entropy and kernel support vector machine. Intell. Data Eng. Anal. 1177, 539–549 (2020)

    CrossRef  Google Scholar 

  30. Jiang, X., Hu, B., Chandra Satapathy, S., Wang, S.-H., Zhang, Y.-D.: Fingerspelling identification for Chinese sign language via AlexNet-based transfer learning and Adam optimizer. Sci. Program. 2020, 1–10 (2020)

    Google Scholar 

Download references

Acknowledgements

This work was supported by National Philosophy and Social Sciences Foundation (20BTQ065), Natural Science Foundation of Jiangsu Higher Education Institutions of China (19KJA310002), The Philosophy and Social Science Research Foundation Project of Universities of Jiangsu Province (2017SJB0668).

Author information

Authors and Affiliations

  1. Nanjing Normal University of Special Education, Nanjing, 210038, China

    Yalan Gao, Rongxin Zhu, Ruina Gao, Yuxiang Weng & Xianwei Jiang

Authors
  1. Yalan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rongxin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruina Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuxiang Weng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xianwei Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianwei Jiang .

Editor information

Editors and Affiliations

  1. Hunan Normal University, Changsha, China

    Weina Fu

  2. University of Jinan, Jinan, China

    Yuan Xu

  3. University of Leicester, Leicester, UK

    Shui-Hua Wang

  4. University of Leicester, Leicester, UK

    Yudong Zhang

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Gao, Y., Zhu, R., Gao, R., Weng, Y., Jiang, X. (2021). An Optimized Seven-Layer Convolutional Neural Network with Data Augmentation for Classification of Chinese Fingerspelling Sign Language. In: Fu, W., Xu, Y., Wang, SH., Zhang, Y. (eds) Multimedia Technology and Enhanced Learning. ICMTEL 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 388. Springer, Cham. https://doi.org/10.1007/978-3-030-82565-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-82565-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-82564-5

  • Online ISBN: 978-3-030-82565-2

  • eBook Packages: Computer ScienceComputer Science (R0)