Advertisement

Evaluating Deep Models for Dynamic Brazilian Sign Language Recognition

  • Lucas Amaral
  • Givanildo L. N. Júnior
  • Tiago Vieira
  • Thales VieiraEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11401)

Abstract

We propose and investigate the use of deep models for dynamic gesture recognition, focusing on the recognition of dynamic signs of the Brazilian Sign Language (Libras) from depth data. We evaluate variants and combinations of convolutional and recurrent neural networks, including LRCNs and 3D CNNs models. Experiments were performed with a novel depth dataset composed of dynamic signs representing letters of the alphabet and common words in Libras. An evaluation of the proposed models reveals that the best performing deep model achieves over 99% accuracy, and greatly outperforms a baseline method.

Keywords

Brazilian Sign Language Dynamic sign language recognition Deep learning 

Notes

Acknowledgements

The authors would like to thank CNPq and FAPEAL for partially financing this research.

References

  1. 1.
    Bastos, I.L., Angelo, M.F., Loula, A.C.: Recognition of static gestures applied to Brazilian sign language (libras). In: SIBGRAPI, pp. 305–312. IEEE (2015)Google Scholar
  2. 2.
    Brito, L.F., Langevin, R.: The sublexical structure of a sign language. Mathematiques Inform. et Sciences Humaines 125, 17–40 (1994)Google Scholar
  3. 3.
    Escobedo Cardenas, E., Camara-Chavez, G.: Fusion of deep learning descriptors for gesture recognition. In: Mendoza, M., Velastín, S. (eds.) CIARP 2017. LNCS, vol. 10657, pp. 212–219. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-75193-1_26CrossRefGoogle Scholar
  4. 4.
    Cardenas, E.J.E., Chávez, G.C.: Finger spelling recognition from depth data using direction cosines and histogram of cumulative magnitudes. In: SIBGRAPI, pp. 173–179. IEEE (2015)Google Scholar
  5. 5.
    Cheok, M.J., Omar, Z., Jaward, M.H.: A review of hand gesture and sign language recognition techniques. Int. J. Mach. Learn. Cybern. 10(1), 131–153 (2019)CrossRefGoogle Scholar
  6. 6.
    Chollet, F., et al.: Keras (2015). https://keras.io
  7. 7.
    Donahue, J., et al.: Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625–2634. IEEE (2015)Google Scholar
  8. 8.
  9. 9.
    Ji, S., Xu, W., Yang, M., Yu, K.: 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013)CrossRefGoogle Scholar
  10. 10.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  11. 11.
    de Souza Pereira Moreira, G., Matuck, G.R., Saotome, O., da Cunha, A.M.: Recognizing the Brazilian signs language alphabet with neural networks over visual 3D data sensor. In: Bazzan, A.L.C., Pichara, K. (eds.) IBERAMIA 2014. LNCS (LNAI), vol. 8864, pp. 637–648. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-12027-0_51CrossRefGoogle Scholar
  12. 12.
    Pizzolato, E.B., dos Santos Anjo, M., Pedroso, G.C.: Automatic recognition of finger spelling for libras based on a two-layer architecture. In: Proceedings of the 2010 ACM Symposium on Applied Computing, pp. 969–973. ACM (2010)Google Scholar
  13. 13.
    de Quadros, R.M., Karnopp, L.B.: Língua brasileira de sinais: estudos linguísticos. Artmed, Porto Alegre (2004)Google Scholar
  14. 14.
    Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 4489–4497. IEEE (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Lucas Amaral
    • 1
  • Givanildo L. N. Júnior
    • 1
  • Tiago Vieira
    • 1
  • Thales Vieira
    • 2
    Email author
  1. 1.Institute of ComputingFederal University of AlagoasMaceióBrazil
  2. 2.Institute of MathematicsFederal University of AlagoasMaceióBrazil

Personalised recommendations