Sign Languages Recognition Based on Neural Network Architecture

  • Manuele PalmeriEmail author
  • Filippo Vella
  • Ignazio Infantino
  • Salvatore Gaglio
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 76)


In the last years, many steps forward have been made in speech and natural languages recognition and were developed many virtual assistants such as Apple’s Siri, Google Now and Microsoft Cortana. Unfortunately, not everyone can use voice to communicate to other people and digital devices. Our system is a first step for extending the possibility of using virtual assistants to speech impaired people by providing an artificial sign languages recognition based on neural network architecture.


Sign languages ASL American sign language Sign recognition Kinect Recurrent neural network RNN Deep learning 


  1. 1.
    Bellugi, U., Fischer, S.: A comparison of sign language and spoken language. Cognition 1(2), 173–200 (1972)CrossRefGoogle Scholar
  2. 2.
    Pigou, L., Dieleman, S., Kindermans, P.-J., Schrauwen, B.: Sign language recognition using convolutional neural networks. In: Workshop at the European Conference on Computer Vision, pp. 572–578. Springer (2014)Google Scholar
  3. 3.
    Oz, C., Leu, M.C.: American sign language word recognition with a sensory glove using artificial neural networks. Eng. Appl. Artif. Intell. 24(7), 1204–1213 (2011)CrossRefGoogle Scholar
  4. 4.
    Cooper, H., Ong, E.-J., Pugeault, N., Bowden, R.: Sign language recognition using sub-units. J. Mach. Learn. Res. 13, 2205–2231 (2012)zbMATHGoogle Scholar
  5. 5.
    Gattupalli, S., Ghaderi, A., Athitsos, V.: Evaluation of deep learning based pose estimation for sign language recognition. In: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, p. 12. ACM (2016)Google Scholar
  6. 6.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Cogn. Model. 5(3), 1 (1988)Google Scholar
  7. 7.
    Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 855–868 (2009)CrossRefGoogle Scholar
  8. 8.
    Sak, H., Senior, A.W., Beaufays, F.: Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In: Interspeech, pp. 338–342 (2014)Google Scholar
  9. 9.
    Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)CrossRefGoogle Scholar
  10. 10.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  11. 11.
    Park, Y., Moon, S., Suh, I.H.: Tracking human-like natural motion using deep recurrent neural networks (2016). arXiv preprint arXiv:1604.04528
  12. 12.
    Vella, F., Infantino, I., Scardino, G.: Person identification through entropy oriented mean shift clustering of human gaze patterns. Multimed. Tools Appl. 76(2), 2289–2313 (2017)Google Scholar
  13. 13.
    Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Zeiler, M.D., Ranzato, M., Monga, R., Mao, M., Yang, K., Le, Q.V., Nguyen, P., Senior, A., Vanhoucke, V., Dean, J., et al.: On rectified linear units for speech processing. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3517–3521. IEEE (2013)Google Scholar
  15. 15.
    Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of the ICML, vol. 30 (2013)Google Scholar
  16. 16.
    Kingma, D., Ba, J.: Adam: A method for stochastic optimization (2014). arXiv preprint arXiv:1412.6980

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Manuele Palmeri
    • 2
    Email author
  • Filippo Vella
    • 1
  • Ignazio Infantino
    • 1
  • Salvatore Gaglio
    • 1
    • 2
  1. 1.Italian National Research Council of ItalyICAR-CNRPalermoItaly
  2. 2.DIIDUniversity of PalermoPalermoItaly

Personalised recommendations