Sign Language Recognition Model Combining Non-manual Markers and Handshapes

  • Luis QuesadaEmail author
  • Gabriela Marín
  • Luis A. Guerrero
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10069)


People with disabilities have fewer opportunities. Technological developments should be used to help these people to have more opportunities. In this paper we present partial results of a research project which aims to help people with disabilities, specifically deaf and hard of hearing. We present a sign language recognition model. The model takes advantage of the natural user interfaces (NUI) and a classification algorithm (support vector machines). Moreover, we combine handshapes (signs) and non-manual markers (associated to emotions and face gestures) in the recognition process to enhance the sign language expressivity recognition. Additionally, non-manual markers representation is proposed. A model evaluation is also reported.


Sign language recognition Handshapes recognition Non-manual markers recognition Intel RealSense 



This work was partially supported by the Escuela de Ciencias de la Computación e Informática at Universidad de Costa Rica grant No. 320-B5-291, by Centro de Investigaciones en Tecnologías de la Información y Comunicación de la Universidad de Costa Rica, and Consejo Nacional para Investigaciones Científicas y Tecnológicas (CONICIT) of the Government of Costa Rica.


  1. 1.
    Lewis, P., Simons, G., Fennig, C.: Ethnologue: Languages of the World. SIL International (2009)Google Scholar
  2. 2.
    Oviedo, A.: Descripción General Básica de la LESCO (2012)Google Scholar
  3. 3.
    Woodward, J.: Sign language varieties in Costa Rica. Sign Lang. Stud. 73(1), 329–345 (1991)CrossRefGoogle Scholar
  4. 4.
    Caridakis, G., Asteriadis, S., Karpouzis, K.: Non-manual cues in automatic sign language recognition. Pers. Ubiquit. Comput. 18(1), 37–46 (2014)CrossRefGoogle Scholar
  5. 5.
    Zafrulla, Z., Brashear, H., Starner, T., Hamilton, H., Presti, P.: American sign language recognition with the kinect. In: Proceedings of the 13th International Conference Multimodal Interfaces, pp. 279–286 (2011)Google Scholar
  6. 6.
    Agarwal, A., Thakur, M.K.: Sign language recognition using Microsoft Kinect. In: Sixth International Conference on Contemporary Computing, pp. 181–185 (2013)Google Scholar
  7. 7.
    Huang, J., Zhou, W., Li, W., Li, H.: Sign language recognition using real-sense. In: EEE China Summit and International Conference on Signal and Information Processing (ChinaSIP), pp. 166–170 (2015)Google Scholar
  8. 8.
    Liu, J., Liu, B., Zhang, S., Yang, F., Yang, P., Metaxas, D.N., Neidle, C.: Non-manual grammatical marker recognition based on multi-scale, spatio-temporal analysis of head pose and facial expressions. Image Vis. Comput. 32(10), 671–681 (2014)CrossRefGoogle Scholar
  9. 9.
    Yang, H.D., Lee, S.W.: Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine. Pattern Recognit. Lett. 34(16), 2051–2056 (2013)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Nguyen, T.D., Ranganath, S.: Facial expressions in American sign language: tracking and recognition. Pattern Recognit. 45(5), 1877–1891 (2012)CrossRefzbMATHGoogle Scholar
  11. 11.
    Quesada, L., López, G., Guerrero, L.A.: Sign language recognition using leap motion. In: García-Chamizo, J.M., et al. (eds.) UCAml 2015. LNCS, vol. 9454, pp. 277–288. Springer, Heidelberg (2015)Google Scholar
  12. 12.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (1995)zbMATHGoogle Scholar
  13. 13.
    Antonakos, E., Roussos, A., Zafeiriou, S.: A survey on mouth modeling and analysis for Sign Language recognition. In: International Conference on Automatic Face and Gesture Recognition, pp. 1–7 (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Luis Quesada
    • 1
    • 2
    Email author
  • Gabriela Marín
    • 1
    • 2
  • Luis A. Guerrero
    • 1
    • 2
  1. 1.Escuela de Ciencias de la Computación e InformáticaUniversidad de Costa RicaSan PedroCosta Rica
  2. 2.Centro de Investigaciones en Tecnologías de La Información y ComunicaciónUniversidad de Costa RicaSan PedroCosta Rica

Personalised recommendations