Sign Languague Recognition Without Frame-Sequencing Constraints: A Proof of Concept on the Argentinian Sign Language

  • Franco RonchettiEmail author
  • Facundo Quiroga
  • César Estrebou
  • Laura Lanzarini
  • Alejandro Rosete
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10022)


Automatic sign language recognition (SLR) is an important topic within the areas of human-computer interaction and machine learning. On the one hand, it poses a complex challenge that requires the intervention of various knowledge areas, such as video processing, image processing, intelligent systems and linguistics. On the other hand, robust recognition of sign language could assist in the translation process and the integration of hearing-impaired people, as well as the teaching of sign language for the hearing population.

SLR systems usually employ Hidden Markov Models, Dynamic Time Warping or similar models to recognize signs. Such techniques exploit the sequential ordering of frames to reduce the number of hypothesis. This paper presents a general probabilistic model for sign classification that combines sub-classifiers based on different types of features such as position, movement and handshape. The model employs a bag-of-words approach in all classification steps, to explore the hypothesis that ordering is not essential for recognition. The proposed model achieved an accuracy rate of 97 % on an Argentinian Sign Language dataset containing 64 classes of signs and 3200 samples, providing some evidence that indeed recognition without ordering is possible.


Sign language recognition Bag-of-words Argentinian sign language 


  1. 1.
    Cooper, H., Holt, B., Bowden, R.: Sign language recognition. In: Moeslund, T.B., Hilton, A., Krüger, V., Sigal, L. (eds.) Visual Analysis of Humans: Looking at People, Chap. 27, pp. 539–562. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    Cooper, H., Ong, E.J., Pugeault, N., Bowden, R.: Sign language recognition using sub-units. J. Mach. Learn. Res. 13, 2205–2231 (2012)zbMATHGoogle Scholar
  3. 3.
    Kadir, T., Bowden, R., Ong, E., Zisserman, A.: Minimal training, large lexicon, unconstrained sign language recognition. In: British Machine Vision Conference, pp. 96.1–96.10 (2004)Google Scholar
  4. 4.
    Matuck, G.R., Moreira, G.S.P., Saotome, O., da Cunha, A.M.: Recognizing the Brazilian Signs language alphabet with neural networks over visual 3D data sensor. In: Bazzan, A.L.C., Pichara, K. (eds.) IBERAMIA 2014. LNCS, vol. 8864, pp. 637–648. Springer, Heidelberg (2014)Google Scholar
  5. 5.
    Pugeault, N., Bowden, R.: Spelling it out: real-time ASL fingerspelling recognition. In: 1st IEEE Workshop on Consumers Depth Cameras for Computer Vision, in Conjunction with ICCV 2011 (2011)Google Scholar
  6. 6.
    Ronchetti, F., Quiroga, F., Lanzarini, L., Estrebou, C.: Distribution of action movements (DAM): a descriptor for human action recognition. Front. Comput. Sci. 9(6), 956–965 (2015)CrossRefGoogle Scholar
  7. 7.
    Ronchetti, F., Quiroga, F., Lanzarini, L., Estrebou, C.: Handshape recognition for argentinian sign language using probsom. J. Comput. Sci. Technol. 16(1), 1–5 (2016)Google Scholar
  8. 8.
    Roussos, A., Theodorakis, S., Pitsikalis, V., Maragos, P.: Hand tracking and affine shape-appearance handshape sub-units in continuous sign language recognition. In: Kutulakos, K.N. (ed.) ECCV 2010 Workshops, Part I. LNCS, vol. 6553, pp. 258–272. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  9. 9.
    Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)CrossRefGoogle Scholar
  10. 10.
    Von Agris, U., Zieren, J., Canzler, U., Bauer, B., Kraiss, K.F.: Recent developments in visual sign language recognition. Univers. Access Inf. Soc. 6(4), 323–362 (2008)CrossRefGoogle Scholar
  11. 11.
    Zieren, J., Kraiss, K.-F.: Robust person-independent visual sign language recognition. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds.) IbPRIA 2005. LNCS, vol. 3522, pp. 520–528. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Franco Ronchetti
    • 1
    Email author
  • Facundo Quiroga
    • 1
  • César Estrebou
    • 1
  • Laura Lanzarini
    • 1
  • Alejandro Rosete
    • 2
  1. 1.Instituto de Investigación en Informática LIDI, Facultad de informáticaUniversidad Nacional de La PlataLa PlataArgentina
  2. 2.Instituto Superior Politécnico Jose Antonio EcheverríaHavanaCuba

Personalised recommendations