Skip to main content
SpringerLink
Log in

Vision-Based Portuguese Sign Language Recognition System

  • Conference paper
New Perspectives in Information Systems and Technologies, Volume 1

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 275))

Abstract

Vision-based hand gesture recognition is an area of active current research in computer vision and machine learning. Being a natural way of human interaction, it is an area where many researchers are working on, with the goal of making human computer interaction (HCI) easier and natural, without the need for any extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them, for example, to convey information. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. Hand gestures are a powerful human communication modality with lots of potential applications and in this context we have sign language recognition, the communication method of deaf people. Sign languages are not standard and universal and the grammars differ from country to country. In this paper, a real-time system able to interpret the Portuguese Sign Language is presented and described. Experiments showed that the system was able to reliably recognize the vowels in real-time, with an accuracy of 99.4% with one dataset of features and an accuracy of 99.6% with a second dataset of features. Although the implemented solution was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Wikipedia. Língua gestual portuguesa. 2012 September 9 (2013), http://pt.wikipedia.org/wiki/Lingua_gestual_portuguesa (cited 2013)

  2. Vijay, P.K., et al.: Recent Developments in Sign Language Recognition: A Review. International Journal on Advanced Computer Engineering and Communication Technology 1(2), 21–26 (2012)

    Google Scholar 

  3. Mingqiang, Y., Idiyo, K., Joseph, R.: A Survey of Shape Feature Extraction Techniques. Pattern Recognition, 43–90 (2008)

    Google Scholar 

  4. Miner, R.: RapidMiner: Report the Future (December 2011), http://rapid-i.com/

  5. Mitra, S., Acharya, T.: Gesture recognition: A Survey. IEEE Transactions on Systems, Man and Cybernetics, 311–324 (2007)

    Google Scholar 

  6. Murthy, G.R.S., Jadon, R.S.: A Review of Vision Based Hand Gestures Recognition. International Journal of Information Technology and Knowledge Management 2(2), 405–410 (2009)

    Google Scholar 

  7. Wachs, J.P., Stern, H., Edan, Y.: Cluster Labeling and Parameter Estimation for the Automated Setup of a Hand-Gesture Recognition System. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 35(6), 932–944 (2005)

    Article  Google Scholar 

  8. Trigueiros, P., Ribeiro, F., Reis, L.P.: A Comparative Study of different image features for hand gesture machine learning. In: 5th International Conference on Agents and Artificial Intelligence, Barcelona, Spain (2013)

    Google Scholar 

  9. Trigueiros, P., Ribeiro, F.: Vision-based Hand WheelChair Control. In: 12th International Conference on Autonomous Robot Systems and Competitions, Guimarães, Portugal (2012)

    Google Scholar 

  10. Wang, C.-C., Wang, K.-C.: Hand Posture Recognition Using Adaboost with SIFT for Human Robot Interaction. In: Proceedings of the International Conference on Advanced Robotics 2008, Jeju, Korea (2008)

    Google Scholar 

  11. Conseil, S., Bourenname, S., Martin, L.: Comparison of Fourier Descriptors and Hu Moments for Hand Posture Recognition. In: 15th European Signal Processing Conference (EUSIPCO) 2007, Poznan, Poland, pp. 1960–1964 (2007)

    Google Scholar 

  12. Barczak, A.L.C., et al.: Analysis of Feature Invariance and Discrimination for Hand Images: Fourier Descriptors versus Moment Invariants. In: International Conference Image and Vision Computing 2011, New Zeland (2011)

    Google Scholar 

  13. Bourennane, S., Fossati, C.: Comparison of shape descriptors for hand posture recognition in video. Signal, Image and Video Processing 6(1), 147–157 (2010)

    Article  Google Scholar 

  14. Triesch, J., Malsburg, C.V.D.: Robust Classification of Hand Postures against Complex Backgrounds. In: International Conference on Automatic Face and Gesture Recognition, Killington, Vermont, USA (1996)

    Google Scholar 

  15. Huynh, D.Q.: Evaluation of Three Local Descriptors on Low Resolution Images for Robot Navigation. In: 24th International Conference Image and Vision Computing, Wellington, New Zealand (2009)

    Google Scholar 

  16. Fang, Y., et al.: Hand Posture Recognition with Co-Training. In: 19th International Conference on Pattern Recognition, Tampa, FL, USA (2008)

    Google Scholar 

  17. Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: Proceedings of the Eleventh Annual Conference on Computational Learning Theory 1998, pp. 92–100. ACM, Madison (1998)

    Google Scholar 

  18. Tara, R.Y., Santosa, P.I., Adji, T.B.: Sign Language Recognition in Robot Teleoperation using Centroid Distance Fourier Descriptors. International Journal of Computer Applications 48(2) (2012)

    Google Scholar 

  19. Faria, B.M., Lau, N., Reis, L.P.: Classification of Facial Expressions Using Data Mining and machine Learning Algorithms. In: 4a Conferência Ibérica de Sistemas e Tecnologias de Informação, Póvoa de Varim, Portugal (2009)

    Google Scholar 

  20. Faria, B.M., Reis, L.P., Lau, N.: Cerebral Palsy EEG Signals Classification: Facial Expressions and Thoughts for Driving an Intelligent Wheelchair. In: 2012 IEEE 12th International Conference on Data Mining Workshops (ICDMW) (2012)

    Google Scholar 

  21. Gillian, N.E.: Gesture Recognition for Musician Computer Interaction. In: Music Department 2011, Faculty of Arts, p. 206. Humanities and Social Sciences, Belfast (2011)

    Google Scholar 

  22. Faria, B.M., et al.: Machine Learning Algorithms applied to the Classification of Robotic Soccer Formations and Opponent Teams. In: IEEE Conference on Cybernetics and Intelligent Systems (CIS) 2010, Singapore, pp. 344–349 (2010)

    Google Scholar 

  23. Mannini, A., Sabatini, A.M.: Machine learning methods for classifying human physical activity from on-body accelerometers. Sensors (Basel) 10(2), 1154–1175 (2010)

    Article  Google Scholar 

  24. Maldonado-Báscon, S., et al.: Road-Sign detection and Recognition Based on Support Vector Machines. IEEE Transactions on Intelligent Transportation Systems, 264–278 (2007)

    Google Scholar 

  25. Vicen-Bueno, R., et al.: Complexity Reduction in Neural Networks Appplied to Traffic Sign Recognition Tasks (2004)

    Google Scholar 

  26. Trigueiros, P., Ribeiro, F., Reis, L.P.: A comparison of machine learning algorithms applied to hand gesture recognition. In: 7th Iberian Conference on Information Systems and Technologies, Madrid, Spain (2012)

    Google Scholar 

  27. Rego, P., Moreira, P.M., Reis, L.P.: Serious games for rehabilitation: A survey and a classification towards a taxonomy. In: 2010 5th Iberian Conference on Information Systems and Technologies (CISTI) (2010)

    Google Scholar 

  28. Rego, P.A., Moreira, P.M., Reis, L.P.: Natural user interfaces in serious games for rehabilitation. In: 2011 6th Iberian Conference on Information Systems and Technologies, CISTI (2011)

    Google Scholar 

  29. Mendes, L., et al.: Virtual centre for the rehabilitation of road accident victims (VICERAVI). In: 2012 7th Iberian Conference on Information Systems and Technologies, CISTI (2012)

    Google Scholar 

  30. Rego, P.A., Moreira, P.M., Reis, L.P.: New Forms of Interaction in Serious Games for Rehabilitation. In: Handbook of Research on Serious Games as Educational, Business and Research Tools, pp. 1188–1211. IGI Global (2012)

    Google Scholar 

  31. Montesano, L., et al.: Towards an Intelligent Wheelchair System for Users With Cerebral Palsy. IEEE Transactions on Neural Systems and Rehabilitation Engineering 18(2), 193–202 (2010)

    Article  Google Scholar 

  32. Braga, R.A., et al.: IntellWheels: modular development platform for intelligent wheelchairs. J. Rehabil. Res. Dev. 48(9), 1061–1076 (2011)

    Article  Google Scholar 

  33. Faria, B.M., et al.: Evaluation of distinct input methods of an intelligent wheelchair in simulated and real environments: a performance and usability study. Assistive Technology: The Official Journal of RESNA 25(2), 88–98 (2013)

    Article  MathSciNet  Google Scholar 

  34. Faria, B.M., et al.: A Methodology for Creating Intelligent Wheelchair Users’ Profiles. In: 4th International Conference on Agents and Artificial Intelligence, Algarve, Portugal (2012)

    Google Scholar 

  35. Ke, W., et al.: Real-Time Hand Gesture Recognition for Service Robot, pp. 976–979 (2010)

    Google Scholar 

  36. Oshita, M., Matsunaga, T.: Automatic learning of gesture recognition model using SOM and SVM. In: Bebis, G., et al. (eds.) ISVC 2010, Part I. LNCS, vol. 6453, pp. 751–759. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  37. Almeida, R., Reis, L.P., Jorge, A.M.: Analysis and Forecast of Team Formation in the Simulated Robotic Soccer Domain. In: Lopes, L.S., Lau, N., Mariano, P., Rocha, L.M. (eds.) EPIA 2009. LNCS, vol. 5816, pp. 239–250. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  38. Oka, K., Sato, Y., Koike, H.: Real-time fingertip tracking and gesture recognition. IEEE Computer Graphics and Applications 22(6), 64–71 (2002)

    Article  Google Scholar 

  39. Perrin, S., Cassinelli, A., Ishikawa, M.: Gesture recognition using laser-based tracking system. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition, Seoul, South Korea (2004)

    Google Scholar 

  40. Binh, N.D., Shuichi, E., Ejima, T.: Real-Time Hand Tracking and Gesture Recognition System. In: Proceedings of International Conference on Graphics, Vision and Image 2005, Cairo, Egypt (2005)

    Google Scholar 

  41. Chen, F.-S., Fu, C.-M., Huang, C.-L.: Hand gesture recognition using a real-time tracking method and hidden Markov models. Image and Vision Computing 21(8), 745–758 (2003)

    Article  Google Scholar 

  42. Kelly, D., McDonald, J., Markham, C.: Recognition of Spatiotemporal Gestures in Sign Language Using Gesture Threshold HMMs. In: Wang, L., et al. (eds.) Machine Learning for Vision-Based Motion Analysis, pp. 307–348. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  43. Zafrulla, Z., et al.: American sign language recognition with the kinect. In: 13th International Conference on Multimodal Interfaces 2011, pp. 279–286. ACM, Alicante (2011)

    Google Scholar 

  44. Chowdhury, J.R.: Kinect Sensor for Xbox Gaming, IIT Kharagpur (2012)

    Google Scholar 

  45. Andersen, M.R., et al.: Kinect Depth Sensor Evaluation for Computer Vision Applications. In: Technical report ECE-TR-6 2012, Department of Engineering – Electrical and Computer Engineering, Aarhus University, p. 37 (2012)

    Google Scholar 

  46. Cooper, H., Bowden, R.: Large lexicon detection of sign language. In: Lew, M., Sebe, N., Huang, T.S., Bakker, E.M. (eds.) HCI 2007. LNCS, vol. 4796, pp. 88–97. Springer, Heidelberg (2007)

    Google Scholar 

  47. Trigueiros, P., Ribeiro, F., Reis, L.P.: Vision Based Referee Sign Language Recognition System for the RoboCup MSL League. In: 17th Annual Robocup International Symposium, Eindhoven, Holland (2013)

    Google Scholar 

  48. Trigueiros, P., Ribeiro, F., Reis, L.P.: Vision-based Gesture Recognition System for Human-Computer Interaction. In: IV ECCOMAS Thematic Conference on Computational Vision and Medical Image Processing. Taylor and Francis, Publication, Funchal (2013)

    Google Scholar 

  49. Lieberman, Z., Watson, T., Castro, A.: openFrameworks. 2004 10 October (2013) [cited 2011; openFrameworks is an open source C++ toolkit designed to assist the creative process by providing a simple and intuitive framework for experimentation], http://www.openframeworks.cc/

  50. Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library, 1st edn. O’Reilly Media (2008)

    Google Scholar 

  51. OpenNI. The standard framework for 3D sensing (2013), http://www.openni.org/

  52. King, D.E.: Dlib-ml: A Machine Learning Toolkit. Journal of Machine Learning Research 10, 1755–1758 (2009)

    Google Scholar 

  53. Zhang, D., Lu, G.: A comparative Study of Fourier Descriptors for Shape Representation and Retrieval. In: Proc. of 5th Asian Conference on Computer Vision (ACCV). Springer, Melbourne (2002)

    Google Scholar 

  54. Kauppinen, H., Seppanen, T., Pietikainen, M.: An experimental comparison of autoregressive and Fourier-based descriptors in 2D shape classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 17(2), 201–207 (1995)

    Article  Google Scholar 

  55. Zhang, D., Lu, G.: A Comparative Study on Shape Retrieval Using Fourier Descriptors with Different Shape Signatures. Journal of Visual Communication and Image Representation 14(1), 41–60 (2003)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Instituto Politécnico do Porto, Porto, Portugal

    Paulo Trigueiros

  2. Departamento de Electrónica Industrial da Universidade do Minho, Campus de Azurém, 4800-058, Guimarães, Portugal

    Fernando Ribeiro

  3. EEUM – Escola de Engenharia da Universidade do Minho – DSI, Campus de Azurém, 4800-058, Guimarães, Portugal

    Luís Paulo Reis

Authors
  1. Paulo Trigueiros
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernando Ribeiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luís Paulo Reis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paulo Trigueiros .

Editor information

Editors and Affiliations

  1. Universidade de Coimbra & LIACC, Rio Tinto, Portugal

    Álvaro Rocha

  2. Instituto Superior de Estatística e Gestão de Informação, Universidade Nova de Lisboa, Lisboa, Portugal

    Ana Maria Correia

  3. Department of Business Information Systems, Auckland University of Technology, Auckland, New Zealand

    Felix . B Tan

  4. Empirica GmbH, Bonn, Germany

    Karl . A Stroetmann

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Trigueiros, P., Ribeiro, F., Reis, L.P. (2014). Vision-Based Portuguese Sign Language Recognition System. In: Rocha, Á., Correia, A., Tan, F., Stroetmann, K. (eds) New Perspectives in Information Systems and Technologies, Volume 1. Advances in Intelligent Systems and Computing, vol 275. Springer, Cham. https://doi.org/10.1007/978-3-319-05951-8_57

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-05951-8_57

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-05950-1

  • Online ISBN: 978-3-319-05951-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation