Abstract
The use of Augmented Reality (AR) has increased markedly as an educational tool due to the implementation of smart devices (PCs, smartphones or tablets), that complement the interaction with the real world, allowing a better perception of the environment for people with hearing impairment by the additional information it generates. This article focuses on developing a low-cost functional application that is interactive and easy to manipulate for people with hearing disabilities or people who work or socialize with them, with the aim of facilitating the process of learning sign language. The implementation was carried out using Augmented Reality along with the Kinect V2 device, due to the fidelity of the depth camera and RGB camera, which enable the design of the learning and evaluation process using Unity software. Finally, the results obtained by carrying out the experimental method and performing the application’s operational tests for prototype validation are presented, achieving a gesture recognition accuracy of 90%. In this way, a reinforcement and complement is allowed in the learning of Ecuadorian sign language in an interactive way, providing an improvement in the quality of life of people with hearing impairment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tora Mildred, A.: Analisis del estado auditivo del personal militar del ejercito valorado en el dispensario medico de tolemaida en 2017 (2019)
Consejo Nacional para la igualdad de Discapacidades. Estadísticas de Discapacidad (2019) [Online]. Available: https://www.consejodiscapacidades.gob.ec/estadisticas-de-discapacidad/. Accessed: 09 Sept 2019
Lenguaje de señas entre niños sordos de padres sordos y oyentes [Online]. Available: https://cybertesis.unmsm.edu.pe/bitstream/handle/20.500.12672/1229/Garcia_bi.pdf?sequence=1&isAllowed=y. Accessed: 05 Aug 2020
Raúl francisco lara b: el lugar de la lengua de señas ecuatoriana (lsec) en la constitución subjetiva de niños sordos, no. 1, pp. 6–8 (2017)
FENASEC.EC [Online]. Available: http://www.fenasec.ec/diccionario-lsec.html. Accessed: 05 Aug 2020
Ahmed, M., Idrees, M., Ul Abideen, Z., Mumtaz, R., Khalique, S.: Deaf talk using 3D animated sign language: a sign language interpreter using Microsoft’s kinect v2. In: Proceedings 2016 SAI Computer Conference, SAI 2016, pp. 330–335 (2016)
Abreu, J.G., Teixeira, J.M., Figueiredo, L.S., Teichrieb, V.: Evaluating sign language recognition using the myo armband. In: Proceedings—18th Symposium on Virtual Augment. Reality, SVR 2016, pp. 64–70 (2016)
Tecnología y lengua de signos, una alianza que elimina barreras—Nobbot [Online]. Available: https://www.nobbot.com/personas/tecnologia-lengua-de-signos/. Accessed: 09 July 2020
Cerón, C., Archundia, E., Garcés, A., Beltrán, B., Migliolo, J.: Diseño de escenarios de aprendizaje con interfaces naturales y realidad aumentada para apoyar la inclusión de estudiantes con discapacidad auditiva en la educación media superior. Res. Comput. Sci. 144(1), 191–201 (2017)
Lachat, E., Macher, H., Mittet, M.A., Landes, T., Grussenmeyer, P.: First experiences with kinect V2 sensor for close range 3D modelling. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. ISPRS Arch. 40(5W4), 93–100 (2015)
Repositorio Digital UTE: Diseño de una guía interactiva de los museos en lengua de señas ecuatoriana para turistas nacionales y extranjeros sordos, Quito 2018. Accessed: 27 July 2020
Lengua de señas ecuatoriana LSEC Sign Language Alfabeto Dactilológico Manual Alphabet – Inclusive Routes [Online]. Available: Accessed 27 July 2020
Guante traductor de lengua de señas | Psicología—Neuropsicología [Online]. Available: http://bancodeideas.gob.ec/proyecto/view?data=aWQ9NjU1MA%3D%3D. Accessed: 03 Aug 2020
Basogain, X., Olabe, M., Espinosa, K., Olabe, C.R.J.C.: Realidad Aumentada en la Educación: una tecnología emergente. Semana 5, 12–15 (2010)
Aplicaciones de los dispositivos móviles augmented reality, an evolution of the application (2012)
Silverstein, E., Snyder, M.: Comparative analysis of respiratory motion tracking using Microsoft Kinect v2 sensor, no. February, pp. 193–204 (2018)
Liu, T., Li, L., Zhang, X.: Real-Time 3D Virtual Dressing Based on Users' Skeletons, no. Icsai, pp. 1378–1382 (2017)
Yang, L., Zhang, L., Dong, H., Alelaiwi, A., El Saddik, A.: Evaluating and improving the depth accuracy of kinect for Windows v2. IEEE Sens. J. 15(8), 4275–4285 (2015)
Tokuoka, M., et al.: Development of Gesture Recognition Sub-system for BELONG Increasing the Sense of Immersion for Dinosaurian Environment Learning, vol. 1, no. Csedu, pp. 493–498 (2017)
Visual Gesture Builder (VGB)—TheWindowsUpdate.com [Online]. Available: http://thewindowsupdate.com/2017/05/04/visual-gesture-builder-vgb/. Accessed: 30 July 2020
Morelli, R.D.: Modelado paramétrico 3d, render y animación con software libre: interacción freecad + blender (2015)
Ma, L., Sacks, R., Kattel, U., Bloch, T.: 3D object classification using geometric features and pairwise relationships. Comput. Civ. Infrastruct. Eng. 33(2), 152–164 (2018)
Bos, A., Kuhn, I., Luigi, R., Guarese, M.: Educational Technology and Its Contributions in Students’ Focus and Attention Regarding Augmented Reality Environments and the Use of Sensors, no. June 2019 (2020)
Garcia, C.A., Caiza, G., Naranjo, J.E., Ortiz, A., Garcia, M.V.: An approach of training virtual environment for teaching electro-pneumatic systems. IFAC-PapersOnLine 52(9), 318–323 (2019)
Tierra, C., Fernanda, G., Andrade, M., Carlos, J.: Aplicación de Realidad Aumentada para visualizar variables en Procesos Industriales
De Creación, M., Ouazzani, I., Peralta Donate, J.: Manual De Creación D Videojuego Con Unity Universidad Carlos Iii Manual De Creación De Videojuego Con Unity 3D (2016)
Koca, B.A.: Augmented Reality Application for Preschool Children with Unity 3D Platform, pp. 19–22 (2019)
VisualStudio: Visual Studio para PC y Mac (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Baldeon, K., Oñate, W., Caiza, G. (2022). Augmented Reality for Learning Sign Language Using Kinect Tool. In: Rocha, Á., Fajardo-Toro, C.H., Rodríguez, J.M.R. (eds) Developments and Advances in Defense and Security . Smart Innovation, Systems and Technologies, vol 255. Springer, Singapore. https://doi.org/10.1007/978-981-16-4884-7_37
Download citation
DOI: https://doi.org/10.1007/978-981-16-4884-7_37
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-4883-0
Online ISBN: 978-981-16-4884-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)