Gesture-Based Children Computer Interaction for Inclusive Education: A Systematic Literature Review

  • Pablo Torres-CarriónEmail author
  • Carina González-González
  • César Bernal-Bravo
  • Alfonso Infante-Moro
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 895)


Gestural interfaces are closely related with cognition and physical activity, and can be powerful tools for cognitive training and motor skills. Their use has been proposed by researchers in various areas, including education, and within this field, inclusive education. In this study, a systematic literature review about children computer gestural interactions (touch, body, face and motion) and on its application to digital educational resources for learning disabilities has been conducted. Applying the Torres-Carrión method, a “conceptual mindfact” and research problem has been structured, as a basis to build the search script, to be applied in the selected scientific databases (Scopus, WoS and Google Scholar). Five research questions are proposed, which involves standards of gesture-based computer interaction for children, design guides, methods and instruments, non-invasive interaction environments and personalization of didactic resources for children with special needs, in particular children with Down’ syndrome. As a final product, a list of relevant magazines and databases of the area has been obtained; 47 valid papers were analyzed to answer the research questions, and they are organized in a structured way, allowing the researcher to establish a valid context from which to focus future research.


Gestural computer interaction Children computer interaction Inclusive education Systematic review 


  1. 1.
    Torres-Carrion, P., González-González, C.S., Mora Carreño, A.: Methodology of emotional evaluation in education and rehabilitation activities for people with Down syndrome. In: XV International Conference on Human Computer Interaction, pp. 12–15 (2014)Google Scholar
  2. 2.
    Torres-Carrión, P.: Evaluación de Estrategias de Aprendizaje con HCI Kinect en alumnos con Síndrome de Down (2017).
  3. 3.
    Torres-Carrión, P., Gonzalez-Gonzalez, C.: Usability study of didactical resources to children with Down syndrome. In: Guerrero-Garcia, J., González-Calleros, J.M., Muñoz-Arteaga, J., Collazos, C.A. (eds.) Human-Computer Interaction Series, pp. 127–148. Springer, Heidelberg (2017). Scholar
  4. 4.
    Swartz, M.K.: The PRISMA statement: a guideline for systematic reviews and meta-analyses. J. Pediatr. Health Care. 25, 1–2 (2011)CrossRefGoogle Scholar
  5. 5.
    Torres-Carrión, P., González-González, C., Aciar, S., Rodríguez-Morales, G.: Methodology for systematic literature review applied to engineering and education. In: EDUCON2018 – IEEE Global Engineering Education Conference. IEEE Xplore Digital Library, Santa Cruz de Tenerife - España (2018)Google Scholar
  6. 6.
    Kitchenham, B.: Procedures for Performing Systematic Reviews. Joint Technical report, Keele, UK (2004)Google Scholar
  7. 7.
    Bacca, J., Baldiris, S., Fabregat, R.: Augmented reality trends in education: a systematic review of research and applications. J. Educ. Technol. Soc. 14, 133–149 (2014)Google Scholar
  8. 8.
    Yarosh, S., Radu, I., Hunter, S., Rosenbaum, E.: Examining values: an analysis of nine years of IDC research. In: Proceedings of the 10th International Conference on Interaction Design and Children, pp. 136–144. ACM (2011)Google Scholar
  9. 9.
    Sheu, F.R., Chen, N.S.: Taking a signal: a review of gesture-based computing research in education. Comput. Educ. 78, 268–277 (2014)CrossRefGoogle Scholar
  10. 10.
    Boucenna, S., et al.: Interactive technologies for autistic children: a review. Cognit. Comput. 6, 722–740 (2014)CrossRefGoogle Scholar
  11. 11.
    Benton, L., Johnson, H.: Widening participation in technology design: a review of the involvement of children with special educational needs and disabilities. Int. J. Child-Comput. Interact. 3–4, 23–40 (2015)CrossRefGoogle Scholar
  12. 12.
    Bevan, N.: International standards for HCI and usability. Int. J. Hum.-Comput. Stud. 55, 533–552 (2001)CrossRefGoogle Scholar
  13. 13.
    ISO: ISO/DIS 9241-960. Ergonomics of human-system interaction - Part 960: Framework and guidance for gesture interactions, Geneva, Switzerland (2015)Google Scholar
  14. 14.
    ISO: ISO 9241-210:2010. Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems, Geneva, Switzerland (2015)Google Scholar
  15. 15.
    González-González, C.S., Cairós-González, M., Navarro-Adelantado, V.: EMODIANA: Un instrumento para la evaluación subjetiva de emociones en niños y niñas. Actas del XIV Congr. Int. Interacción Pers. (2013)Google Scholar
  16. 16.
    UNESCO: The International Standard Classification of Education 2011. UNESCO Institute for Statistics, Montreal, Quebec, Canada (2012)Google Scholar
  17. 17.
    Dhiman, A.K.: Knowledge discovery in databases and libraries. DESIDOC J. Libr. Inf. Technol. 31, 446–451 (2011)CrossRefGoogle Scholar
  18. 18.
    Jarus, T., et al.: Effect of internal versus external focus of attention on implicit motor learning in children with developmental coordination disorder. Res. Dev. Disabil. 37, 119–126 (2015)CrossRefGoogle Scholar
  19. 19.
    Cheng, Y., Huang, R.: Using virtual reality environment to improve joint attention associated with pervasive developmental disorder. Res. Dev. Disabil. 33, 2141–2152 (2012)CrossRefGoogle Scholar
  20. 20.
    Fernández-López, Á., Rodríguez-Fórtiz, M.J., Rodríguez-Almendros, M.L., Martínez-Segura, M.J.: Mobile learning technology based on iOS devices to support students with special education needs. Comput. Educ. 61, 77–90 (2013)CrossRefGoogle Scholar
  21. 21.
    Toki, E.I., Pange, J.: The design of an expert system for the e-assessment and treatment plan of preschoolers’ speech and language disorders. Procedia - Soc. Behav. Sci. 9, 815–819 (2010)CrossRefGoogle Scholar
  22. 22.
    Cano, M.-D., Sanchez-Iborra, R.: On the use of a multimedia platform for music education with handicapped children: a case study. Comput. Educ. 87, 254–276 (2015)CrossRefGoogle Scholar
  23. 23.
    Parés, N., et al.: Starting research in interaction design with visuals for low-functioning children in the autistic spectrum: a protocol. Cyberpsychol. Behav. 9, 218–223 (2006)CrossRefGoogle Scholar
  24. 24.
    Mich, O., Pianta, E., Mana, N.: Interactive stories and exercises with dynamic feedback for improving reading comprehension skills in deaf children. Comput. Educ. 65, 34–44 (2013)CrossRefGoogle Scholar
  25. 25.
    Shih, C.-H., Shih, C.-T., Chu, C.-L.: Assisting people with multiple disabilities actively correct abnormal standing posture with a Nintendo Wii balance board through controlling environmental stimulation. Res. Dev. Disabil. 31, 936–942 (2010)CrossRefGoogle Scholar
  26. 26.
    Ahmed, S., Parsons, D.: Abductive science inquiry using mobile devices in the classroom. Comput. Educ. 63, 62–72 (2013)CrossRefGoogle Scholar
  27. 27.
    Crisco, J.J., Schwartz, J.B., Wilcox, B., Costa, L., Kerman, K.: Design and kinematic evaluation of a novel joint-specific play controller: application for wrist and forearm therapy. Phys. Ther. 95, 1061–1066 (2015)CrossRefGoogle Scholar
  28. 28.
    Cai, Y., Chia, N.K.H., Thalmann, D., Kee, N.K.N., Zheng, J., Thalmann, N.M.: Design and development of a Virtual Dolphinarium for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 208–217 (2013)CrossRefGoogle Scholar
  29. 29.
    Wuang, Y.-P., Chiang, C.-S., Su, C.-Y., Wang, C.-C.: Effectiveness of virtual reality using Wii gaming technology in children with Down syndrome. Res. Dev. Disabil. 32, 312–321 (2011)CrossRefGoogle Scholar
  30. 30.
    Lahiri, U., Bekele, E., Dohrmann, E., Warren, Z., Sarkar, N.: Design of a virtual reality based adaptive response technology for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 55–64 (2013)CrossRefGoogle Scholar
  31. 31.
    de la Guía, E., Lozano, M.D., Penichet, V.M.R.: Educational games based on distributed and tangible user interfaces to stimulate cognitive abilities in children with ADHD. Br. J. Educ. Technol. 46, 664–678 (2015)CrossRefGoogle Scholar
  32. 32.
    Seo, Y.-J., Woo, H.: The identification, implementation, and evaluation of critical user interface design features of computer-assisted instruction programs in mathematics for students with learning disabilities. Comput. Educ. 55, 363–377 (2010)CrossRefGoogle Scholar
  33. 33.
    Torrente, J., Freire, M., Moreno-Ger, P., Fernández-Manjón, B.: Evaluation of semi-automatically generated accessible interfaces for educational games. Comput. Educ. 83, 103–117 (2015)CrossRefGoogle Scholar
  34. 34.
    Stasolla, F., et al.: Computer and microswitch-based programs to improve academic activities by six children with cerebral palsy. Res. Dev. Disabil. 45–46, 1–13 (2015)CrossRefGoogle Scholar
  35. 35.
    Marco, J., Cerezo, E., Baldassarri, S.: Bringing tabletop technology to all: evaluating a tangible farm game with kindergarten and special needs children. Pers. Ubiquitous Comput. 17, 1577–1591 (2012)CrossRefGoogle Scholar
  36. 36.
    Shahin, M., et al.: Tabby talks: an automated tool for the assessment of childhood apraxia of speech. Speech Commun. 70, 49–64 (2015)CrossRefGoogle Scholar
  37. 37.
    Cantón, P., González, Á.L., Mariscal, G., Ruiz, C.: Applying new interaction paradigms to the education of children with special educational needs. In: Miesenberger, K., Karshmer, A., Penaz, P., Zagler, W. (eds.) ICCHP 2012. LNCS, vol. 7382, pp. 65–72. Springer, Heidelberg (2012). Scholar
  38. 38.
    Hall, S.S., Hammond, J.L., Hirt, M., Reiss, A.L.: A “learning platform” approach to outcome measurement in fragile X syndrome: a preliminary psychometric study. J. Intellect. Disabil. Res. 56, 947–960, 14p. (2012)CrossRefGoogle Scholar
  39. 39.
    Marti, P., Giusti, L.: A robot companion for inclusive games: a user-centred design perspective. In: 2010 IEEE International Conference on Robotics and Automation, pp. 4348–4353. IEEE (2010)Google Scholar
  40. 40.
    Keskinen, T., Heimonen, T., Turunen, M., Rajaniemi, J.-P., Kauppinen, S.: SymbolChat: a flexible picture-based communication platform for users with intellectual disabilities. Interact. Comput. 24, 374–386 (2012)CrossRefGoogle Scholar
  41. 41.
    Mombarg, R., Jelsma, D., Hartman, E.: Effect of Wii-intervention on balance of children with poor motor performance. Res. Dev. Disabil. 34, 2996–3003 (2013)CrossRefGoogle Scholar
  42. 42.
    Doyle, T., Arnedillo-Sánchez, I.: Using multimedia to reveal the hidden code of everyday behaviour to children with autistic spectrum disorders (ASDs). Comput. Educ. 56, 357–369 (2011)CrossRefGoogle Scholar
  43. 43.
    Dovis, S., Van der Oord, S., Wiers, R.W., Prins, P.J.M.: Improving executive functioning in children with ADHD: training multiple executive functions within the context of a computer game. A randomized double-blind placebo controlled trial. PLoS ONE 10, e0121651 (2015)CrossRefGoogle Scholar
  44. 44.
    Feng, J., Lazar, J., Kumin, L., Ozok, A.: Computer usage by children with Down syndrome: challenges and future research. ACM Trans. Access. Comput. 2, 1–44 (2010)CrossRefGoogle Scholar
  45. 45.
    Torres-Carrión, P., Sarmiento-Guerrero, C., Torres-Diaz, J.C., Barba-Guamán, L.: Educational math game for stimulation of children with dyscalculia (2018) Google Scholar
  46. 46.
    Jong, J.-T., Hong, J.-C., Yen, C.-Y.: Persistence temperament associated with children playing math games between touch panel and embodied interaction. J. Comput. Assist. Learn. 29, 569–578 (2013)CrossRefGoogle Scholar
  47. 47.
    Hamzah, M.S.J., Shamsuddin, S., Miskam, M.A., Yussof, H., Hashim, K.S.: Development of interaction scenarios based on pre-school curriculum in robotic intervention for children with autism. Procedia Comput. Sci. 42, 214–221 (2014)CrossRefGoogle Scholar
  48. 48.
    Fletcher-Watson, S., Pain, H., Hammond, S., Humphry, A., McConachie, H.: Designing for young children with autism spectrum disorder: a case study of an iPad app. Int. J. Child-Comput. Interact. 7, 1–14 (2016)CrossRefGoogle Scholar
  49. 49.
    Jelsma, D., Ferguson, G.D., Smits-Engelsman, B.C.M., Geuze, R.H.: Short-term motor learning of dynamic balance control in children with probable developmental coordination disorder. Res. Dev. Disabil. 38, 213–222 (2015)CrossRefGoogle Scholar
  50. 50.
    Sanna, A., Lamberti, F., Paravati, G., Rocha, F.D.: A kinect-based interface to animate virtual characters. J. Multimodal User Interfaces 7, 269–279 (2013)CrossRefGoogle Scholar
  51. 51.
    Levac, D., Rivard, L., Missiuna, C.: Defining the active ingredients of interactive computer play interventions for children with neuromotor impairments: a scoping review. Res. Dev. Disabil. 33, 214–223 (2012)CrossRefGoogle Scholar
  52. 52.
    Wagner, A., Rudraraju, R., Datla, S., Banerjee, A., Sudame, M., Gray, J.: Programming by voice. In: Proceedings of the 2012 ACM Annual Conference Extended Abstracts on Human Factors in Computing Systems Extended Abstracts - CHI EA 2012, p. 2087. ACM Press, New York (2012)Google Scholar
  53. 53.
    Tressoldi, P., Vio, C., Iozzino, R.: Efficacy of an intervention to improve fluency in children with developmental dyslexia in a regular orthography. J. Learn. Disabil. 40, 203–209, 7p. (2007)CrossRefGoogle Scholar
  54. 54.
    Rosenblum, S., Regev, N.: Timing abilities among children with developmental coordination disorders (DCD) in comparison to children with typical development. Res. Dev. Disabil. 34, 218–227 (2013)CrossRefGoogle Scholar
  55. 55.
    Campigotto, R., McEwen, R., Demmans Epp, C.: Especially social: exploring the use of an iOS application in special needs classrooms. Comput. Educ. 60, 74–86 (2013)CrossRefGoogle Scholar
  56. 56.
    Jones, A.C., Scanlon, E., Clough, G.: Mobile learning: two case studies of supporting inquiry learning in informal and semiformal settings. Comput. Educ. 61, 21–32 (2013)CrossRefGoogle Scholar
  57. 57.
    Hassan, A., Danish, Z.: Multi-touch user interfaces to treat social, communicative and collaborative impairments in children with autism: a review. Int. J. Comput. Appl. 117, 35–39 (2015)Google Scholar
  58. 58.
    Tabatabaei, S.M., Chalechale, A.: Using DLBP texture descriptors and SVM for Down syndrome recognition. In: 2014 4th International Conference on Computer and Knowledge Engineering (ICCKE), pp. 554–558. IEEE (2014)Google Scholar
  59. 59.
    Mahmoud, A.F.A., Belal, M.A.F., Helmy, Y.M.K.: Towards an intelligent tutoring system to Down syndrome. Int. J. Comput. Sci. Inf. Technol. 6, 129–137 (2014)Google Scholar
  60. 60.
    Gordon, I., Pierce, M.D., Bartlett, M.S., Tanaka, J.W.: Training facial expression production in children on the autism spectrum. J. Autism Dev. Disord. 44, 2486–2498 (2014)CrossRefGoogle Scholar
  61. 61.
    Visu-Petra, L., Benga, O., Tincas, I., Miclea, M.: Visual-spatial processing in children and adolescents with Down’s syndrome: a computerized assessment of memory skills. J. Intellect. Disabil. Res. 51, 942–952, 11p. (2007)CrossRefGoogle Scholar
  62. 62.
    Imhof, B., Scheiter, K., Edelmann, J., Gerjets, P.: Learning about locomotion patterns: effective use of multiple pictures and motion-indicating arrows. Comput. Educ. 65, 45–55 (2013)CrossRefGoogle Scholar
  63. 63.
    Ratz, C.: Do students with Down syndrome have a specific learning profile for reading? Res. Dev. Disabil. 34, 4504–4514 (2013)CrossRefGoogle Scholar
  64. 64.
    Næss, K.-A.B., Melby-Lervåg, M., Hulme, C., Lyster, S.-A.H.: Reading skills in children with Down syndrome: a meta-analytic review. Res. Dev. Disabil. 33, 737–747 (2012)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Universidad Técnica Particular de LojaLojaEcuador
  2. 2.Universidad de la LagunaLa LagunaSpain
  3. 3.Universidad Rey Juan CarlosMadridSpain
  4. 4.Universidad de HuelvaHuelvaSpain

Personalised recommendations