Abstract
Autism spectrum disorder (ASD) is a neurobiological developmental disorder that affects communication and social interaction. Autistic children often have difficulty communicating effectively, both verbally and nonverbally. Good verbal and nonverbal communication skills help children with autism participate in social interactions. In this paper, we present a systematic literature review focused on analyzing AR/VR/Mixed reality technologies used to improve communication skills in children with ASD based on research published over the last ten years and available in relevant scientific databases. We reviewed 22 studies showing target area communication, how AR/VR/Mixed reality is used in the context of communication skills in ASD children, user experience and accessibility, technology limitations, and findings. This systematic review of the literature shows that the majority of the communication areas targeted are verbal and nonverbal simultaneously. Overall, the use of AR/VR/Mixed reality can be an alternative to improve children’s communication. Technologies that can facilitate the different abilities and emotions of children with ASD are considered in developing AR/VR/Mixed reality in the future.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Kodak, T., Bergmann, S.: Autism spectrum disorder. Pediatr. Clin. North Am. 67(3), 525–535 (2020). https://doi.org/10.1016/j.pcl.2020.02.007
Maenner, M.J., et al.: Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2020. MMWR Surveill. Summ. 72(2), 1–14 (2023). https://doi.org/10.15585/mmwr.ss7202a1
La Valle, C., Plesa-Skwerer, D., Tager-Flusberg, H.: Comparing the pragmatic speech profiles of minimally verbal and verbally fluent individuals with autism spectrum disorder. J. Autism Dev. Disord. 50(10), 3699–3713 (2020). https://doi.org/10.1007/s10803-020-04421-7
Baixauli-Fortea, I., Miranda Casas, A., Berenguer-Forner, C., Colomer-Diago, C., Roselló-Miranda, B.: Pragmatic competence of children with autism spectrum disorder. Impact of theory of mind, verbal working memory, ADHD symptoms, and structural language. Appl. Neuropsychol. Child 8(2), 101–112 (2019). https://doi.org/10.1080/21622965.2017.1392861
Kasilingam, N., Waddington, H., Van Der Meer, L.: Early intervention for children with autism spectrum disorder in New Zealand: what children get and what parents want. Int. J. Disabil. Dev. Educ. 68(4), 521–537 (2021). https://doi.org/10.1080/1034912X.2019.1696949
Vietze, P., Lax, L.E.: Early intervention ABA for toddlers with ASD: effect of age and amount. Curr. Psychol. 39(4), 1234–1244 (2020). https://doi.org/10.1007/s12144-018-9812-z
Taryadi, Kurniawan, I.: The improvement of autism spectrum disorders on children communication ability with PECS method multimedia augmented reality-based. J. Phys.: Conf. Ser. (2018). https://doi.org/10.1088/1742-6596/947/1/012009
Roberts-Yates, C., Silvera-Tawil, D.: Better education opportunities for students with autism and intellectual disabilities through digital technology. Int. J. Spec. Educ. 34(1), 197–210 (2019)
Cabero-Almenara, J., Roig-Vila, R.: The motivation of technological scenarios in Augmented Reality (AR): results of different experiments. Appl. Sci. 9(14), 2907 (2019). https://doi.org/10.3390/app9142907
Wohlgenannt, I., Simons, A., Stieglitz, S.: Virtual reality. Bus. Inf. Syst. Eng. 62(5), 455–461 (2020). https://doi.org/10.1007/s12599-020-00658-9
Tang, Y.M., Au, K.M., Lau, H.C.W., Ho, G.T.S., Wu, C.H.: Evaluating the effectiveness of learning design with mixed reality (MR) in higher education. Virtual Real. 24(4), 797–807 (2020). https://doi.org/10.1007/s10055-020-00427-9
Keshav, N.U., Vogt-Lowell, K., Vahabzadeh, A., Sahin, N.T.: Digital attention-related augmented-reality game: significant correlation between student game performance and validated clinical measures of attention-deficit/hyperactivity disorder (ADHD). Children 6(6) (2019). https://doi.org/10.3390/children6060072
Sahin, N.T., Abdus-Sabur, R., Keshav, N.U., Liu, R., Salisbury, J.P., Vahabzadeh, A.: Case study of a digital augmented reality intervention for autism in school classrooms: associated with improved social communication, cognition, and motivation via educator and parent assessment. Front. Educ. (Lausanne) 3 (2018). https://doi.org/10.3389/feduc.2018.00057
Ip, H.H.S., et al.: Enhance emotional and social adaptation skills for children with autism spectrum disorder: a virtual reality enabled approach. Comput. Educ. 117, 1–15 (2018). https://doi.org/10.1016/j.compedu.2017.09.010
Lahiri, U., Bekele, E., Dohrmann, E., Warren, Z., Sarkar, N.: Design of a virtual reality based adaptive response technology for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 21(1), 55–64 (2013). https://doi.org/10.1109/TNSRE.2012.2218618
Zhao, H., Swanson, A.R., Weitlauf, A.S., Warren, Z.E., Sarkar, N.: Hand-in-hand: a communication-enhancement collaborative virtual reality system for promoting social interaction in children with autism spectrum disorders. IEEE Trans. Hum. Mach. Syst. 48(2), 136–148 (2018). https://doi.org/10.1109/THMS.2018.2791562
Halabi, O., Abou El-Seoud, S., Alja’am, J., Alpona, H., Al-Hemadi, M., Al-Hassan, D.: Design of Immersive Virtual Reality System to Improve Communication Skills in Individuals with Autism. Int. J. Emerg. Technol. Learn. (iJET) 12(05), 50 (2017). https://doi.org/10.3991/ijet.v12i05.6766
Parsons, S.: Learning to work together: designing a multi-user virtual reality game for social collaboration and perspective-taking for children with autism. Int. J. Child. Comput. Interact. 6, 28–38 (2015). https://doi.org/10.1016/j.ijcci.2015.12.002
Mohd, C.K.N.C.K., Shahbodin, F., Suparjoh, S., Khidir, N.A.S.M.: Application of augmented reality in learning bakery for autism spectrum disorder. Int. J. Eng. Adv. Technol. 9(1), 2616–2620 (2019). https://doi.org/10.35940/ijeat.A9853.109119
Cai, Y., Chia, N.K.H., Thalmann, D., Kee, N.K.N., Zheng, J., Thalmann, N.M.: Design and development of a Virtual Dolphinarium for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 21(2), 208–217 (2013). https://doi.org/10.1109/TNSRE.2013.2240700
Herrero, J.F., Lorenzo, G.: An immersive virtual reality educational intervention on people with autism spectrum disorders (ASD) for the development of communication skills and problem solving. Educ. Inf. Technol. (Dordr.) 25(3), 1689–1722 (2020). https://doi.org/10.1007/s10639-019-10050-0
Hutson, J.: Social virtual reality: neurodivergence and inclusivity in the metaverse. Societies 12(4), 102 (2022). https://doi.org/10.3390/soc12040102
Ip, H.H.S., et al.: Virtual reality enabled training for social adaptation in inclusive education settings for school-aged children with Autism Spectrum Disorder (ASD). In: Cheung, S., Kwok, Lf., Shang, J., Wang, A., Kwan, R. (eds.) ICBL 2016. LNCS, vol. 9757, pp. 94–102. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41165-1_9
Yuan, S.N.V., Ip, H.H.S.: Using virtual reality to train emotional and social skills in children with autism spectrum disorder. Lond. J. Prim. Care (Abingdon) 10(4), 110–112 (2018). https://doi.org/10.1080/17571472.2018.1483000
Aguiar, Y.P.C., Galy, E., Godde, A., Trémaud, M., Tardif, C.: AutismGuide: a usability guideline to design software solutions for users with autism spectrum disorder. Behav. Inf. Technol. 41(6), 1132–1150 (2022). https://doi.org/10.1080/0144929X.2020.1856927
Nubia, R.M., Fabian, G.R., Wilson, R.A., Wilmer, P.B.: Development of a mobile application in augmented reality to improve the communication field of autistic children at a Neurorehabilitar Clinic. In: 2015 Workshop on Engineering Applications - International Congress on Engineering (WEA), pp. 1–6. IEEE (2015). https://doi.org/10.1109/WEA.2015.7370154
Bhatt, S.K., De Leon, N.I., Al-Jumaily, A.: Augmented reality game therapy for children with autism spectrum disorder. Int. J. Smart Sens. Intell. Syst. 7(2), 519–536 (2022). https://doi.org/10.21307/ijssis-2017-668
Bai, Z., Blackwell, A.F., Coulouris, G.: Through the looking glass: pretend play for children with autism. In: 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 49–58. IEEE (2013). https://doi.org/10.1109/ISMAR.2013.6671763
Lin, L.-Y., Lin, C.-H., Chuang, T.-Y., Loh, S.C., Chu, S.Y.: Using home-based augmented reality storybook training modules for facilitating emotional functioning and socialization of children with autism spectrum disorder. Int. J. Dev. Disabil. 1–8 (2023). https://doi.org/10.1080/20473869.2023.2202454
Zhao, H., Swanson, A., Weitlauf, A., Warren, Z., Sarkar, N.: A novel collaborative virtual reality game for children with ASD to foster social interaction. In: Antona, M., Stephanidis, C. (eds.) UAHCI 2016. LNCS, vol. 9739, pp. 276–288. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40238-3_27
Keshav, N.U., Salisbury, J.P., Vahabzadeh, A., Sahin, N.T.: Social communication coaching smartglasses: well tolerated in a diverse sample of children and adults with autism. JMIR Mhealth Uhealth 5(9) (2017). https://doi.org/10.2196/mhealth.8534
Liu, R., Salisbury, J.P., Vahabzadeh, A., Sahin, N.T.: Feasibility of an autism-focused augmented reality smartglasses system for social communication and behavioral coaching. Front Pediatr 5 (2017). https://doi.org/10.3389/fped.2017.00145
Moon, J., Ke, F.: Effects of adaptive prompts in virtual reality-based social skills training for children with autism. J. Autism Dev. Disord. (2023). https://doi.org/10.1007/s10803-023-06021-7
Ezeh, N.G., Anidi, O.C., Nwokolo, B.O.: Body language as a communicative aid amongst language impaired students: managing disabilities. Engl. Lang. Teach. 14(6), 125 (2021). https://doi.org/10.5539/elt.v14n6p125
Purnell, L.: Cross cultural communication: verbal and non-verbal communication, interpretation and translation. In: Douglas, M., Pacquiao, D., Purnell, L. (eds.) Global Applications of Culturally Competent Health Care: Guidelines for Practice, pp. 131–142. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-69332-3_14
Franchini, M., et al.: Variability in verbal and nonverbal communication in infants at risk for autism spectrum disorder: predictors and outcomes. J. Autism Dev. Disord. 48(10), 3417–3431 (2018). https://doi.org/10.1007/s10803-018-3607-9
Watkins, L., Kuhn, M., Ledbetter-Cho, K., Gevarter, C., O’Reilly, M.: Evidence-based social communication interventions for children with autism spectrum disorder. Indian J. Pediatr. 84(1), 68–75 (2017). https://doi.org/10.1007/s12098-015-1938-5
Febriantini, W.A. Fitriati, R., Oktaviani, L.: An analysis of verbal and non-verbal communication in autistic children. J. Res. Lang. Educ. (JoRLE) 2(1), 53–56 (2021). https://doi.org/10.1111/j.1460
Fong, K.N.K., Ma, W.Y., Pang, H.K., Tang, P.P.K., Law, L.L.F.: Immediate effects of coloured overlays on the reading performance of preschool children with an autism spectrum disorder using eye tracking. Res. Dev. Disabil. 89, 141–148 (2019). https://doi.org/10.1016/j.ridd.2019.03.012
Engel, K.S., Ehri, L.C.: Reading comprehension instruction for young students with autism: forming contextual connections. J. Autism Dev. Disord. 51(4), 1266–1280 (2021). https://doi.org/10.1007/s10803-020-04596-z
Daniels, S.: Visual Learning and Teaching: An Essential Guide for Educators K–8. Free Spirit Publishing, Minneapolis (2018)
Solis, M., Reutebuch, C.K., Falcomata, T., Jimenez, Z., Cravalho, D.: Reading intervention for students with ASD in the middle grades: an alternating treatment study of embedded interests reading and expository text conditions. Dev. Neurorehabil. 25(1), 45–58 (2022). https://doi.org/10.1080/17518423.2021.1942279
Zafiri, M., Konstantinidou, A., Pliogou, V.: The application of differentiated instruction in reading and writing to a boy with autism in early childhood education. A case study. Univ. J. Educ. Res. 7(12), 2609–2626 (2019). https://doi.org/10.13189/ujer.2019.071208
Wang, X., Laffey, J., Xing, W., Galyen, K., Stichter, J.: Fostering verbal and non-verbal social interactions in a 3D collaborative virtual learning environment: a case study of youth with Autism Spectrum Disorders learning social competence in iSocial. Educ. Tech. Res. Dev. 65(4), 1015–1039 (2017). https://doi.org/10.1007/s11423-017-9512-7
Chung, C.H., Chen, C.H.: Augmented reality based social stories training system for promoting the social skills of children with autism. In: Soares, M., Falcão, C., Ahram, T. (eds.) Advances in Ergonomics Modeling, Usability & Special Populations. Advances in Intelligent Systems and Computing, vol. 486, pp. 495–505. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-41685-4_44
Light, J., McNaughton, D.: Communicative competence for individuals who require augmentative and alternative communication: a new definition for a new era of communication? Augment. Altern. Commun. 30(1), 1–18 (2014). https://doi.org/10.3109/07434618.2014.885080
McKenney, E.L.W., Bristol, R.M.: Supporting intensive interventions for students with autism spectrum disorder: Performance feedback and discrete trial teaching. Sch. Psychol. Q. 30(1), 8–22 (2015). https://doi.org/10.1037/spq0000060
Tay, H.Y., Kee, K.N.N.: Effective questioning and feedback for learners with autism in an inclusive classroom. Cogent Educ. 6(1), 1634920 (2019). https://doi.org/10.1080/2331186X.2019.1634920
Bateman, K.J., et al.: Visual supports to increase conversation engagement for preschoolers with autism spectrum disorder during mealtimes: an initial investigation. J. Early Interv. (2022). https://doi.org/10.1177/10538151221111762
Meadan, H., Ostrosky, M.M., Triplett, B., Michna, A., Fettig, A.: Using visual supports with young children with autism spectrum disorder. Counc. Except. Child. (2016)
Munir, Kaosar, R.N., Rasim, Murtadha, I., Shahbodin, F., Riza, L.S.: Expert system using the educational game to determine children’s autism levels using forward chaining. Linguist. Cult. Rev. (2021). https://doi.org/10.37028/lingcure.v5nS1.1499
Chen, J., Wang, G., Zhang, K.: Personalized intelligent intervention and precise evaluation for children with autism spectrum disorder. In: Proceedings of DELFI Workshops, p. 127 (2020). https://doi.org/10.18420/delfi2020-ws-114
Khoirunnisa, A.N., Munir, Dewi, L.: Design and prototype development of augmented reality in reading learning for autism. Computers 12(3), 55 (2023). https://doi.org/10.3390/computers12030055
Acknowledgment
We would like to appreciate and thank PMDSU Scholarship under Direktorat Jenderal Pendidikan Tinggi, Kementerian Pendidikan dan Kebudayaan Republik Indonesia for support throughout this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Khoirunnisa, A.N., Munir, Dewi, L., Rasim, Azizah, N.N., Alivia, Z.P. (2024). The Use of Augmented Reality, Virtual Reality, and Mixed Reality in Communication Children’s with ASD: Systematic Literature Review. In: Badioze Zaman, H., et al. Advances in Visual Informatics. IVIC 2023. Lecture Notes in Computer Science, vol 14322. Springer, Singapore. https://doi.org/10.1007/978-981-99-7339-2_16
Download citation
DOI: https://doi.org/10.1007/978-981-99-7339-2_16
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-7338-5
Online ISBN: 978-981-99-7339-2
eBook Packages: Computer ScienceComputer Science (R0)