Skip to main content
SpringerLink
Log in

The Use of Augmented Reality, Virtual Reality, and Mixed Reality in Communication Children’s with ASD: Systematic Literature Review

  • Conference paper
  • First Online:
Advances in Visual Informatics (IVIC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14322))

Included in the following conference series:

  • 406 Accesses

Abstract

Autism spectrum disorder (ASD) is a neurobiological developmental disorder that affects communication and social interaction. Autistic children often have difficulty communicating effectively, both verbally and nonverbally. Good verbal and nonverbal communication skills help children with autism participate in social interactions. In this paper, we present a systematic literature review focused on analyzing AR/VR/Mixed reality technologies used to improve communication skills in children with ASD based on research published over the last ten years and available in relevant scientific databases. We reviewed 22 studies showing target area communication, how AR/VR/Mixed reality is used in the context of communication skills in ASD children, user experience and accessibility, technology limitations, and findings. This systematic review of the literature shows that the majority of the communication areas targeted are verbal and nonverbal simultaneously. Overall, the use of AR/VR/Mixed reality can be an alternative to improve children’s communication. Technologies that can facilitate the different abilities and emotions of children with ASD are considered in developing AR/VR/Mixed reality in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kodak, T., Bergmann, S.: Autism spectrum disorder. Pediatr. Clin. North Am. 67(3), 525–535 (2020). https://doi.org/10.1016/j.pcl.2020.02.007

    Article  Google Scholar 

  2. Maenner, M.J., et al.: Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2020. MMWR Surveill. Summ. 72(2), 1–14 (2023). https://doi.org/10.15585/mmwr.ss7202a1

    Article  Google Scholar 

  3. La Valle, C., Plesa-Skwerer, D., Tager-Flusberg, H.: Comparing the pragmatic speech profiles of minimally verbal and verbally fluent individuals with autism spectrum disorder. J. Autism Dev. Disord. 50(10), 3699–3713 (2020). https://doi.org/10.1007/s10803-020-04421-7

    Article  Google Scholar 

  4. Baixauli-Fortea, I., Miranda Casas, A., Berenguer-Forner, C., Colomer-Diago, C., Roselló-Miranda, B.: Pragmatic competence of children with autism spectrum disorder. Impact of theory of mind, verbal working memory, ADHD symptoms, and structural language. Appl. Neuropsychol. Child 8(2), 101–112 (2019). https://doi.org/10.1080/21622965.2017.1392861

  5. Kasilingam, N., Waddington, H., Van Der Meer, L.: Early intervention for children with autism spectrum disorder in New Zealand: what children get and what parents want. Int. J. Disabil. Dev. Educ. 68(4), 521–537 (2021). https://doi.org/10.1080/1034912X.2019.1696949

    Article  Google Scholar 

  6. Vietze, P., Lax, L.E.: Early intervention ABA for toddlers with ASD: effect of age and amount. Curr. Psychol. 39(4), 1234–1244 (2020). https://doi.org/10.1007/s12144-018-9812-z

    Article  Google Scholar 

  7. Taryadi, Kurniawan, I.: The improvement of autism spectrum disorders on children communication ability with PECS method multimedia augmented reality-based. J. Phys.: Conf. Ser. (2018). https://doi.org/10.1088/1742-6596/947/1/012009

  8. Roberts-Yates, C., Silvera-Tawil, D.: Better education opportunities for students with autism and intellectual disabilities through digital technology. Int. J. Spec. Educ. 34(1), 197–210 (2019)

    Google Scholar 

  9. Cabero-Almenara, J., Roig-Vila, R.: The motivation of technological scenarios in Augmented Reality (AR): results of different experiments. Appl. Sci. 9(14), 2907 (2019). https://doi.org/10.3390/app9142907

    Article  Google Scholar 

  10. Wohlgenannt, I., Simons, A., Stieglitz, S.: Virtual reality. Bus. Inf. Syst. Eng. 62(5), 455–461 (2020). https://doi.org/10.1007/s12599-020-00658-9

    Article  Google Scholar 

  11. Tang, Y.M., Au, K.M., Lau, H.C.W., Ho, G.T.S., Wu, C.H.: Evaluating the effectiveness of learning design with mixed reality (MR) in higher education. Virtual Real. 24(4), 797–807 (2020). https://doi.org/10.1007/s10055-020-00427-9

    Article  Google Scholar 

  12. Keshav, N.U., Vogt-Lowell, K., Vahabzadeh, A., Sahin, N.T.: Digital attention-related augmented-reality game: significant correlation between student game performance and validated clinical measures of attention-deficit/hyperactivity disorder (ADHD). Children 6(6) (2019). https://doi.org/10.3390/children6060072

  13. Sahin, N.T., Abdus-Sabur, R., Keshav, N.U., Liu, R., Salisbury, J.P., Vahabzadeh, A.: Case study of a digital augmented reality intervention for autism in school classrooms: associated with improved social communication, cognition, and motivation via educator and parent assessment. Front. Educ. (Lausanne) 3 (2018). https://doi.org/10.3389/feduc.2018.00057

  14. Ip, H.H.S., et al.: Enhance emotional and social adaptation skills for children with autism spectrum disorder: a virtual reality enabled approach. Comput. Educ. 117, 1–15 (2018). https://doi.org/10.1016/j.compedu.2017.09.010

    Article  Google Scholar 

  15. Lahiri, U., Bekele, E., Dohrmann, E., Warren, Z., Sarkar, N.: Design of a virtual reality based adaptive response technology for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 21(1), 55–64 (2013). https://doi.org/10.1109/TNSRE.2012.2218618

    Article  Google Scholar 

  16. Zhao, H., Swanson, A.R., Weitlauf, A.S., Warren, Z.E., Sarkar, N.: Hand-in-hand: a communication-enhancement collaborative virtual reality system for promoting social interaction in children with autism spectrum disorders. IEEE Trans. Hum. Mach. Syst. 48(2), 136–148 (2018). https://doi.org/10.1109/THMS.2018.2791562

    Article  Google Scholar 

  17. Halabi, O., Abou El-Seoud, S., Alja’am, J., Alpona, H., Al-Hemadi, M., Al-Hassan, D.: Design of Immersive Virtual Reality System to Improve Communication Skills in Individuals with Autism. Int. J. Emerg. Technol. Learn. (iJET) 12(05), 50 (2017). https://doi.org/10.3991/ijet.v12i05.6766

  18. Parsons, S.: Learning to work together: designing a multi-user virtual reality game for social collaboration and perspective-taking for children with autism. Int. J. Child. Comput. Interact. 6, 28–38 (2015). https://doi.org/10.1016/j.ijcci.2015.12.002

    Article  Google Scholar 

  19. Mohd, C.K.N.C.K., Shahbodin, F., Suparjoh, S., Khidir, N.A.S.M.: Application of augmented reality in learning bakery for autism spectrum disorder. Int. J. Eng. Adv. Technol. 9(1), 2616–2620 (2019). https://doi.org/10.35940/ijeat.A9853.109119

    Article  Google Scholar 

  20. Cai, Y., Chia, N.K.H., Thalmann, D., Kee, N.K.N., Zheng, J., Thalmann, N.M.: Design and development of a Virtual Dolphinarium for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 21(2), 208–217 (2013). https://doi.org/10.1109/TNSRE.2013.2240700

    Article  Google Scholar 

  21. Herrero, J.F., Lorenzo, G.: An immersive virtual reality educational intervention on people with autism spectrum disorders (ASD) for the development of communication skills and problem solving. Educ. Inf. Technol. (Dordr.) 25(3), 1689–1722 (2020). https://doi.org/10.1007/s10639-019-10050-0

    Article  Google Scholar 

  22. Hutson, J.: Social virtual reality: neurodivergence and inclusivity in the metaverse. Societies 12(4), 102 (2022). https://doi.org/10.3390/soc12040102

    Article  Google Scholar 

  23. Ip, H.H.S., et al.: Virtual reality enabled training for social adaptation in inclusive education settings for school-aged children with Autism Spectrum Disorder (ASD). In: Cheung, S., Kwok, Lf., Shang, J., Wang, A., Kwan, R. (eds.) ICBL 2016. LNCS, vol. 9757, pp. 94–102. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41165-1_9

  24. Yuan, S.N.V., Ip, H.H.S.: Using virtual reality to train emotional and social skills in children with autism spectrum disorder. Lond. J. Prim. Care (Abingdon) 10(4), 110–112 (2018). https://doi.org/10.1080/17571472.2018.1483000

    Article  Google Scholar 

  25. Aguiar, Y.P.C., Galy, E., Godde, A., Trémaud, M., Tardif, C.: AutismGuide: a usability guideline to design software solutions for users with autism spectrum disorder. Behav. Inf. Technol. 41(6), 1132–1150 (2022). https://doi.org/10.1080/0144929X.2020.1856927

    Article  Google Scholar 

  26. Nubia, R.M., Fabian, G.R., Wilson, R.A., Wilmer, P.B.: Development of a mobile application in augmented reality to improve the communication field of autistic children at a Neurorehabilitar Clinic. In: 2015 Workshop on Engineering Applications - International Congress on Engineering (WEA), pp. 1–6. IEEE (2015). https://doi.org/10.1109/WEA.2015.7370154

  27. Bhatt, S.K., De Leon, N.I., Al-Jumaily, A.: Augmented reality game therapy for children with autism spectrum disorder. Int. J. Smart Sens. Intell. Syst. 7(2), 519–536 (2022). https://doi.org/10.21307/ijssis-2017-668

    Article  Google Scholar 

  28. Bai, Z., Blackwell, A.F., Coulouris, G.: Through the looking glass: pretend play for children with autism. In: 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 49–58. IEEE (2013). https://doi.org/10.1109/ISMAR.2013.6671763

  29. Lin, L.-Y., Lin, C.-H., Chuang, T.-Y., Loh, S.C., Chu, S.Y.: Using home-based augmented reality storybook training modules for facilitating emotional functioning and socialization of children with autism spectrum disorder. Int. J. Dev. Disabil. 1–8 (2023). https://doi.org/10.1080/20473869.2023.2202454

  30. Zhao, H., Swanson, A., Weitlauf, A., Warren, Z., Sarkar, N.: A novel collaborative virtual reality game for children with ASD to foster social interaction. In: Antona, M., Stephanidis, C. (eds.) UAHCI 2016. LNCS, vol. 9739, pp. 276–288. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40238-3_27

  31. Keshav, N.U., Salisbury, J.P., Vahabzadeh, A., Sahin, N.T.: Social communication coaching smartglasses: well tolerated in a diverse sample of children and adults with autism. JMIR Mhealth Uhealth 5(9) (2017). https://doi.org/10.2196/mhealth.8534

  32. Liu, R., Salisbury, J.P., Vahabzadeh, A., Sahin, N.T.: Feasibility of an autism-focused augmented reality smartglasses system for social communication and behavioral coaching. Front Pediatr 5 (2017). https://doi.org/10.3389/fped.2017.00145

  33. Moon, J., Ke, F.: Effects of adaptive prompts in virtual reality-based social skills training for children with autism. J. Autism Dev. Disord. (2023). https://doi.org/10.1007/s10803-023-06021-7

  34. Ezeh, N.G., Anidi, O.C., Nwokolo, B.O.: Body language as a communicative aid amongst language impaired students: managing disabilities. Engl. Lang. Teach. 14(6), 125 (2021). https://doi.org/10.5539/elt.v14n6p125

    Article  Google Scholar 

  35. Purnell, L.: Cross cultural communication: verbal and non-verbal communication, interpretation and translation. In: Douglas, M., Pacquiao, D., Purnell, L. (eds.) Global Applications of Culturally Competent Health Care: Guidelines for Practice, pp. 131–142. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-69332-3_14

  36. Franchini, M., et al.: Variability in verbal and nonverbal communication in infants at risk for autism spectrum disorder: predictors and outcomes. J. Autism Dev. Disord. 48(10), 3417–3431 (2018). https://doi.org/10.1007/s10803-018-3607-9

    Article  Google Scholar 

  37. Watkins, L., Kuhn, M., Ledbetter-Cho, K., Gevarter, C., O’Reilly, M.: Evidence-based social communication interventions for children with autism spectrum disorder. Indian J. Pediatr. 84(1), 68–75 (2017). https://doi.org/10.1007/s12098-015-1938-5

    Article  Google Scholar 

  38. Febriantini, W.A. Fitriati, R., Oktaviani, L.: An analysis of verbal and non-verbal communication in autistic children. J. Res. Lang. Educ. (JoRLE) 2(1), 53–56 (2021). https://doi.org/10.1111/j.1460

  39. Fong, K.N.K., Ma, W.Y., Pang, H.K., Tang, P.P.K., Law, L.L.F.: Immediate effects of coloured overlays on the reading performance of preschool children with an autism spectrum disorder using eye tracking. Res. Dev. Disabil. 89, 141–148 (2019). https://doi.org/10.1016/j.ridd.2019.03.012

    Article  Google Scholar 

  40. Engel, K.S., Ehri, L.C.: Reading comprehension instruction for young students with autism: forming contextual connections. J. Autism Dev. Disord. 51(4), 1266–1280 (2021). https://doi.org/10.1007/s10803-020-04596-z

    Article  Google Scholar 

  41. Daniels, S.: Visual Learning and Teaching: An Essential Guide for Educators K–8. Free Spirit Publishing, Minneapolis (2018)

    Google Scholar 

  42. Solis, M., Reutebuch, C.K., Falcomata, T., Jimenez, Z., Cravalho, D.: Reading intervention for students with ASD in the middle grades: an alternating treatment study of embedded interests reading and expository text conditions. Dev. Neurorehabil. 25(1), 45–58 (2022). https://doi.org/10.1080/17518423.2021.1942279

    Article  Google Scholar 

  43. Zafiri, M., Konstantinidou, A., Pliogou, V.: The application of differentiated instruction in reading and writing to a boy with autism in early childhood education. A case study. Univ. J. Educ. Res. 7(12), 2609–2626 (2019). https://doi.org/10.13189/ujer.2019.071208

    Article  Google Scholar 

  44. Wang, X., Laffey, J., Xing, W., Galyen, K., Stichter, J.: Fostering verbal and non-verbal social interactions in a 3D collaborative virtual learning environment: a case study of youth with Autism Spectrum Disorders learning social competence in iSocial. Educ. Tech. Res. Dev. 65(4), 1015–1039 (2017). https://doi.org/10.1007/s11423-017-9512-7

    Article  Google Scholar 

  45. Chung, C.H., Chen, C.H.: Augmented reality based social stories training system for promoting the social skills of children with autism. In: Soares, M., Falcão, C., Ahram, T. (eds.) Advances in Ergonomics Modeling, Usability & Special Populations. Advances in Intelligent Systems and Computing, vol. 486, pp. 495–505. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-41685-4_44

  46. Light, J., McNaughton, D.: Communicative competence for individuals who require augmentative and alternative communication: a new definition for a new era of communication? Augment. Altern. Commun. 30(1), 1–18 (2014). https://doi.org/10.3109/07434618.2014.885080

    Article  Google Scholar 

  47. McKenney, E.L.W., Bristol, R.M.: Supporting intensive interventions for students with autism spectrum disorder: Performance feedback and discrete trial teaching. Sch. Psychol. Q. 30(1), 8–22 (2015). https://doi.org/10.1037/spq0000060

    Article  Google Scholar 

  48. Tay, H.Y., Kee, K.N.N.: Effective questioning and feedback for learners with autism in an inclusive classroom. Cogent Educ. 6(1), 1634920 (2019). https://doi.org/10.1080/2331186X.2019.1634920

    Article  Google Scholar 

  49. Bateman, K.J., et al.: Visual supports to increase conversation engagement for preschoolers with autism spectrum disorder during mealtimes: an initial investigation. J. Early Interv. (2022). https://doi.org/10.1177/10538151221111762

  50. Meadan, H., Ostrosky, M.M., Triplett, B., Michna, A., Fettig, A.: Using visual supports with young children with autism spectrum disorder. Counc. Except. Child. (2016)

    Google Scholar 

  51. Munir, Kaosar, R.N., Rasim, Murtadha, I., Shahbodin, F., Riza, L.S.: Expert system using the educational game to determine children’s autism levels using forward chaining. Linguist. Cult. Rev. (2021). https://doi.org/10.37028/lingcure.v5nS1.1499

  52. Chen, J., Wang, G., Zhang, K.: Personalized intelligent intervention and precise evaluation for children with autism spectrum disorder. In: Proceedings of DELFI Workshops, p. 127 (2020). https://doi.org/10.18420/delfi2020-ws-114

  53. Khoirunnisa, A.N., Munir, Dewi, L.: Design and prototype development of augmented reality in reading learning for autism. Computers 12(3), 55 (2023). https://doi.org/10.3390/computers12030055

Download references

Acknowledgment

We would like to appreciate and thank PMDSU Scholarship under Direktorat Jenderal Pendidikan Tinggi, Kementerian Pendidikan dan Kebudayaan Republik Indonesia for support throughout this research.

Author information

Authors and Affiliations

  1. Curriculum Development Study Program, Indonesia University of Education, Bandung, Indonesia

    Azizah Nurul Khoirunnisa & Laksmi Dewi

  2. Computer Science Education Study Program, Indonesia University of Education, Bandung, Indonesia

    Munir,  Rasim, Nissa Nur Azizah & Zsalzsa Puspa Alivia

Authors
  1. Azizah Nurul Khoirunnisa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Munir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Laksmi Dewi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rasim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nissa Nur Azizah
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zsalzsa Puspa Alivia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Azizah Nurul Khoirunnisa .

Editor information

Editors and Affiliations

  1. Universiti Tenaga Nasional, Kajang, Selangor, Malaysia

    Halimah Badioze Zaman

  2. University of Cambridge, Cambridge, UK

    Peter Robinson

  3. Dublin City University, Dublin, Ireland

    Alan F. Smeaton

  4. MIT Sloan School of Management, Asia School of Business, Cambridge, MA, USA

    Renato Lima De Oliveira

  5. University of Southern Denmark, Odense, Denmark

    Bo Nørregaard Jørgensen

  6. National Central University, Jhongli, Taiwan

    Timothy K. Shih

  7. Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia

    Rabiah Abdul Kadir

  8. Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia

    Ummul Hanan Mohamad

  9. Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia

    Mohammad Nazir Ahmad

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khoirunnisa, A.N., Munir, Dewi, L., Rasim, Azizah, N.N., Alivia, Z.P. (2024). The Use of Augmented Reality, Virtual Reality, and Mixed Reality in Communication Children’s with ASD: Systematic Literature Review. In: Badioze Zaman, H., et al. Advances in Visual Informatics. IVIC 2023. Lecture Notes in Computer Science, vol 14322. Springer, Singapore. https://doi.org/10.1007/978-981-99-7339-2_16

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-7339-2_16

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-7338-5

  • Online ISBN: 978-981-99-7339-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation