Skip to main content
SpringerLink
Log in

Exploring the design space of virtual tutors for children with autism spectrum disorder

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Children with autism spectrum disorder (ASD) require long-term interventions with professional tutors. In this paper, we explore the design space of virtual tutors in digital learning systems that may facilitate effective ASD interventions. We conducted a classroom observational study to examine how children with ASD learn and interact with tutors, organized a design workshop with the children to understand their visual preferences for virtual tutors, and held a focus group with tutors and individual interviews with parents to investigate their teaching methods, techniques, and challenges at school and at home. We identified three types of interactions and five types of learning behaviors in children with ASD. We also found tutors and parents followed a structured training approach and often used personalized rewards to mediate the intervention. Based on previous research and our empirical results, we proposed 12 design considerations for designing virtual tutors’ visual, audio, kinesthetic, and interaction features. We also presented a design case for developing a virtual tutor in a language learning app to help children with ASD learn social communication skills. We conclude with a discussion of the advantages and challenges of designing and employing virtual tutors in educational contexts for children with ASD.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are not publicly available due to the ethical issue but are available from the corresponding author on reasonable request.

Notes

  1. “Reinforcement” refers to strengthening certain behaviors. “Extinction” means stopping certain behaviors. “Stimulus Control” is when certain behaviors occur again due to the beneficial consequences of such behaviors in the past. “Generalization” means certain behaviors learned in a context can be transferred to other situations (Chown, 2016).

  2. Animation link: https://www.youtube.com/watch?v=l5Ho-Efa4vk.

  3. Due to ethical reasons, we were only allowed to use the photos for data analysis within our research team.

  4. CO#Number is the abbreviation used to refer to each of the main results in Classroom Observation.

  5. FG#Number is the abbreviation used to refer to each of the results in Focus Group.

  6. DW#Number is the abbreviation used to refer to each of the results in the Design Workshop.

  7. SI#Number is the abbreviation used to refer to each of the results in the Semi-Structured Interview.

References

  • Alzyoudi, M., Sartawi, A., & Almuhiri, O. (2015). The impact of video modelling on improving social skills in children with autism. British Journal of Special Education, 42(1), 53–68. https://doi.org/10.1111/1467-8578.12057

    Article  Google Scholar 

  • American Psychiatric Association, & American Psychiatric Association (Eds.). (2013). Diagnostic and statistical manual of mental disorders: DSM-5 (5th ed). American Psychiatric Association.

  • Banda, D. R., Matuszny, R. M., & Turkan, S. (2007). Video modeling strategies to enhance appropriate behaviors in children with autism spectrum disorders. Teaching Exceptional Children, 39(6), 47–52. https://doi.org/10.1177/004005990703900607

    Article  Google Scholar 

  • Bono, V., Narzisi, A., Jouen, A.-L., Tilmont, E., Hommel, S., Jamal, W., Xavier, J., Billeci, L., Maharatna, K., Wald, M., Chetouani, M., Cohen, D., Muratori, F., & MICHELANGELO Study Group. (2016). GOLIAH: A gaming platform for home-based intervention in autism – principles and design. Frontiers in Psychiatry, 7, 70. https://www.frontiersin.org/article/10.3389/fpsyt.2016.00070

  • Brambilla, P., Hardan, A., di Nemi, S. U., Perez, J., Soares, J. C., & Barale, F. (2003). Brain anatomy and development in autism: Review of structural MRI studies. Brain Research Bulletin, 61(6), 557–569. https://doi.org/10.1016/j.brainresbull.2003.06.001

    Article  Google Scholar 

  • Brosnan, M., Johnson, H., Grawmeyer, B., Chapman, E., & Benton, L. (2015). Emotion recognition in animated compared to human stimuli in adolescents with autism spectrum disorder. Journal of Autism and Developmental Disorders, 45(6), 1785–1796. https://doi.org/10.1007/s10803-014-2338-9

    Article  Google Scholar 

  • Cai, Y., Chia, N. K. H., Thalmann, D., Kee, N. K. N., Zheng, J., & Thalmann, N. M. (2013). Design and development of a virtual dolphinarium for children with autism. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 21(2), 208–217. https://doi.org/10.1109/TNSRE.2013.2240700

    Article  Google Scholar 

  • Carter, E. J., Hyde, J., Williams, D. L., & Hodgins, J. K. (2016). Investigating the influence of avatar facial characteristics on the social behaviors of children with autism. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI'16), ACM, New York, NY, USA (pp. 140–151). https://doi.org/10.1145/2858036.2858345

  • Chown, N. (2016). Understanding and evaluating autism theory. Jessica Kingsley Publishers.

  • Cook, S. W., Friedman, H. S., Duggan, K. A., Cui, J., & Popescu, V. (2017). Hand gesture and mathematics learning: Lessons from an avatar. Cognitive Science, 41(2), 518–535. https://doi.org/10.1111/cogs.12344

    Article  Google Scholar 

  • Escobedo, L., Ibarra, C., Hernandez, J., Alvelais, M., & Tentori, M. (2014). Smart objects to support the discrimination training of children with autism. Personal and Ubiquitous Computing, 18(6), 1485–1497. https://doi.org/10.1007/s00779-013-0750-3

    Article  Google Scholar 

  • Fan, M., Fan, J., Jin, S., Antle, A. N., & Pasquier, P. (2018). EmoStory: A game-based system supporting children’s emotional development. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA'18), ACM, New York, NY, USA, Paper LBW058 (pp. 1–6). https://doi.org/10.1145/3170427.3188594

  • Francis, P., Balbo, S., & Firth, L. (2009). Towards co-design with users who have autism spectrum disorders. Universal Access in the Information Society, 8(3), 123–135. https://doi.org/10.1007/s10209-008-0143-y

    Article  Google Scholar 

  • Frank Lopresti, E., Mihailidis, A., & Kirsch, N. (2004). Assistive technology for cognitive rehabilitation: State of the art. Neuropsychological Rehabilitation, 14(1–2), 5–39. https://doi.org/10.1080/09602010343000101

    Article  Google Scholar 

  • Gabriels, R. L., & Hill, D. E. (2010). Growing Up with Autism: Working with School-Age Children and Adolescents. Guilford Press.

  • Gibbs, G. R. (2018). Analyzing Qualitative Data. SAGE Publications Ltd.https://doi.org/10.4135/9781526441867.

  • Gordon, I., Pierce, M. D., Bartlett, M. S., & Tanaka, J. W. (2014). Training facial expression production in children on the autism spectrum. Journal of Autism and Developmental Disorders, 44(10), 2486–2498. https://doi.org/10.1007/s10803-014-2118-6

    Article  Google Scholar 

  • Grandgeorge, M., & Masataka, N. (2016). Atypical color preference in children with autism spectrum disorder. Frontiers in Psychology, 7, 1976. https://www.frontiersin.org/article/10.3389/fpsyg.2016.01976

  • Harrold, N., Tan, C. T., Rosser, D., & Leong, T. W. (2014). CopyMe: A portable real-time feedback expression recognition game for children. In CHI'14 Extended Abstracts on Human Factors in Computing Systems (CHI EA'14), ACM, New York, NY, USA (pp. 1195–1200). https://doi.org/10.1145/2559206.2581279

  • Ho, T. Q., Gadke, D. L., Henington, C., Evans-McCleon, T. N., & Justice, C. A. (2019). The effects of animated video modeling on joint attention and social engagement in children with autism spectrum disorder. Research in Autism Spectrum Disorders, 58, 83–95. https://doi.org/10.1016/j.rasd.2018.09.004

    Article  Google Scholar 

  • Hsu, C.-W., & Teoh, Y.-S. (2017). Investigating event memory in children with autism spectrum disorder: Effects of a computer-mediated interview. Journal of Autism and Developmental Disorders, 47(2), 359–372. https://doi.org/10.1007/s10803-016-2959-2

    Article  Google Scholar 

  • Hutt, C., & Ounsted, C. (1966). The biological significance of gaze aversion with particular reference to the syndrome of infantile autism. Behavioral Science, 11(5), 346–356. https://doi.org/10.1002/bs.3830110504

    Article  Google Scholar 

  • Javed, H., Connor, O. B., & Cabibihan, J.-J. (2015). Thomas and friends: Implications for the design of social robots and their role as social story telling agents for children with autism. IEEE International Conference on Robotics and Biomimetics (ROBIO), 2015, 1145–1150. https://doi.org/10.1109/ROBIO.2015.7418926

    Article  Google Scholar 

  • Kandalaft, M. R., Didehbani, N., Krawczyk, D. C., Allen, T. T., & Chapman, S. B. (2013). Virtual reality social cognition training for young adults with high-functioning autism. Journal of Autism and Developmental Disorders, 43(1), 34–44. https://doi.org/10.1007/s10803-012-1544-6

    Article  Google Scholar 

  • Karst, J. S., & Van Hecke, A. V. (2012). Parent and family impact of autism spectrum disorders: A review and proposed model for intervention evaluation. Clinical Child and Family Psychology Review, 15(3), 247–277. https://doi.org/10.1007/s10567-012-0119-6

    Article  Google Scholar 

  • Kasari, C., & Patterson, S. (2012). Interventions addressing social impairment in autism. Current Psychiatry Reports, 14(6), 713–725. https://doi.org/10.1007/s11920-012-0317-4

    Article  Google Scholar 

  • Kashinath, S., Woods, J., & Goldstein, H. (2006). Enhancing generalized teaching strategy use in daily routines by parents of children with autism. Journal of Speech, Language, and Hearing Research, 49(3), 466–485. https://doi.org/10.1044/1092-4388(2006/036)

    Article  Google Scholar 

  • Keen, D., Brannigan, K. L., & Cuskelly, M. (2007). Toilet training for children with autism: The effects of video modeling. Journal of Developmental and Physical Disabilities, 19(4), 291–303. https://doi.org/10.1007/s10882-007-9044-x

    Article  Google Scholar 

  • Kelley, J. F. (1983). An empirical methodology for writing user-friendly natural language computer applications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'83), ACM, New York, NY, USA (pp. 193–196). https://doi.org/10.1145/800045.801609

  • Kim, Y., Thayne, J., & Wei, Q. (2017). An embodied agent helps anxious students in mathematics learning. Educational Technology Research and Development, 65(1), 219–235. https://doi.org/10.1007/s11423-016-9476-z

    Article  Google Scholar 

  • Koegel, R. L., Russo, D. C., & Rincover, A. (1977). Assessing and training teachers in the generauzed use of behavior modification with autistic children. Journal of Applied Behavior Analysis, 10(2), 197–205. https://doi.org/10.1901/jaba.1977.10-197

    Article  Google Scholar 

  • Ledford, J. R., Gast, D. L., Luscre, D., & Ayres, K. M. (2007). Observational and incidental learning by children with autism during small group instruction. Journal of Autism and Developmental Disorders, 38(1), 86. https://doi.org/10.1007/s10803-007-0363-7

    Article  Google Scholar 

  • Leijdekkers, P., Gay, V., & Wong, F. (2013). Capture my emotion: A mobile app to improve emotion learning for autistic children using sensors. In Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems, Porto Portugal (pp. 381–384). https://doi.org/10.1109/CBMS.2013.6627821

  • Lorenzo, G., Lledó, A., Pomares, J., & Roig, R. (2016). Design and application of an immersive virtual reality system to enhance emotional skills for children with autism spectrum disorders. Computers & Education, 98, 192–205. https://doi.org/10.1016/j.compedu.2016.03.018

    Article  Google Scholar 

  • Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55(1), 3–9. https://doi.org/10.1037/0022-006X.55.1.3

    Article  Google Scholar 

  • Merriman, B., & Guerin, S. (2006). Using children’s drawings as data in child-centred research. The Irish Journal of Psychology, 27(1–2), 48–57. https://doi.org/10.1080/03033910.2006.10446227

    Article  Google Scholar 

  • Millen, L., Cobb, S., & Patel, H. (2011). Participatory design approach with children with autism. International Journal on Disability and Human Development, 10(4), 289–294. https://doi.org/10.1515/IJDHD.2011.048

  • Moore, D., McGrath, P., & Thorpe, J. (2000). Computer-aided learning for people with autism – A framework for research and development. Innovations in Education and Training International, 37(3), 218–228. https://doi.org/10.1080/13558000050138452

    Article  Google Scholar 

  • Moreno, R., Mayer, R. E., Spires, H. A., & Lester, J. C. (2001). The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19(2), 177–213. https://doi.org/10.1207/S1532690XCI1902_02

    Article  Google Scholar 

  • Mottron, L. (2017). Should we change targets and methods of early intervention in autism, in favor of a strengths-based education? European Child & Adolescent Psychiatry, 26(7), 815–825. https://doi.org/10.1007/s00787-017-0955-5

    Article  Google Scholar 

  • Mower, E., Black, M. P., Flores, E., Williams, M., & Narayanan, S. (2011). Rachel: Design of an emotionally targeted interactive agent for children with autism. IEEE International Conference on Multimedia and Expo, 2011, 1–6. https://doi.org/10.1109/ICME.2011.6011990

    Article  Google Scholar 

  • Nonnis, A., & Bryan-Kinns, N. (2019). Mazi: Tangible technologies as a channel for collaborative play. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI'19), ACM, New York, NY, USA (pp. 1–13). https://doi.org/10.1145/3290605.3300670

  • Ozsivadjian, A., Knott, F., & Magiati, I. (2012). Parent and child perspectives on the nature of anxiety in children and young people with autism spectrum disorders: A focus group study. Autism, 16(2), 107–121. https://doi.org/10.1177/1362361311431703

    Article  Google Scholar 

  • Rabiee, F. (2004). Focus-group interview and data analysis. Proceedings of the Nutrition Society, 63(4), 655–660. https://doi.org/10.1079/PNS2004399

    Article  Google Scholar 

  • Ragone, G. (2020). Designing embodied musical interaction for children with autism. In Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS'20), ACM, New York, NY, USA (Article 104, pp. 1–4). https://doi.org/10.1145/3373625.3417077

  • Robins, B., Dautenhahn, K., Boekhorst, R. T., & Billard, A. (2005). Robotic assistants in therapy and education of children with autism: Can a small humanoid robot help encourage social interaction skills? Universal Access in the Information Society, 4(2), 105–120. https://doi.org/10.1007/s10209-005-0116-3

    Article  Google Scholar 

  • Saadatzi, M. N., Pennington, R. C., Welch, K. C., Graham, J. H., & Scott, R. E. (2017). The use of an autonomous pedagogical agent and automatic speech recognition for teaching sight words to students with autism spectrum disorder. Journal of Special Education Technology, 32(3), 173–183. https://doi.org/10.1177/0162643417715751

    Article  Google Scholar 

  • Sanders, J. L. (2009). Qualitative or quantitative differences between asperger’s disorder and autism? historical considerations. Journal of Autism and Developmental Disorders, 39(11), 1560. https://doi.org/10.1007/s10803-009-0798-0

    Article  Google Scholar 

  • Sarokoff, R. A., & Sturmey, P. (2008). The effects of instructions, rehearsal, modeling, and feedback on acquisition and generalization of staff use of discrete trial teaching and student correct responses. Research in Autism Spectrum Disorders, 2(1), 125–136. https://doi.org/10.1016/j.rasd.2007.04.002

    Article  Google Scholar 

  • Scassellati, B., Admoni, H., & Matarić, M. (2012). Robots for use in autism research. Annual Review of Biomedical Engineering, 14(1), 275–294. https://doi.org/10.1146/annurev-bioeng-071811-150036

    Article  Google Scholar 

  • Stahmer, A. C., Collings, N. M., & Palinkas, L. A. (2005). Early intervention practices for children with autism: Descriptions from community providers. Focus on Autism and Other Developmental Disabilities, 20(2), 66–79. https://doi.org/10.1177/10883576050200020301

    Article  Google Scholar 

  • Tan, C. T., Harrold, N., & Rosser, D. (2013). Can you CopyMe? An expression mimicking serious game. In SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications (SA'13), ACM, New York, NY, USA (Article 73, pp. 1–4). https://doi.org/10.1145/2543651.2543657

  • Tartaro, A., Cassell, J., Ratz, C., Lira, J., & Nanclares-Nogués, V. (2014). Accessing peer social interaction: Using authorable virtual peer technology as a component of a group social skills intervention program. ACM Transactions on Accessible Computing (TACCESS). https://doi.org/10.1145/2700434

    Article  Google Scholar 

  • Tinwell, A. (2014). The uncanny valley in games and animation. A K Peters/CRC Press. https://doi.org/10.1201/b17830

  • Tsai, W.-T., Lee, I.-J., & Chen, C.-H. (2021). Inclusion of third-person perspective in CAVE-like immersive 3D virtual reality role-playing games for social reciprocity training of children with an autism spectrum disorder. Universal Access in the Information Society, 20(2), 375–389. https://doi.org/10.1007/s10209-020-00724-9

    Article  Google Scholar 

  • Virués-Ortega, J. (2010). Applied behavior analytic intervention for autism in early childhood: Meta-analysis, meta-regression and dose–response meta-analysis of multiple outcomes. Clinical Psychology Review, 30(4), 387–399. https://doi.org/10.1016/j.cpr.2010.01.008

    Article  Google Scholar 

  • Wilson, C., Brereton, M., Ploderer, B., & Sitbon, L. (2019). Co-design beyond words: “Moments of Interaction” with minimally-verbal children on the autism spectrum. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI'19),ACM, New York, NY, USA, Paper 21, (pp. 1–15). https://doi.org/10.1145/3290605.3300251

  • Wilson, C., Brereton, M., Ploderer, B., Sitbon, L., & Saggers, B. (2017). Digital strategies for supporting strengths- and interests-based learning with children with autism. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS'17), ACM, New York, NY, USA (pp. 52–61). https://doi.org/10.1145/3132525.3132553

  • Zhang, L., Fu, Q., Swanson, A., Weitlauf, A., Warren, Z., & Sarkar, N. (2018). Design and evaluation of a collaborative virtual environment (CoMove) for autism spectrum disorder intervention. ACM Transactions on Accessible Computing, 11(2), 11, 1–22. https://doi.org/10.1145/3209687

  • Zhao, W., Liu, X., Qiu, T., & Luo, X. (2020). Virtual avatar-based life coaching for children with autism spectrum disorder. Computer, 53(2), 26–34. https://doi.org/10.1109/MC.2019.2915979

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Min Fan, Liyan Sheng these authors contributed equally to this work.

Authors and Affiliations

  1. School of Animation and Digital Arts, Communication University of China, Beijing, China

    Jiazhou Wu, Min Fan, Liyan Sheng & Guoyu Sun

Authors
  1. Jiazhou Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liyan Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guoyu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Min Fan or Guoyu Sun.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A

Table 3 Demographic information for children
Full size table

Appendix B

Table 4 Demographic information for tutors
Full size table

Appendix C

Table 5 Demographic information for parents
Full size table

Appendix D

Table 6 A summary of the design considerations and the applications in a design case
Full size table

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, J., Fan, M., Sheng, L. et al. Exploring the design space of virtual tutors for children with autism spectrum disorder. Educ Inf Technol 28, 16531–16560 (2023). https://doi.org/10.1007/s10639-023-11838-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-023-11838-x

Keywords

Search

Navigation