The Asia-Pacific Education Researcher

, Volume 27, Issue 3, pp 227–243 | Cite as

Augmented Reality Plus Concept Map Technique to Teach Children with ASD to Use Social Cues When Meeting and Greeting

  • I-Jui LeeEmail author
  • Chien-Hsu Chen
  • Chuan-Po Wang
  • Chi-Hsuan Chung
Regular Article


Autism spectrum disorders (ASD) are characterized by a reduced ability to appropriately express social greetings. Studies have indicated that individuals with ASD might not recognize the crucial nonverbal cues that usually aid social interaction. Social reciprocity depends on the ability to empathize with others, to be aware of emotional and interpersonal cues, and to respond appropriately; it requires joint attention and nonverbal social skills. Fortunately, there is evidence-based research which shows that augmented reality (AR) attracts the attention of children with ASD and allows them to focus on social cues. AR has also been proved effective for teaching social skills. However, there is a lack of appropriate instructional scaffolds in AR applications to help students organize learning materials. Therefore, in this study, we use AR combined with concept map (CM) strategy as a training tool to focus on the standard nonverbal social cues to teach children with ASD how to appropriately reciprocate when they greet others. The learner can integrate the AR with CM strategy to visually conceptualize the social scenarios in a tabletop role-play training platform. Single-subject research with a multiple-baselines across-subject design was used in this study. Our results showed substantial increases in the children’s target responses during the intervention phases compared with the baseline phases. Generalization probes were administered during baseline (4–8 sessions for 0.5–1 month), intervention (10 sessions for 1.2 months), and maintenance phases (4–8 sessions for 0.5–1 month) to assess the generalization and maintenance of learned skills. The three-phase test data suggest that the AR with CM intervention was moderately effective in teaching the target greeting responses to children with ASD. The practical and developmental implications of the findings are discussed.


Autism Augmented reality Concept mapping Interactive learning Tabletop role-play game Teaching/learning strategies 

Supplementary material

40299_2018_382_MOESM1_ESM.docx (29 kb)
Supplementary material 1 (DOCX 29 kb)


  1. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (DSM-5 ® ). Washington, DC: American Psychiatric Pub.CrossRefGoogle Scholar
  2. Attwood, T. (2006). The complete guide to Asperger’s syndrome. Philadelphia: Jessica Kingsley Publishers.Google Scholar
  3. Axe, J. B., & Evans, C. J. (2012). Using video modeling to teach children with PDD-NOS to respond to facial expressions. Research in Autism Spectrum Disorders, 6(3), 1176–1185.CrossRefGoogle Scholar
  4. Ayres, K. M., & Langone, J. (2005). Intervention and instruction with video for students with autism: A review of the literature. Education and Training in Developmental Disabilities, 40(2), 183–196.Google Scholar
  5. Bai, Z., Blackwell, A. F., & Coulouris, G. (2015a). Using augmented reality to elicit pretend play for children with autism. IEEE Transactions on Visualization and Computer Graphics, 21(5), 598–610.CrossRefGoogle Scholar
  6. Bai, Z., Blackwell, A. F., & Coulouris, G. (2015b, April). Exploring expressive augmented reality: The FingAR puppet system for social pretend play. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 1035–1044). ACM.Google Scholar
  7. Baker, J. (2001). Social skills picture book: Teaching communication, play and emotion. Arlington, TX: Future Horizons.Google Scholar
  8. Bellini, S., & Akullian, J. (2007). A meta-analysis of video modeling and video self-modeling interventions for children and adolescents with autism spectrum disorders. Exceptional children, 73(3), 264–287.CrossRefGoogle Scholar
  9. Billinghurst, M. (2002). Augmented reality in education. Seattle WA: New Horizons for Learning.Google Scholar
  10. Boelte, S., & Hallmayer, J. (2013). Autism spectrum conditions: FAQs on autism, Asperger syndrome, and atypical autism answered by international experts. Ashland, OH: Hogrefe Publishing.Google Scholar
  11. Broadhead, P. (2006). Developing an understanding of young children’s learning through play: The place of observation, interaction and reflection. British Educational Research Journal, 32(2), 191–207.CrossRefGoogle Scholar
  12. Castelli, F. (2005). Understanding emotions from standardized facial expressions in autism and normal development. Autism, 9(4), 428–449.CrossRefGoogle Scholar
  13. Charlop-Christy, M. H., & Daneshvar, S. (2003). Using video modeling to teach perspective taking to children with autism. Journal of Positive Behavior Interventions, 5(1), 12–21.CrossRefGoogle Scholar
  14. Charlop-Christy, M. H., Le, L., & Freeman, K. A. (2000). A comparison of video modeling with in vivo modeling for teaching children with autism. Journal of Autism and Developmental Disorders, 30(6), 537–552.CrossRefGoogle Scholar
  15. Charsky, D., & Ressler, W. (2011). “Games are made for fun”: Lessons on the effects of concept maps in the classroom use of computer games. Computers & Education, 56(3), 604–615.CrossRefGoogle Scholar
  16. Chen, C. H., Chou, Y. Y., & Huang, C. Y. (2016). An augmented-reality-based concept map to support mobile learning for science. The Asia-Pacific Education Researcher, 25(4), 567–578.CrossRefGoogle Scholar
  17. Chen, C. H., Lee, I. J., & Lin, L. Y. (2015). Augmented reality-based self-facial modeling to promote the emotional expression and social skills of adolescents with autism spectrum disorders. Research in Developmental Disabilities, 36, 396–403.CrossRefGoogle Scholar
  18. Chen, C. H., Wu, F. G., & Zhung, S. K. (2006). Design an augmented reality teaching system with concept mapping technique. In Proceedings of the Annual Conference of JSSD the 53rd Annual Conference of JSSD (pp. 250–251). Japanese Society for the Science of Design.Google Scholar
  19. Cheng, K. H., & Tsai, C. C. (2013). Affordances of augmented reality in science learning: Suggestions for future research. Journal of Science Education and Technology, 22(4), 449–462.CrossRefGoogle Scholar
  20. Di Serio, Á., Ibáñez, M. B., & Kloos, C. D. (2013). Impact of an augmented reality system on students’ motivation for a visual art course. Computers & Education, 68, 586–596.CrossRefGoogle Scholar
  21. Dube, W. V., MacDonald, R. P., Mansfield, R. C., Holcomb, W. L., & Ahearn, W. H. (2004). Toward a behavioral analysis of joint attention. The Behavior Analyst, 27(2), 197–207.CrossRefGoogle Scholar
  22. Elsabbagh, M., Volein, A., Holmboe, K., Tucker, L., Csibra, G., Baron-Cohen, S., et al. (2009). Visual orienting in the early broader autism phenotype: Disengagement and facilitation. Journal of Child Psychology and Psychiatry, 50(5), 637–642.CrossRefGoogle Scholar
  23. Escobedo, L., Tentori, M., Quintana, E., Favela, J., & Garcia-Rosas, D. (2014). Using augmented reality to help children with autism stay focused. IEEE Pervasive Computing, 13(1), 38–46.CrossRefGoogle Scholar
  24. Facoetti, A., Ruffino, M., Peru, A., Paganoni, P., & Chelazzi, L. (2008). Sluggish engagement and disengagement of non-spatial attention in dyslexic children. Cortex, 44(9), 1221–1233.CrossRefGoogle Scholar
  25. Farr, W., Yuill, N., & Hinske, S. (2012). An augmented toy and social interaction in children with autism. International Journal of Arts and Technology, 5(2–4), 104–125.CrossRefGoogle Scholar
  26. Farr, W., Yuill, N., & Raffle, H. (2009, April). Collaborative benefits of a tangible interface for autistic children. In Proceedings of the CHI.Google Scholar
  27. Fodstad, J. C., Matson, J. L., Hess, J., & Neal, D. (2009). Social and communication behaviours in infants and toddlers with autism and pervasive developmental disorder-not otherwise specified. Developmental Neurorehabilitation, 12(3), 152–157.CrossRefGoogle Scholar
  28. Gardner, S., & Wolfe, P. (2013). Use of video modeling and video prompting interventions for teaching daily living skills to individuals with autism spectrum disorders: A review. Research and Practice for Persons with Severe Disabilities, 38(2), 73–87.CrossRefGoogle Scholar
  29. Graham, M., Zook, M., & Boulton, A. (2013). Augmented reality in urban places: Contested content and the duplicity of code. Transactions of the Institute of British Geographers, 38(3), 464–479.CrossRefGoogle Scholar
  30. Grandin, T. (2008). Thinking in pictures, expanded edition: My life with autism. New York: Vintage.Google Scholar
  31. Gray, C. A. (1998). Social stories and comic strip conversations with students with Asperger syndrome and high-functioning autism. In E. Schopler, G. Mesibov, & L. Kunce (Eds.), Asperger syndrome or high-functioning autism? (pp. 167–198). New York: Plenum Press.CrossRefGoogle Scholar
  32. Heath, M. A., Sheen, D., Leavy, D., Young, E., & Money, K. (2005). Bibliotherapy: A resource to facilitate emotional healing and growth. School Psychology International, 26(5), 563–580.CrossRefGoogle Scholar
  33. Hedley, N. R. (2003, August). Empirical evidence for advanced geographic visualization interface use. In Proceedings of the International Cartographic Congress, Durban, South Africa.Google Scholar
  34. Heinze-Fry, J. A., & Novak, J. D. (1990). Concept mapping brings long-term movement toward meaningful learning. Science Education, 74(4), 461–472.CrossRefGoogle Scholar
  35. Hobson, R. P., & Lee, A. (1998). Hello and goodbye: A study of social engagement in autism. Journal of Autism and Developmental Disorders, 28(2), 117–127.CrossRefGoogle Scholar
  36. Hornecker, E., & Buur, J. (2006, April). Getting a grip on tangible interaction: A framework on physical space and social interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 437–446). ACM.Google Scholar
  37. Hoyek, N., Di Rienzo, F., Collet, C., Creveaux, T., & Guillot, A. (2014). Hand mental rotation is not systematically altered by actual body position: Laterality judgment versus same–different comparison tasks. Attention, Perception, & Psychophysics, 76(2), 519–526.CrossRefGoogle Scholar
  38. Hwang, G. J., Sung, H. Y., Hung, C. M., Huang, I., & Tsai, C. C. (2012). Development of a personalized educational computer game based on students’ learning styles. Educational Technology Research and Development, 60(4), 623–638.CrossRefGoogle Scholar
  39. Ibanez, L. V., Messinger, D. S., Newell, L., Lambert, B., & Sheskin, M. (2008). Visual disengagement in the infant siblings of children with an autism spectrum disorder (ASD). Autism, 12(5), 473–485.CrossRefGoogle Scholar
  40. Jarrold, C. (2003). A review of research into pretend play in autism. Autism, 7(4), 379–390.CrossRefGoogle Scholar
  41. Kaufmann, H., & Schmalstieg, D. (2003). Mathematics and geometry education with collaborative augmented reality. Computers & Graphics, 27(3), 339–345.CrossRefGoogle Scholar
  42. Kikuchi, Y., Senju, A., Akechi, H., Tojo, Y., Osanai, H., & Hasegawa, T. (2011). Atypical disengagement from faces and its modulation by the control of eye fixation in children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 41(5), 629–645.CrossRefGoogle Scholar
  43. Kinchin, I. M. (2012). Concept mapping and the fundamental problem of moving between knowledge structures. Journal for Educators, Teachers and Trainers, 4(1), 96–106.Google Scholar
  44. Klatzky, R. L., Wu, B., Shelton, D., & Stetten, G. (2008). Effectiveness of augmented-reality visualization versus cognitive mediation for learning actions in near space. ACM Transactions on Applied Perception (TAP). Scholar
  45. Landry, R., & Bryson, S. E. (2004). Impaired disengagement of attention in young children with autism. Journal of Child Psychology and Psychiatry, 45(6), 1115–1122.CrossRefGoogle Scholar
  46. LeGoff, D. B. (2004). Use of LEGO© as a therapeutic medium for improving social competence. Journal of Autism and Developmental Disorders, 34(5), 557–571.CrossRefGoogle Scholar
  47. Lillard, A. S., Lerner, M. D., Hopkins, E. J., Dore, R. A., Smith, E. D., & Palmquist, C. M. (2013). The impact of pretend play on children’s development: A review of the evidence. Psychological Bulletin, 139(1), 1.CrossRefGoogle Scholar
  48. Lo, C.-Y. (2016). Incorporating concept map to design augmented reality learning system in inquiry teaching for natural science courses. National Taichung University of Science and Technology.Google Scholar
  49. Marshall, P. (2007, February). Do tangible interfaces enhance learning? In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (pp. 163–170). ACM.Google Scholar
  50. Matson, J. L. (2009). Applied behavior analysis for children with autism spectrum disorders. New York: Springer.CrossRefGoogle Scholar
  51. Nikopoulos, C. K., & Keenan, M. (2007). Using video modeling to teach complex social sequences to children with autism. Journal of Autism and Developmental Disorders, 37(4), 678–693.CrossRefGoogle Scholar
  52. Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  53. Reynhout, G., & Carter, M. (2006). Social Stories™ for children with disabilities. Journal of Autism and Developmental Disorders, 36(4), 445–469.CrossRefGoogle Scholar
  54. Roberts, V., & Joiner, R. (2007). Investigating the efficacy of concept mapping with pupils with autistic spectrum disorder. British Journal of Special Education, 34(3), 127–135.CrossRefGoogle Scholar
  55. Schwendimann, B. A. (2015). Concept maps as versatile tools to integrate complex ideas: From kindergarten to higher and professional education. Knowledge Management & E-Learning: An International Journal (KM&EL), 7(1), 73–99.Google Scholar
  56. Shelton, B. E. (2003). How augmented reality helps students learn dynamic spatial relationships. Doctoral dissertation. University of Washington.Google Scholar
  57. Shelton, B. E., & Hedley, N. R. (2002). Using augmented reality for teaching earth-sun relationships to undergraduate geography students. In Augmented Reality Toolkit, The First IEEE International Workshop (8 pp.). IEEE.Google Scholar
  58. Shelton, B. E., & Hedley, N. R. (2004). Exploring a cognitive basis for learning spatial relationships with augmented reality. Technology, Instruction, Cognition and Learning, 1(4), 323.Google Scholar
  59. Sherer, M., Pierce, K. L., Paredes, S., Kisacky, K. L., Ingersoll, B., & Schreibman, L. (2001). Enhancing conversation skills in children with autism via video technology: Which is better, “self” or “other” as a model? Behavior Modification, 25(1), 140–158.CrossRefGoogle Scholar
  60. Shipley-Benamou, R., Lutzker, J. R., & Taubman, M. (2002). Teaching daily living skills to children with autism through instructional video modeling. Journal of Positive Behavior Interventions, 4(3), 166–177.CrossRefGoogle Scholar
  61. Sparrow, S. S. (2011). Vineland adaptive behavior scales. In J. S. Kreutzer, J. DeLuca, & B. Caplan (Eds.), Encyclopedia of clinical neuropsychology (pp. 2618–2621). New York: Springer.CrossRefGoogle Scholar
  62. Sucksmith, E., Allison, C., Baron-Cohen, S., Chakrabarti, B., & Hoekstra, R. A. (2013). Empathy and emotion recognition in people with autism, first-degree relatives, and controls. Neuropsychologia, 51(1), 98–105.CrossRefGoogle Scholar
  63. Sun, J. C. Y., & Chen, A. Y. Z. (2016). Effects of integrating dynamic concept maps with interactive response system on elementary school students’ motivation and learning outcome: The case of anti-phishing education. Computers & Education, 102, 117–127.CrossRefGoogle Scholar
  64. Tajeddin, Z., & Tabatabaei, S. (2016). Concept mapping as a reading strategy: Does it scaffold comprehension and recall. The Reading Matrix: An International Online Journal, 16(1), 194–208.Google Scholar
  65. Taylor, B. A., Hoch, H., Potter, B., Rodriguez, A., Spinnato, D., & Kalaigian, M. (2005). Manipulating establishing operations to promote initiations toward peers in children with autism. Research in Developmental Disabilities, 26(4), 385–392.CrossRefGoogle Scholar
  66. van der Geest, J. N., Kemner, C., Camfferman, G., Verbaten, M. N., & van Engeland, H. (2001). Eye movements, visual attention, and autism: A saccadic reaction time study using the gap and overlap paradigm. Biological Psychiatry, 50(8), 614–619.CrossRefGoogle Scholar
  67. Weiss, M. J., & Harris, S. L. (2001). Reaching out, joining. Teaching social skills to young children with autism. Bethesda, MD: Woodbine House.Google Scholar
  68. Wexler, M., Kosslyn, S. M., & Berthoz, A. (1998). Motor processes in mental rotation. Cognition, 68(1), 77–94.CrossRefGoogle Scholar
  69. Wing, L., & Gould, J. (1979). Severe impairments of social interaction and associated abnormalities in children: Epidemiology and classification. Journal of Autism and Developmental Disorders, 9(1), 11–29.CrossRefGoogle Scholar
  70. Wraga, M., Thompson, W. L., Alpert, N. M., & Kosslyn, S. M. (2003). Implicit transfer of motor strategies in mental rotation. Brain and Cognition, 52(2), 135–143.CrossRefGoogle Scholar
  71. Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49.CrossRefGoogle Scholar
  72. Yuill, N., Strieth, S., Roake, C., Aspden, R., & Todd, B. (2007). Brief report: Designing a playground for children with autistic spectrum disorders—Effects on playful peer interactions. Journal of Autism and Developmental Disorders, 37(6), 1192–1196.CrossRefGoogle Scholar
  73. Zaman, B., Abeele, V. V., Markopoulos, P., & Marshall, P. (2012). The evolving field of tangible interaction for children: The challenge of empirical validation. Personal and Ubiquitous Computing, 16(4), 367–378.CrossRefGoogle Scholar

Copyright information

© De La Salle University 2018

Authors and Affiliations

  • I-Jui Lee
    • 1
    Email author
  • Chien-Hsu Chen
    • 2
    • 3
  • Chuan-Po Wang
    • 4
  • Chi-Hsuan Chung
    • 2
  1. 1.Ergonomics and Interaction Design Lab, Department of Industrial DesignNational Taipei University of TechnologyTaipeiTaiwan, ROC
  2. 2.Ergonomics and Interaction Design Lab, Department of Industrial DesignNational Cheng Kung UniversityTainanTaiwan, ROC
  3. 3.Hierarchical Green-Energy Materials (Hi-GEM) Research CenterNational Cheng Kung UniversityTainanTaiwan
  4. 4.Department of Innovative Living DesignOverseas Chinese UniversityTaichungTaiwan

Personalised recommendations