Longitudinal Impact of Autonomous Robot-Mediated Joint Attention Intervention for Young Children with ASD

  • Zhi ZhengEmail author
  • Guangtao Nie
  • Amy Swanson
  • Amy Weitlauf
  • Zachary Warren
  • Nilanjan Sarkar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9979)


Literature suggests that a robot is able to capture attention and elicit positive social communication behaviors in many children with ASD. However, there are few studies reported regarding the longitudinal impact of autonomous robot-mediated interventions. In this paper, we introduce a new autonomous robotic system for teaching children with ASD joint attention skills, which is among the core developmental impairments in ASD. This system automatically tracks the participant’s behavior during intervention and adaptively adjusts the interaction pattern based on the participant’s performance. Based on this system, we report a longitudinal robot-mediated joint attention intervention on a user study for 6 children with ASD. First, four sessions of one-target interventions were conducted for each participant. Then we tested their joint attention skills after 8 months in two sessions on: (1) response to one target; (2) response to two targets. The results showed that this autonomous robotic system was able to elicit improved one-target joint attention performance in young children with ASD over the course of 8 months. The result also suggested that the joint attention skills that the participants practiced in the one-target sessions might help them interact in a more difficult task. The robot also attracted the participants’ attention constantly during this long term intervention.


Robot-mediated joint attention skills training Children with ASD Longitudinal study 



This study was partially supported by: the Hobbs Grant from the Vanderbilt Kennedy Center, a Vanderbilt University Innovation and Discovery in Engineering and Science (IDEAS) grant, National Science Foundation Grant 1264462, and the National Institute of Health Grants 1R01MH091102-01A1and 5R21MH103518-02. This work also got core supports from NICHD (P30HD15052) and NCATS (UL1TR000445-06).


  1. 1.
    Autism Spectrum Disorders Prevalence Rate. Autism Speaks and Center for Disease Control (CDC) (2011)Google Scholar
  2. 2.
    Ganz, M.L.: The lifetime distribution of the incremental societal costs of autism. Arch. Pediatr. Adolesc. Med. Am. Med. Assoc. 161, 343–349 (2007)CrossRefGoogle Scholar
  3. 3.
    Warren, Z.E., Stone, W.L.: Best practices: early diagnosis and psychological assessment. In: Amaral, D., Geschwind, D., Dawson, G. (eds.) Autism Spectrum Disorders, pp. 1271–1282. Oxford University Press, New York (2011)Google Scholar
  4. 4.
    Zheng, Z., Fu, Q., Zhao, H., Swanson, A., Weitlauf, A., Warren, Z., Sarkar, N.: Design of a computer-assisted system for teaching attentional skills to toddlers with ASD. In: Antona, M., Stephanidis, C. (eds.) UAHCI 2015. LNCS, vol. 9177, pp. 721–730. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  5. 5.
    Weir, S., Emanuel, R.: Using LOGO to catalyse communication in an autistic child. Department of Artificial Intelligence, University of Edinburgh (1976)Google Scholar
  6. 6.
    Robins, B., Dautenhahn, K., Dubowski, J.: Does appearance matter in the interaction of children with autism with a humanoid robot? Interac. Stud. 7, 509–542 (2006)CrossRefGoogle Scholar
  7. 7.
    Pierno, A.C., Mari, M., Lusher, D., Castiello, U.: Robotic movement elicits visuomotor priming in children with autism. Neuropsychologia 46, 448–454 (2008)CrossRefGoogle Scholar
  8. 8.
    Robins, B., Dautenhahn, K., Boekhorst, R., Billard, A.: Robotic assistants in therapy and education of children with autism: can a small humanoid robot help encourage social interaction skills? Univ. Access Inf. Soc. 4, 105–120 (2005)CrossRefGoogle Scholar
  9. 9.
    Kozima, H., Nakagawa, C., Yasuda, Y.: Children–robot interaction: a pilot study in autism therapy. Prog. Brain Res. 164, 385–400 (2007)CrossRefGoogle Scholar
  10. 10.
    Feil-Seifer, D., Matarić, M.J.: Toward socially assistive robotics for augmenting interventions for children with autism spectrum disorders. In: Experimental Robotics, pp. 201–210. Springer, Heidelberg (2009)Google Scholar
  11. 11.
    Zheng, Z., Das, S., Young, E.M., Swanson, A., Warren, Z., Sarkar, N.: Autonomous robot-mediated imitation learning for children with autism. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 2707–2712. IEEE (2014)Google Scholar
  12. 12.
    Fujimoto, I., Matsumoto, T., De Silva, P.S., Kobayashi, M., Higashi, M.: Study on an assistive robot for improving imitation skill of children with autism. In: Ge, S.S., Li, H., Cabibihan, J.-J., Tan, Y.K. (eds.) ICSR 2010. LNCS, vol. 6414, pp. 232–242. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.
    Bekele, E.T., Lahiri, U., Swanson, A.R., Crittendon, J.A., Warren, Z.E., Sarkar, N.: A step towards developing adaptive robot-mediated intervention architecture (ARIA) for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 289–299 (2013)CrossRefGoogle Scholar
  14. 14.
    Warren, Z.E., Zheng, Z., Swanson, A.R., Bekele, E., Zhang, L., Crittendon, J.A., Weitlauf, A.F., Sarkar, N.: Can robotic interaction improve joint attention skills? J. Autism Dev. Disord. 45(11), 3726–3734 (2015)Google Scholar
  15. 15.
    Wainer, J., Robins, B., Amirabdollahian, F., Dautenhahn, K.: Using the humanoid robot KASPAR to autonomously play triadic games and facilitate collaborative play among children with autism. IEEE Trans. Auton. Mental Deve. 6, 183–199 (2014)CrossRefGoogle Scholar
  16. 16.
    Poon, K.K., Watson, L.R., Baranek, G.T., Poe, M.D.: To what extent do joint attention, imitation, and object play behaviors in infancy predict later communication and intellectual functioning in ASD? J. Autism Dev. Disord. (2011), 1–11. doi: 10.1007/s10803-011-1349-z
  17. 17.
    Zheng, Z., Zhang, L., Bekele, E., Swanson, A., Crittendon, J., Warren, Z., Sarkar, N.: Impact of robot-mediated interaction system on joint attention skills for children with autism In: 2013 IEEE International Conference on Rehabilitation Robotics (ICORR). IEEE (2013)Google Scholar
  18. 18.
  19. 19.
    Kajopoulos, J., Wong, A.H.Y., Yuen, A.W.C., Dung, T.A., Kee, T.Y., Wykowska, A.: robot-assisted training of joint attention skills in children diagnosed with autism. In: Tapus, A., et al. (eds.) ICSR 2015. LNCS, vol. 9388, pp. 296–305. Springer, Heidelberg (2015)Google Scholar
  20. 20.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 532–539. IEEE (2013)Google Scholar
  21. 21.
    Diagnostic and Statistical Manual of Mental Disorders: Quick reference to the Diagnostic Criteria from DSM-IV-TR. American Psychiatric Association, Washington D.C. (2000)Google Scholar
  22. 22.
    Lord, C., Rutter, M., DiLavore, P., Risi, S., Gotham, K., Bishop, S.: Autism Diagnostic Observation Schedule–2nd edition (ADOS-2). Western Psychological Services, Torrance (2012)Google Scholar
  23. 23.
    Mullen, E.M.: Mullen Scales of Early Learning. Pearson San Antonio, TX (1995)Google Scholar
  24. 24.
    Constantino, J.N., Gruber, C.P.: The Social Responsiveness Scale. Western Psychological Services, Los Angeles (2002)Google Scholar
  25. 25.
    Rutter, M., Bailey, A., Lord, C.: The Social Communication Questionnaire. Western Psychological Services, Los Angeles (2010)Google Scholar
  26. 26.
    Demchak, M.: Response prompting and fading methods: a review. Am. J. Ment. Retard. 94, 603–615 (1990)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Zhi Zheng
    • 1
    Email author
  • Guangtao Nie
    • 1
  • Amy Swanson
    • 2
  • Amy Weitlauf
    • 2
  • Zachary Warren
    • 2
  • Nilanjan Sarkar
    • 1
    • 3
  1. 1.Electrical Engineering and Computer Science DepartmentNashvilleUSA
  2. 2.Vanderbilt Kennedy Center Treatment and Research Institute for Autism Spectrum DisorderNashvilleUSA
  3. 3.Department of Mechanical EngineeringVanderbilt UniversityNashvilleUSA

Personalised recommendations