Interactive testbed for research in autism—the SARA project

  • Diana Arellano
  • Reinhold Rauh
  • Benjamin Krautheim
  • Marc Spicker
  • Ulrich Max Schaller
  • Volker Helzle
  • Oliver Deussen
Long Paper


The project Stylized Animations for Research on Autism (SARA) aims to develop a better understanding of the cognitive processes behind emotional categorization in children and adolescents with high-functioning autism spectrum disorder (ASD), in comparison with neurotypically developed peers. To this end, we combine novel real-time non-photorealistic rendering (NPR) algorithms, emotional facial animations, and eye-tracking technologies in a framework that serves as an interactive testbed for empirical research. In this paper, we focus on three experiments that: (1) validate real-time facial animations of virtual characters, (2) evaluate the NPR algorithms to create abstracted facial expressions, and (3) elucidate the relation between eye gaze behavior, ASD and alexithymia (i.e., difficulties in expressing ones emotions). The results show that our animations can be used in the proposed experiments; however, more evaluation is needed regarding the NPR abstractions, especially with individuals with ASD. Finally, even though no correlation was found between gaze behavior, ASD and alexithymia, the study opened several questions that will be addressed in future work.


Autism spectrum disorder Facial animation Eye tracking Stylization 



The SARA project (officially: Impact of Non-Photorealistic Rendering for the understanding of emotional facial expressions by children and adolescents with high-functioning Autism Spectrum Disorders) is funded by DFG - German Research Foundation (AR 892/1-1, DE 620/18-1, RA 764/4-1). Thanks to Kai Goetz for the graphics.


  1. 1.
    Center for Disease Control and Prevention: Facts About ASD. Retrieved April 06, 2015, from Center for Disease Control and Prevention. (2014)
  2. 2.
    Baio, J.: Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2010. Surveill. Summ. 63, 1–21 (2014)Google Scholar
  3. 3.
    Elsabbagh, M., Divan, G., Koh, Y., Kim, Y., Kauchali, S., Marcín, C., Montiel-Nava, C., Patel, V., Paula, C., Wang, C., Yasamy, M., Fombonne, E.: Global prevalence of autism and other pervasive developmental disorders. Autism Res. 5(3), 160–179 (2012)CrossRefGoogle Scholar
  4. 4.
    Saemundsen, E., Magnússon, P., Georgsdóttir, I., Egilsson, E., Rafnsson, V.: Prevalence of autism spectrum disorders in an icelandic birth cohort. BMJ Open. 3(6), e002748 (2013)CrossRefGoogle Scholar
  5. 5.
    Chita-Tegmark, M.: Attention allocation in ASD: a review and meta-analysis of eye-tracking studies. Rev. J. Autism Dev. Disord. 3(3), 209–223 (2016)CrossRefGoogle Scholar
  6. 6.
    Uljarevic, M., Hamilton, A.: Recognition of emotions in autism: a formal meta-analysis. J. Autism Dev. Disord. 43(7), 1517–1526 (2013)CrossRefGoogle Scholar
  7. 7.
    Whyte, E., Smyth, J., Scherf, S.: Designing serious game interventions for individuals with autism. J. Autism Dev. Disord. 45(12), 3820–3831 (2014)CrossRefGoogle Scholar
  8. 8.
    Milne, M., Luerssen, M., Lewis, T., Leibbrandt, R., Powers, D.: Designing and evaluating interactive agents as social skills tutors for children with autism spectrum disorder. In: Perez-Marin, D., Pascual-Nieto, I., (eds.) Conversational Agents and Natural Language Interaction: Techniques and Effective Practices, pp. 23–48 (2011)Google Scholar
  9. 9.
    Grawemeyer, B., Johnson, H., Brosnan, M., Ashwin, E., Benton, L.: Developing an embodied pedagogical agent with and for young people with autism spectrum disorder. In: ITS 2012. LNCS, vol. 7315, pp. 262–267. Springer, Heidelberg (2012)Google Scholar
  10. 10.
    Alcorn, A., Pain, H., Rajendran, G., Smith, T., Lemon, O., Porayska-Pomsta, K., Foster, M.E., Avramides, K., Frauenberger, C., Bernardini, S.: Social communication between virtual characters and children with autism. In: Artificial Intelligence in Education: 15th International Conference, pp. 7–14 (2011)Google Scholar
  11. 11.
    Serret, S., Hun, S., Iakimova, G., Lozada, J., Anastassova, M., Santos, A., Vesperini, S., Askenazy, F.: Facing the challenge of teaching emotions to individuals with low- and high-functioning autism using a new serious game: a pilot study. Mol. Autism 5(37) (2014).
  12. 12.
    Alves, S., Marques, A., Queirós, C., Orvalho, V.: LIFEisGAME prototype: a serious game about emotions for children with autism spectrum disorders. PsychNol. J. 11(3), 191–211 (2013)Google Scholar
  13. 13.
    Rice, L., Wall, C., Fogel, A., Shic, F.: Computer-assisted face processing instruction improves emotion recognition, mentalizing, and social skills in students with ASD. J. Autism Dev. Disord. 45(7), 1–11 (2015)CrossRefGoogle Scholar
  14. 14.
    Tanaka, J.W., Wolf, J.M., Klaiman, C., Koenig, K., Cockburn, J., Herlihy, L., Brown, C., Stahl, S., Kaiser, M.D., Schultz, R.T.: Using computerized games to teach face recognition skills to children with autism spectrum disorder: the let’s face it! program. J. Child Psychol. Psychiatry 51(8), 944–952 (2010)CrossRefGoogle Scholar
  15. 15.
    American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders: DSM-5, 5th edn. American Psychiatric Publishing, Arlington (2013)Google Scholar
  16. 16.
    World Health Organization: The ICD-10 Classification of Mental and Behavioural Disorders: Diagnostic Criteria for Research, 10th edn. Geneva: World Health Organization, Switzerland (1993)Google Scholar
  17. 17.
    Krautheim, B.: The influence of alexithymia on gaze behavior patterns in male adolescents with autism spectrum disorder. Master’s thesis, Albert-Ludwigs-Universität Freiburg. Institut für Psychologie (2015)Google Scholar
  18. 18.
    Kirchner, J.C., Hatri, A., Heekeren, H.R., Dziobek, I.: Autistic symptomatology, face processing abilities, and eye fixation patterns. J. Autism Dev. Disord. 41(2), 158–167 (2011)CrossRefGoogle Scholar
  19. 19.
    Klin, A., Jones, W., Schultz, R., Volkmar, F., Cohen, D.: Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Arch. Gen. Psychiatry 59(9), 809–816 (2002)CrossRefGoogle Scholar
  20. 20.
    Senju, A., Johnson, M.H.: Atypical eye contact in autism: models, mechanisms and development. Neurosci. Biobehav. Rev. 33(8), 1204–1214 (2009)CrossRefGoogle Scholar
  21. 21.
    Sawyer, A.C.P., Williamson, P., Young, R.L.: Can gaze avoidance explain why individuals with asperger’s syndrome can’t recognise emotions from facial expressions? J. Autism Dev. Disord. 42(4), 606–618 (2012)CrossRefGoogle Scholar
  22. 22.
    Bird, G., Press, C., Richardson, D.: The role of alexithymia in reduced eye-fixation in autism spectrum conditions. J. Autism Dev. disord. 41(11), 1556–1564 (2011)CrossRefGoogle Scholar
  23. 23.
    Bird, G., Cook, R.: Mixed emotions: the contribution of alexithymia to the emotional symptoms of autism. Transl. Psychiatry 3, e285 (2013)CrossRefGoogle Scholar
  24. 24.
    Rauh, R., Schaller, U.: Categorical perception of emotional facial expressions in video clips with natural and artificial actors: a pilot study. Tech. Rep. ALU-KJPP-2009-001, University of Freiburg (2009)Google Scholar
  25. 25.
    Arellano, D., Helzle, V., Schaller, U., Rauh, R.: Animated faces, abstractions and autism. In: Intelligent Virtual Agents 2014, Lecture Notes in Computer Science, vol. 8637, pp. 22–25 (2014)Google Scholar
  26. 26.
    Helzle, V., Biehn, C., Schlömer, T., Linner, F.: Adaptable setup for performance driven facial animation. In: ACM SIGGRAPH 2004 Sketches, p. 54 (2004)Google Scholar
  27. 27.
    Ekman, P., Friesen, W., Hager, J.: The Facial Action Coding System. Weidenfeld & Nicolson, London (2002)Google Scholar
  28. 28.
    Peirce, J.W.: Generating stimuli for neuroscience using PsychoPy. Front. Neuroinform. 2(10) (2008)Google Scholar
  29. 29.
    Arellano, D., Schaller, U.M., Rauh, R., Helzle, V., Spicker, M., Deussen, O.: Intelligent Virtual Agents: 15th International Conference, IVA 2015, Delft, The Netherlands, August 26–28, 2015, Proceedings, chap. On the Trail of Facial Processing in Autism Spectrum Disorders, pp. 432–441. Springer International Publishing, Cham (2015)Google Scholar
  30. 30.
    Law Smith, M.J., Montagne, B., Perrett, D.I., Gill, M., Gallagher, L.: Detecting subtle facial emotion recognition deficits in high-functioning autism using dynamic stimuli of varying intensities. Neuropsychologia 48(9), 2777–2781 (2010)CrossRefGoogle Scholar
  31. 31.
    Lozier, L.M., Vanmeter, J.W., Marsh, A.: Impairments in facial affect recognition associated with autism spectrum disorders: A meta-analysis. Dev. Psychopathol. 26(4 Pt 1), 933–945 (2014)CrossRefGoogle Scholar
  32. 32.
    Gooch, B., Gooch, A.: Non-Photorealistic Rendering. A.K. Peters, Natick (2001)zbMATHGoogle Scholar
  33. 33.
    DeCarlo, D., Santella, A.: Stylization and abstraction of photographs. ACM Trans. Graph. 21(3), 769–776 (2002)CrossRefGoogle Scholar
  34. 34.
    Gooch, B., Reinhard, E., Gooch, A.: Human facial illustrations: creation and psychophysical evaluation. ACM Trans. Graph. 23(1), 27–44 (2004)CrossRefGoogle Scholar
  35. 35.
    Spicker, M., Arellano, D., Schaller, U., Rauh, R., Helzle, V., Deussen, O.: Emotion recognition in autism spectrum disorder: Does stylization help? In: Proceedings of the ACM Symposium on Applied Perception, SAP ’16, pp. 97–104. ACM (2016)Google Scholar
  36. 36.
    Sifneos, P.: The prevalence of alexithymic characteristics in psychosomatic patients. Psychother. Psychosom. 22, 255–262 (1973)CrossRefGoogle Scholar
  37. 37.
    Weiß, R.H.: Grundintelligenztest Skala 2 - Revision - (CFT 20-R). Hogrefe, Göttingen (2006)Google Scholar
  38. 38.
    Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., Van de Weijer, J. (eds.): Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford University Press, Oxford (2011)Google Scholar
  39. 39.
    Kupfer, J., Brosig, B., Brähler, E.: TAS-26: Toronto-Alexithymie-Skala-26 (deutsche Version). Manual. Hogrefe Verlag GmbH & Co, Göttingen (2001)  Google Scholar
  40. 40.
    Bermond, B., Vorst, H.C., Vingerhoets, A.J., Gerritsen, W.: The amsterdam alexithymia scale: its psychometric values and correlations with other personality traits. Psychother. Psychosom. 68(5), 241–251 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Diana Arellano
    • 1
  • Reinhold Rauh
    • 2
  • Benjamin Krautheim
    • 2
  • Marc Spicker
    • 3
  • Ulrich Max Schaller
    • 2
  • Volker Helzle
    • 1
  • Oliver Deussen
    • 3
  1. 1.Animationsinstitut at Filmakademie Baden-WuerttembergLudwigsburgGermany
  2. 2.Department of Child and Adolescent Psychiatry, Psychotherapy, and Psychosomatics, Medical Center – University of Freiburg, Faculty of MedicineUniversity of FreiburgFreiburgGermany
  3. 3.Chair of Computer Graphics and Media InformaticsUniversity of KonstanzKonstanzGermany

Personalised recommendations