‘Emotiplay’: a serious game for learning about emotions in children with autism: results of a cross-cultural evaluation

Abstract

Children with autism spectrum conditions (ASC) experience difficulties recognizing others’ emotions and mental states. It has been shown that serious games (SG) can produce simplified versions of the socio-emotional world. The current study performed a cross-cultural evaluation (in the UK, Israel and Sweden) of Emotiplay’s SG, a system aimed to teach emotion recognition (ER) to children with ASC in an entertaining, and intrinsically motivating way. Participants were 6–9 year olds with high functioning ASC who used the SG for 8–12 weeks. Measures included face, voice, body, and integrative ER tasks, as well as parent-reported level of autism symptoms, and adaptive socialization. In the UK, 15 children were tested before and after using the SG. In Israel (n = 38) and Sweden (n = 36), children were randomized into a SG or a waiting list control group. In the UK, results revealed that 8 weeks of SG use significantly improved participants’ performance on ER body language and integrative tasks. Parents also reported their children improved their adaptive socialization. In Israel and Sweden, participants using the SG improved significantly more than controls on all ER measures. In addition, parents in the Israeli SG group reported their children showed reduced autism symptoms after using the SG. In conclusion, Emotiplay’s SG is an effective and motivating psycho-educational intervention, cross-culturally teaching ER from faces, voices, body language, and their integration in context to children with high functioning ASC. Local evidence was found for more generalized gains to socialization and reduced autism symptoms.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. 1.

    We prefer the term autism spectrum conditions (ASC) over the more common Autism Spectrum Disorders (ASD), as it views the condition comprehensively, acknowledging strengths as well as difficulties.

  2. 2.

    With the exception of FEFA, which is also available in German, Finnish, and Swedish.

References

  1. 1.

    Golan O, Sinai-Gavrilov Y, Baron-Cohen S (2015) The Cambridge mindreading face-voice battery for children (CAM-C): complex emotion recognition in children with and without autism spectrum conditions. Mol Autism 6:22

    Article  PubMed  PubMed Central  Google Scholar 

  2. 2.

    Kuusikko S, Haapsamo H, Jansson-Verkasalo E, Hurtig T, Mattila M-L, Ebeling H, Jussila K, Bölte S, Moilanen I (2009) Emotion recognition in children and adolescents with autism spectrum disorders. J Autism Dev Disord 39:938–945

    Article  PubMed  Google Scholar 

  3. 3.

    Uljarevic M, Hamilton A (2012) Recognition of emotions in autism: a formal meta-analysis. J Autism Dev Disord 43(7):1517–1526

    Article  Google Scholar 

  4. 4.

    Hobson RP (1986) The autistic child’s appraisal of exprssions of emotions. J Child Psychol Psychiatry 27:321–342

    CAS  Article  PubMed  Google Scholar 

  5. 5.

    Deruelle C, Rondan C, Gepner B, Tardif C (2004) Spatial frequency and face processing in children with autism and asperger syndrome. J Autism Dev Disord 34:199–210

    Article  PubMed  Google Scholar 

  6. 6.

    Golan O, Baron-Cohen S, Hill J (2006) The Cambridge mindreading (CAM) face-voice battery: testing complex emotion recognition in adults with and without Asperger syndrome. J Autism Dev Disord 36:169–183

    Article  PubMed  Google Scholar 

  7. 7.

    Boucher J, Lewis V, Collis GM (2000) Voice processing abilities in children with autism, children with specific language impairments, and young typically developing children. J Child Psychol Psychiatry 41:847–857

    CAS  Article  PubMed  Google Scholar 

  8. 8.

    Golan O, Baron-Cohen S, Hill JJ, Rutherford MD (2007) The “Reading the Mind in the Voice” test-revised: a study of complex emotion recognition in adults with and without autism spectrum conditions. J Autism Dev Disord 37:1096–1106

    Article  PubMed  Google Scholar 

  9. 9.

    Grèzes J, Wicker B, Berthoz S, de Gelder B (2009) A failure to grasp the affective meaning of actions in autism spectrum disorder subjects. Neuropsychologia 47:1816–1825

    Article  PubMed  Google Scholar 

  10. 10.

    Philip RCM, Whalley HC, Stanfield AC, Sprengelmeyer R, Santos IM, Young AW, Atkinson AP, Calder AJ, Johnstone EC, Lawrie SM, Hall J (2010) Deficits in facial, body movement and vocal emotional processing in autism spectrum disorders. Psychol Med 40:1919–1929

    CAS  Article  PubMed  Google Scholar 

  11. 11.

    Capps L, Yirmiya N, Sigman M (1992) Understanding of simple and complex emotions in non-retarded children with autism. J Child Psychol Psychiatry 35:1169–1182

    Article  Google Scholar 

  12. 12.

    Golan O, Baron-Cohen S, Golan Y (2008) The “Reading the Mind in Films” Task [child version]: complex emotion and mental state recognition in children with and without autism spectrum conditions. J Autism Dev Disord 38:1534–1541

    Article  PubMed  Google Scholar 

  13. 13.

    Silverman LB, Bennetto L, Campana E, Tanenhaus MK (2010) Speech-and-gesture integration in high functioning autism. Cognition 115:380–393

    Article  PubMed  Google Scholar 

  14. 14.

    American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Association, Arlington

    Google Scholar 

  15. 15.

    Young RL, Posselt M (2012) Using the transporters DVD as a learning tool for children with autism spectrum disorders (ASD). J Autism Dev Disord 42:984–991

    Article  PubMed  Google Scholar 

  16. 16.

    Hudepohl MB, Robins DL, King TZ, Henrich CC (2015) The role of emotion perception in adaptive functioning of people with autism spectrum disorders. Autism 19:107–112

    Article  PubMed  Google Scholar 

  17. 17.

    Baron-Cohen S (1995) Mindblindness: an essay on autism and theory of mind. MIT Press/Bradford Books, Boston

    Google Scholar 

  18. 18.

    Jones W, Klin A (2013) Attention to eyes is present but in decline in 2–6-month-old infants later diagnosed with autism. Nature 504:427–431

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  19. 19.

    Joens EJH, Gliga T, Bedford R, Charman T, Johnson MH (2014) Developmental pathways to autism: a review of prospective studies of infants at risk. Neurosci Biobehav Rev 39:1–33

    Article  Google Scholar 

  20. 20.

    Landa RJ, Holman KC, Garrett-Mayer E (2007) Social and communication development in toddlers with early and later diagnosis of autism spectrum disorders. Arch Gen Psychiatry 64:853–864

    Article  PubMed  Google Scholar 

  21. 21.

    Johnson MH, Griffin R, Csibra G, Halit H, Farroni T, de Haan M, Tucker LA, Baron-Cohen S, Richards JE (2005) The emergence of the social brain network: evidence from typical and atypical development. Dev Psychopathol 17:599–619

    Article  PubMed  PubMed Central  Google Scholar 

  22. 22.

    Dawson G (2008) Early behavioral intervention, brain plasticity, and the prevention of autism spectrum disorder. Dev Psychopathol 20:775–803

    Article  PubMed  Google Scholar 

  23. 23.

    Howlin P (1998) Psychological and educational treatments for autism. J Child Psychol Psychiatry 39:307–322

    CAS  Article  PubMed  Google Scholar 

  24. 24.

    Bowler DM, Strom E (1998) Elicitation of first-order’theory of mind in children with autism. Autism 2:33–44

    Article  Google Scholar 

  25. 25.

    Fisher N, Happé F (2005) A training study of theory of mind and executive function in children with autistic spectrum disorders. J Autism Dev Disord 35:757–771

    Article  PubMed  Google Scholar 

  26. 26.

    Hadwin J, Baron-Cohen S, Howlin P, Hill K (1996) Can we teach children with autism to understand emotions, belief, or pretence? Dev Psychopathol 8:345

    Article  Google Scholar 

  27. 27.

    Bauminger N (2002) The facilitation of social-emotional understanding and social interaction in high-functioning children with autism: intervention outcomes. J Autism Dev Disord 32:283–298

    Article  PubMed  Google Scholar 

  28. 28.

    Solomon M, Goodlin-Jones BL, Anders TF (2004) A social adjustment enhancement intervention for high functioning autism, Asperger’s syndrome, and pervasive developmental disorder NOS. J Autism Dev Disord 34:649–668

    Article  PubMed  Google Scholar 

  29. 29.

    Zeeland SV, Ashley A, Dapretto M, Ghahremani DG, Poldrack RA, Bookheimer SY (2010) Reward processing in autism. Autism Res 3:53–67

    Google Scholar 

  30. 30.

    Koegel RL, Vernon TW, Koegel LK (2009) Improving social initiations in young children with autism using reinforcers with embedded social interactions. J Autism Dev Disord 39:1240–1251

    Article  PubMed  PubMed Central  Google Scholar 

  31. 31.

    Attwood T (2003) Learning and behavior problems in Asperger syndrome. In: Prior MR (ed) Learning and behavior problems in Asperger syndrome. Guilford Press, New York, London, pp 126–147

    Google Scholar 

  32. 32.

    Golan O, Ashwin E, Granader Y, McClintock S, Day K, Leggett V, Baron-Cohen S (2010) Enhancing emotion recognition in children with autism spectrum conditions: an intervention using animated vehicles with real emotional faces. J Autism Dev Disord 40:269–279

    Article  PubMed  Google Scholar 

  33. 33.

    Baron-Cohen S (2002) The extreme male brain theory of autism. Trends Cogn Sci 6:248–254

    Article  PubMed  Google Scholar 

  34. 34.

    Baron-Cohen S (2006) The hyper-systemizing, assortative mating theory of autism. Prog Neuropsychopharmacol Biol Psychiatry 30:865–872

    Article  PubMed  Google Scholar 

  35. 35.

    Whyte EM, Smyth JM, Scherf KS (2015) Designing serious game interventions for individuals with autism. J Autism Dev Disord 45:3820–3831

    Article  PubMed  Google Scholar 

  36. 36.

    Baranowski T, Buday R, Thompson DI, Baranowski J (2008) Playing for real: video games and stories for health-related behavior change. Am J Prev Med 34:74–82

    Article  PubMed  PubMed Central  Google Scholar 

  37. 37.

    Kapp KM (2012) The gamification of learning and instruction: game-based methods and strategies for training and education. Pfeiffer, San Francisco

    Google Scholar 

  38. 38.

    Bölte S, Golan O, Goodwin MS, Zwaigenbaum L (2010) What can innovative technologies do for autism spectrum disorders? Autism 14:155–159

    Article  PubMed  Google Scholar 

  39. 39.

    Baron-Cohen S, Golan O, Wheelwright S, Hill JJ (2004) Mind reading: the interactive guide to emotions. Jessica Kingsley Limited, London

    Google Scholar 

  40. 40.

    Bölte S, Feineis-Matthews S, Leber S, Dierks T, Hubl D, Poustka F (2002) The development and evaluation of a computer-based program to test and to teach the recognition of facial affect. Int J Circumpolar Health 61(2):61–68

    PubMed  Google Scholar 

  41. 41.

    Bölte S, Ciaramidaro A, Schlitt S, Hainz D, Kliemann D, Beyer A, Poustka F, Freitag C, Walter H (2015) Training-induced plasticity of the social brain in autism spectrum disorder. Br J Psychiatry 207:149–157

    Article  PubMed  Google Scholar 

  42. 42.

    Hopkins IM, Gower MW, Perez TA, Smith DS, Amthor FR, Wimsatt FC, Biasini FJ (2011) Avatar assistant: improving social skills in students with an ASD through a computer-based intervention. J Autism Dev Disord 41:1543–1555

    Article  PubMed  Google Scholar 

  43. 43.

    Silver M, Oakes P (2001) Evaluation of a new computer intervention to teach people with autism or Asperger syndrome to recognize and predict emotions in others. Autism 5:299–316

    CAS  Article  PubMed  Google Scholar 

  44. 44.

    Tanaka JW, Wolf JM, Klaiman C, Koenig K, Cockburn J, Herlihy L, Brown C, Stahl S, Kaiser MD, Schultz RT (2010) Using computerized games to teach face recognition skills to children with autism spectrum disorder: the Let’s Face It! program. J Child Psychol Psychiatry 51:944–952

    Article  PubMed  Google Scholar 

  45. 45.

    O’Reilly H, Pigat D, Fridenson S, Berggren S, Tal S, Golan O, Bölte S, Baron-Cohen S, Lundqvist D (2015) The EU-emotion stimulus set: a validation study. Behav Res Methods 48(2):567–576

    Article  Google Scholar 

  46. 46.

    Lassalle A, Pigat D, O’Reilly H, Berggen S, Fridenson-Hayo S, Tal S, Elfström S, Råde A, Golan O, Bölte S, Baron-Cohen S, Lundqvist D The EU-emotion voice database (submitted)

  47. 47.

    Fridenson-hayo S, Berggren S, Lassalle A, Tal S, Pigat D, Bölte S, Baron-Cohen S, Golan O (2016) Basic and complex emotion recognition in children with autism: cross cultural findings. Mol Autism 7(1):52

    Article  PubMed  PubMed Central  Google Scholar 

  48. 48.

    Wechsler D (2011) Wechsler abbreviated scale of intelligence, 2nd edn. TX, Pearson

    Google Scholar 

  49. 49.

    Wechsler D (2003) Wechsler intelligence scale for children-fourth edition. Psychological Corporation, San Antonio

    Google Scholar 

  50. 50.

    Wechsler D (2002) Wechsler primary and preschool scale of intelligence-third edition. Harcourt Assessment, San Antonio

    Google Scholar 

  51. 51.

    Constantino JN, Gruber C (2012) The social responsiveness scale—second edition (SRS-2). Western Psychological Services, Torrance

    Google Scholar 

  52. 52.

    Bölte S, Poustka F, Constantino JN (2008) Assessing autistic traits: cross-cultural validation of the social responsiveness scale (SRS). Autism Res 1:354–363

    Article  PubMed  Google Scholar 

  53. 53.

    Sparrow SS, Cicchetti DV, Balla DA (2005) Vineland adaptive behavior scales, 2nd edn. American Guidance Service, Circle Pines

    Google Scholar 

  54. 54.

    Klin A, Saulnier CA, Sparrow SS, Cicchetti DV, Volkmar FR, Lord C (2007) Social and communication abilities and disabilities in higher functioning individuals with autism spectrum disorders: the Vineland and the ADOS. J Autism Dev Disord 37:748–759

    Article  PubMed  Google Scholar 

  55. 55.

    American Psychiatric Association (1994) Diagnostic and statistical manual of mental disorders, 4th edn. American Psychiatric Association, Washington, DC

    Google Scholar 

  56. 56.

    World Health Organisation (1994) ICD-10—international classification of diseases, 10th edn. World Health Organization, Geneva

    Google Scholar 

  57. 57.

    Lord C, Rutter M, DiLavore P, Risi S, Gotham K, Bishop SL (2012) Autism diagnostic observation schedule, 2nd edn. Western Psychological Services, Los Angeles

    Google Scholar 

  58. 58.

    Whalen C, Moss D, Ilan AB, Vaupel M, Fielding P, Macdonald K, Cernich S, Symon J (2010) Efficacy of teachtown: basics computer-assisted intervention for the intensive comprehensive autism program in Los Angeles unified school district. Autism 14:179–197

    Article  PubMed  Google Scholar 

  59. 59.

    Lacava PG, Golan O, Baron-Cohen S, Smith Myles B (2007) Using assistive technology to teach emotion recognition to students with Asperger syndrome: a pilot study. Remedial Spec Educ 28:174–181

    Article  Google Scholar 

  60. 60.

    Grynszpan O, Weiss PL, Perez-Diaz F, Gal E (2014) Innovative technology-based interventions for autism spectrum disorders: a meta-analysis. Autism 18:346–361

    Article  PubMed  Google Scholar 

  61. 61.

    de Gelder B, Van den Stock J (2011) Real faces, real emotions: perceiving facial expressions in naturalistic contexts of voices, bodies and scenes. In: Calder JA, Rhodes G, Johnson MH, Haxby JV (eds) The handbook of face perception. Oxford University Press, Oxford, pp 535–550

    Google Scholar 

  62. 62.

    Herba C, Phillips M (2004) Annotation: development of facial expression recognition from childhood to adolescence: behavioural and neurological perspectives. J Child Psychol Psychiatry 45:1185–1198

    Article  PubMed  Google Scholar 

  63. 63.

    Centelles L, Assaiante C, Etchegoyhen K, Bouvard M, Schmitz C (2012) From action to interaction: exploring the contribution of body motion cues to social understanding in typical development and in autism spectrum disorders. J Autism Dev Disord 43:1140–1150

    Article  Google Scholar 

  64. 64.

    Scherer KR, Ladd DR, Silverman KEA (1984) Vocal cues to speaker affect: testing two models. J Acoust Soc Am 76:1346

    Article  Google Scholar 

  65. 65.

    O’Neill M, Jones RSP (1997) Sensory-Perceptual abnormalities in autism: a case for more research? J Autism Dev Disord 27:283–293

    Article  PubMed  Google Scholar 

  66. 66.

    Yoo HJ, Bahn G, Cho IH, Kim EK, Kim JH, Min JW, Lee WH, Seo JS, Jun SS, Bong G, Cho S, Shin MS, Kim BN, Kim JW, Park S, Laugeson EA (2014) A randomized controlled trial of the korean version of the PEERS® parent-assisted social skills training program for teens with ASD. Autism Res 7:145–161

    Article  PubMed  Google Scholar 

  67. 67.

    Lai M-C, Lombardo MV, Baron-Cohen S (2014) Autism. Lancet 383:896–910

    Article  PubMed  Google Scholar 

  68. 68.

    Grelotti DJ, Kaptchuk TJ (2011) Placebo by proxy. BMJ 343:d4345

    Article  PubMed  PubMed Central  Google Scholar 

  69. 69.

    Whalley B, Hyland ME (2013) Placebo by proxy: the effect of parents’ beliefs on therapy for children’s temper tantrums. J Behav Med 36:341–346

    Article  PubMed  Google Scholar 

  70. 70.

    Sandler AD, Bodfish JW (2000) Placebo effects in autism: lessons from secretin. J Dev Behav Pediatr 21:347–350

    CAS  Article  PubMed  Google Scholar 

  71. 71.

    Curie A, Yang K, Kirsch I, Gollub RL, Des Portes V, Kaptchuk TJ, Jensen KB (2015) Placebo responses in genetically determined intellectual disability: a meta-analysis. PLoS ONE 10:1–16

    Article  Google Scholar 

  72. 72.

    Guastella AJ, Gray KM, Rinehart NJ, Alvares GA, Tonge BJ, Hickie IB, Keating CM, Cacciotti-Saija C, Einfeld SL (2015) The effects of a course of intranasal oxytocin on social behaviors in youth diagnosed with autism spectrum disorders: a randomized controlled trial. J Child Psychol Psychiatry Allied Discip 56:444–452

    Article  Google Scholar 

  73. 73.

    Pickles A, Le Couteur A, Leadbitter K, Salomone E, Cole-Fletcher R, Tobin H, Gammer I, Lowry J, Vamvakas G, Byford S, Aldred C, Slonims V, McConachie H, Howlin P, Parr JR, Charman T, Green J (2016) Parent-mediated social communication therapy for young children with autism (PACT): long-term follow-up of a randomised controlled trial. Lancet 388:2501–2509

    Article  PubMed  PubMed Central  Google Scholar 

  74. 74.

    Callenmark B, Kjellin L, Rönnqvist L, Bölte S (2014) Explicit versus implicit social cognition testing in autism spectrum disorder. Autism 18:684–693

    Article  PubMed  PubMed Central  Google Scholar 

  75. 75.

    Grzadzinski R, Carr T, Colombi C, McGuire K, Dufek S, Pickles A, Lord C (2016) Measuring Changes in social communication behaviors: preliminary development of the Brief Observation of Social Communication Change (BOSCC). J Autism Dev Disord 46:2464–2479

    Article  PubMed  Google Scholar 

  76. 76.

    Rimland B (1965) Infantile autism: the syndrome and its implications for a neural theory of behavior. Methuen, London

    Google Scholar 

  77. 77.

    Kouo JL, Egel AL (2016) The effectiveness of interventions in teaching emotion recognition to children with autism spectrum disorder. Rev J Autism Dev Disord 3:254–265

    Article  Google Scholar 

  78. 78.

    Gev T, Rosenan R, Golan O (2016) Unique effects of the transporters animated series and of parental support on emotion recognition skills of children with ASD: results of a randomized controlled trial. Autism Resrearch. doi:10.1002/aur.1717

    Google Scholar 

  79. 79.

    Daley TC (2002) The need for cross-cultural research on the pervasive developmental disorders. Transcult Psychiatry 39:531–550

    Article  Google Scholar 

  80. 80.

    Ravindran N, Myers BJ (2012) Cultural influences on perceptions of health, illness, and disability: a review and focus on autism. J Child Fam Stud 21:311–319

    Article  Google Scholar 

  81. 81.

    Keen D, Couzens D, Muspratt S, Rodger S (2010) The effects of a parent-focused intervention for children with a recent diagnosis of autism spectrum disorder on parenting stress and competence. Res Autism Spectr Disord 4:229–241

    Article  Google Scholar 

  82. 82.

    Boot WR, Blakely DP, Simons DJ (2011) Do action video games improve perception and cognition? Front Psychol 2:226

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the contribution of our research teams at all sites. In the UK: Kirsty Macmillan, Nisha Hickin, Mathilde Matthews, and Kate Lindley Baron-Cohen. In Sweden: Sigrid Elfström, Nina Milenkovic, and Anna Råde. In Israel: Michal Landa, Tal Katan, Galit Govezensky, Keren Hadad, and Reut Meidler. Finally, we would like to thank the study’s participants and their families. This study was conducted as part of the ASC-Inclusion Project and has received funding from the European Community’s Seventh Framework Programme (FP7) under grant Agreement No [289021]. The Swedish authors were also supported by the Swedish Research Council. The UK authors were also supported by the Autism Research Trust, by the MRC, and by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care East of England at Cambridgeshire and Peterborough NHS Foundation Trust. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Author information

Affiliations

Authors

Corresponding author

Correspondence to O. Golan.

Ethics declarations

Conflict of interest

The SG is planned to be sold through Emotiplay in the future. Potential royalties to authors Bölte, Baron-Cohen, and Golan, resulting from intellectual property rights, would be paid to the academic institutions, to be used for autism research.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fridenson-Hayo, S., Berggren, S., Lassalle, A. et al. ‘Emotiplay’: a serious game for learning about emotions in children with autism: results of a cross-cultural evaluation. Eur Child Adolesc Psychiatry 26, 979–992 (2017). https://doi.org/10.1007/s00787-017-0968-0

Download citation

Keywords

  • Autism spectrum condition
  • Emotion recognition
  • Serious games
  • Cross-cultural study
  • Intervention evaluation