Building an Intelligent, Authorable Serious Game for Autistic Children and Their Carers

  • Kaśka Porayska-Pomsta
  • Keith Anderson
  • Sara Bernardini
  • Karen Guldberg
  • Tim Smith
  • Lila Kossivaki
  • Scott Hodgins
  • Ian Lowe
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8253)


This paper introduces the SHARE-IT project, which leverages serious games paradigm to motivate and engage children with autism diagnosis in interactive activities, based on the state-of-the-art autism intervention practices. The aim of SHARE-IT is to formulate, in partnership with schools, parents and industry, the requirements for a robust, intelligent and authorable environment for supporting children in exploring, practicing and acquiring social interaction skills. SHARE-IT focuses on two key challenges: (i) developing robust system architecture and implementation, able to support both continuing development of a serious game for children with autism and its real world use; and (ii) selecting appropriate technologies and techniques to allow for (a) multi-device and operating system deployment, (b) the development of an intelligent serious game for supporting social interaction while (c) allowing the flexibility for the environment to be authored by lay persons. SHARE-IT’s architecture is presented and several considerations of importance to enabling the engineering of an intelligent and authorable serious game are discussed. Examples of technologies developed to date are given throughout and a discussion of future challenges offered.


Autism Spectrum Disorder Facial Expression User Model Autistic Child Authoring Tool 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Medical Research Council: Autism - Research Review (2001) (accessed October 20, 2012)Google Scholar
  2. 2.
    Knapp, M., Romeo, R., Beecham, J.: The economic consequences of autism in the uk. Foundation for People with Learning Disabilities Report (2007)Google Scholar
  3. 3.
    Prizant, B., Wetherby, A., Rubin, E., Laurent, A., Rydell, P.: The SCERTS® Model: A Comprehensive Educational Approach for Children with Autism Spectrum Disorders. Brookes (2006)Google Scholar
  4. 4.
    Wass, S., Porayska-Pomsta, K.: The uses of cognitive training technologies in the treatment of autism spectrum disorders. Autism: International Journal of Research and Practice (in press)Google Scholar
  5. 5.
    Bernardini, S., Porayska-Pomsta, K., Smith, T.J., Avramides, K.: Building autonomous social partners for autistic children. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS (LNAI), vol. 7502, pp. 46–52. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  6. 6.
    de Freitas, S.: Learning in immersive worlds. Technical report, Joint Information Systems Committee, Bristol (2006) (accessed October 20, 2012)Google Scholar
  7. 7.
    Parsons, S., Cobb, S.: State-of-the-art of virtual reality technologies for children on the autism spectrum. European Journal of Special Needs Education 26(3), 355–366 (2011)CrossRefGoogle Scholar
  8. 8.
    Grynszpan, O., Martin, J.C., Nadel, J.: Multimedia interfaces for users with high functioning autism: An empirical investigation. International Journal on Human-Computer Studies 66(8), 628–639 (2008)CrossRefGoogle Scholar
  9. 9.
    Tartaro, A., Cassell, J.: Playing with virtual peers: bootstrapping contingent discourse in children with autism. In: Proceedings of the ICLS 2008, pp. 382–389 (2008)Google Scholar
  10. 10.
    Parsons, S., Mitchell, P.: The potential of virtual reality in social skills training for people with autistic spectrum disorders. Journal of Intellectual Disability Research 46(5), 430–443 (2002)CrossRefGoogle Scholar
  11. 11.
    Anwar, A., Rahman, M., Ferdous, S., Anik, S., Ahmed, S.: A computer game based approach for increasing fluency in the speech of the autistic children. In: Proceedings of the11th IEEE International Conference on Advanced Learning Technologies (ICALT), pp. 17–18 (2011)Google Scholar
  12. 12.
    Rahman, M.M., Ferdous, S., Ahmed, S.I., Anwar, A.: Speech development of autistic children by interactive computer games. Interactive Technology and Smart Education 8(4), 208–223 (2011)CrossRefGoogle Scholar
  13. 13.
    Davis, M., Otero, N., Dautenhahn, K., Nehaniv, C., Powell, S.: Creating a software to promote understanding about narrative in children with autism: Reflecting on the design of feedback and opportunities to reason. In: IEEE 6th International Conference on Development and Learning, Proceedings of the Development and Learning, ICDL 2007, pp. 64–69 (July 2007)Google Scholar
  14. 14.
    Bosseler, A., Massaro, D.: Development and evaluation of a computer-animated tutor for vocabulary and language learning in children with autism. Journal of Autism and Developmental Disorders 33(6), 653–672 (2003)CrossRefGoogle Scholar
  15. 15.
    Massaro, D.W.: Embodied agents in language learning for children with language challenges. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I. (eds.) ICCHP 2006. LNCS, vol. 4061, pp. 809–816. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    Finkelstein, S.L., Nickel, A., Harrison, L., Suma, E.A., Barnes, T.: cMotion: A new game design to teach emotion recognition and programming logic to children using virtual humans. In: Proceedings of the 2009 IEEE Virtual Reality Conference, pp. 249–250 (2009)Google Scholar
  17. 17.
    Abirached, B., Zhang, Y., Aggarwal, J., Tamersoy, B., Fernandes, T., Miranda, J., Orvalho, V.: Improving communication skills of children with asds through interaction with virtual characters. In: 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH), pp. 1–4 (2011)Google Scholar
  18. 18.
    Schuller, B.: ASC-Inclusion Project (accessed October 20, 2012)Google Scholar
  19. 19.
    Hopkins, I.M., Gower, M.W., Perez, T.A., Smith, D.S., Amthor, F.R., et al.: Avatar assistant: Improving social skills in students with an asd through a computer-based intervention. Journal of Autism and Developmental Disorders 41(11), 1543–1555 (2011)CrossRefGoogle Scholar
  20. 20.
    Beaumont, R., Sofronoff, K.: A multi-component social skills intervention for children with asperger syndrome: The junior detective training program. Journal of Child Psychology and Psychiatry 49, 743–753 (2008)CrossRefGoogle Scholar
  21. 21.
    Battocchi, A., Pianesi, F., Tomasini, D., Zancanaro, M., Esposito, G., Venuti, P., Ben Sasson, A., Gal, E., Weiss, P.L.: Collaborative puzzle game: a tabletop interactive game for fostering collaboration in children with autism spectrum disorders (asd). In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ITS 2009, pp. 197–204. ACM, New York (2009)CrossRefGoogle Scholar
  22. 22.
    Barakova, E., van Wanrooij, G., van Limpt, R., Menting, M.: Using an emergent system concept in designing interactive games for autistic children. In: Proceedings of the 6th International Conference on Interaction Design and Children, IDC 2007, pp. 73–76. ACM, New York (2007)Google Scholar
  23. 23.
    Kozima, H., Michalowski, M., Nakagawa, C.: Keepon: A playful robot for research, therapy, and entertainment. International Journal of Social Robotics 1(1), 3–18 (2009)CrossRefGoogle Scholar
  24. 24.
    Dautenhahn, K., Werry, I.: Towards interactive robots in autism therapy: Background, motivation and challenges. Pragmatics and Cognition 12(1), 1–35 (2004)CrossRefGoogle Scholar
  25. 25.
    Milne, M., Luerssen, M., Lewis, T., Leibbrandt, R., Powers, D.: Development of a virtual agent based social tutor for children with autism spectrum disorders. In: Proceedings of the International Joint Conference on Neural Networks, pp. 1–9 (2010)Google Scholar
  26. 26.
    Snape, J., Guy, S., Lin, M., Manocha, D.: Local and global planning for collision-free navigation in video games. In: Proceedings of the 3rd International Planning in Games Workshop, ICAPS 2013 (2013)Google Scholar
  27. 27.
    Jaklin, N., van Toll, W., Roland, G.: Way to go - a framework for multi-level planning in games. In: Proceedings of the 3rd International Planning in Games Workshop, ICAPS 2013 (2013)Google Scholar
  28. 28.
    Do, M., Tran, M.: Blocksworld: An ipad puzzle game. In: Proceedings of the 3rd International Planning in Games Workshop, ICAPS 2013 (2013)Google Scholar
  29. 29.
    Menif, A., Guettier, C., Cazenave, C.: Planning and execution control architecture for infantry serious gaming. In: Proceedings of the 3rd International Planning in Games Workshop, ICAPS 2013 (2013)Google Scholar
  30. 30.
    Riedl, M., Young, M.: Narrative planning: Balancing plot and character. Journal of AI Research 29, 217–268 (2010)zbMATHGoogle Scholar
  31. 31.
    Bakkes, S., Tan, C.T., Pisan, Y.: Personalised gaming. Creative Technologies 3 (2012)Google Scholar
  32. 32.
    Yannakakis, G., Hallam, J.: Modeling and augmenting game entertainment through challenge and curiosity. International Journal on Artificial Intelligence Tools 16(6), 981–999 (2007)CrossRefGoogle Scholar
  33. 33.
    Malone, T.W.: Guidelines for designing educational computer games. Childhood Education 59, 241–247 (1983)CrossRefGoogle Scholar
  34. 34.
    Csikszentmihalyi, M.: The Psychology of Optimal Experience. Harper Perennial, New York (1990)Google Scholar
  35. 35.
    Read, J., McFarlane, S., Cassey, C.: Endurability, engagement and expectations: Measuring children’s fun. In: Proceedings of the International Conference for Interaction Design and Children (2002)Google Scholar
  36. 36.
    Safyan, L., Lagattuta, K.H.: Grownups are not afraid of scary stuff, but kids are: young children’s and adults’ reasoning about children’s, infants’, and adults’ fears. Child Development 79(4), 821–835 (2008)CrossRefGoogle Scholar
  37. 37.
    Burleson, W.: Affective Learning Companions: strategies for empathetic agents with real-time multimodal affective sensing to foster meta-cognitive and meta-affective approaches to learning, motivation and perseverance. PhD thesis, Massachusetts Institute of Technology (2006)Google Scholar
  38. 38.
    Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press (1988)Google Scholar
  39. 39.
    Conati, C.: Probabilistic assessment of user’s emotions in educational games. Journal of Applied Artificial Intelligence, Special Issue on Merging Cognition and Affect in HCI 16 (7-8), 555–575 (2002)CrossRefGoogle Scholar
  40. 40.
    Arroyo, I., Beack, J., Beal, C.R., Woolf, B.P.: Learning with the zone of proximal development with the animalwatch intelligent tutoring system. In: Proceedings of the American Educational Research Association annual meeting, Chicago, IL (2003)Google Scholar
  41. 41.
    Kerawalla, L., O’Connor, J., Underwood, J., du Boulay, B., Holmberg, J., Luckin, R., Smith, H., Tunley, H.: The homework system: using tablet pcs as tools to support continuity of numeracy learning between home and primary school. Educational Media International 44(4), 289–303 (2007)CrossRefGoogle Scholar
  42. 42.
    Chen, Z.H., Deng, Y.C., Chou, C.Y., Chan, T.W.: Active open learner models as animal companions: motivating children to learn through interaction with my-pet and our-pet. International Journal of Artificial Intelligence in Education 17, 145–167 (2007)Google Scholar
  43. 43.
    Pantic, M., Rothkrantz, L.J.M.: Towards an affect-sensitive multimodal human-computer interaction. Special Issue on Multimodal Human-Computer Interaction (HCI) 91, 1370–1390 (2003)Google Scholar
  44. 44.
    Damian, I., Baur, T., Gebhard, P., Porayska-Pomsta, K., André, E.: A software framework for social cue-based interaction with a virtual recruiter. In: Proceedings of the 13th International Conference on Intelligent Virtual Agents, Santa Cruz (2013)Google Scholar
  45. 45.
    Henning, M.: A new approach to object-oriented middleware. IEEE Internet Computing 8(1), 66–75 (2004)CrossRefGoogle Scholar
  46. 46.
    Dias, J., Paiva, A.: Feeling and reasoning: A computational model for emotional characters. In: Bento, C., Cardoso, A., Dias, G. (eds.) EPIA 2005. LNCS (LNAI), vol. 3808, pp. 127–140. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  47. 47.
    Fox, M., Long, D.: PDDL 2.1: An extension to PDDL for expressing temporal planning domains. Journal of Artificial Intelligence Research 20, 61–124 (2003)CrossRefGoogle Scholar
  48. 48.
    Smith, C.A., Lazarus, R.S.: Emotion and adaptation. In: Vlahavas, I., Vrakas, D., Pervin, L.A. (eds.) Handbook of Personality: Theory and Research, pp. 609–637. Guilford, New York (1990)Google Scholar
  49. 49.
    Bernardini, S., Porayska-Pomsta, K., Sampath, H.: Designing an intelligent virtual agent for social communication in autism. In: Proceedings of the Ninth Annual AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Kaśka Porayska-Pomsta
    • 1
  • Keith Anderson
    • 2
  • Sara Bernardini
    • 3
  • Karen Guldberg
    • 4
  • Tim Smith
    • 5
  • Lila Kossivaki
    • 4
  • Scott Hodgins
    • 6
  • Ian Lowe
    • 7
  1. 1.London Knowledge LabInstitute of EducationLondonUK
  2. 2.Tandemis LimitedUK
  3. 3.King’s College LondonUK
  4. 4.School of EducationUniversity of BirminghamUK
  5. 5.Department of Psychological SciencesBirkbeck CollegeUK
  6. 6.Acuity ETS LimitedUK
  7. 7.Topcliffe Primary SchoolBirminghamUK

Personalised recommendations