Analyzing Human-Avatar Interaction with Neurotypical and not Neurotypical Users

  • Esperanza JohnsonEmail author
  • Carlos Gutiérrez López de la Franca
  • Ramón Hervás
  • Tania Mondéjar
  • José Bravo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10069)


Assistive technologies have been used to improve the quality of life of people who have been diagnosed with health issues. In this case, we aim to use an assistive technology in the shape of an affective avatar to help people who have been diagnosed with different forms of Social Communications Disorders (SCD). The designed avatar presents a humanoid face that displays emotions with a subtlety akin to that of real life human emotions, with those emotions changing according to the interactions that the user chooses to perform on the avatar. We have used Blender for the design of the emotions, which are happiness, sadness, surprise, fear and anger, plus a neutral emotion, while Unity was used to dictate the behavior of the avatar when the interactions were performed, which could be positive (caress), negative (poke) or neutral (wait). The avatar has been evaluated by 48 people from different backgrounds and the results show the overall positive reception by the users, as well as the difference between neurotypical and non-neurotypical users in terms of emotion recognition and chosen interactions. A ground truth has been established in terms of prototypic empathic interactions by the users.


Affective computing Affective avatar Human-avatar interaction Social communication disorder Cognitive disabilities Empathy 



This work was conducted in the context of UBIHEALTH project under International Research Staff Exchange Schema (MC-IRSES 316337).


  1. 1.
    Gibson, J., Adams, C., Lockton, E., et al.: Social communication disorder outside autism? A diagnostic classification approach to delineating pragmatic language impairment, high functioning autism and specific language impairment. J. Child Psychol. Psychiatry Allied Disciplines 54(11), 1186–1197 (2013)CrossRefGoogle Scholar
  2. 2.
    LoPresti, E.F., Mihailidis, A., Kirsch, N.: Assistive technology for cognitive rehabilitation: state of the art. Neuropsychol. Rehabil. 14(1–2), 5–39 (2004)CrossRefGoogle Scholar
  3. 3.
    Cheng, Y., Moore, D., McGrath, P., Fan, Y.: Collaborative virtual environment technology for people with autism. In: Fifth IEEE International Conference on Advanced Learning Technologies (ICALT 2005), pp. 247–248. IEEE, July 2005Google Scholar
  4. 4.
    Cereghetti, D., Kleanthous, S., Christophorou, C., et al.: Virtual partners for seniors: analysis of the user’s preferences and expectations on personality and appearance. In: AmI 2015, European Conference on Ambient Intelligence, Athens, Greece, 11–13 November 2015 (2015)Google Scholar
  5. 5.
    Hanke, S., Sandner, E., Stainer-Hochgatterer, A., et al.: The technical specification and architecture of a virtual support partner. In: AmI 2015, European Conference on Ambient Intelligence, Athens, Greece, 11–13 November 2015 (2015)Google Scholar
  6. 6.
    Adjouadi, M., Sesin, A., Ayala, M., Cabrerizo, M.: Remote eye gaze tracking system as a computer interface for persons with severe motor disability. In: Miesenberger, K., Klaus, J., Zagler, W.L., Burger, D. (eds.) ICCHP 2004. LNCS, vol. 3118, pp. 761–769. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Lanyi, C.S., Brown, D.J.: Design of serious games for students with intellectual disability. IHCI 10, 44–54 (2010)Google Scholar
  8. 8.
    Norris, A.E., Weger, H., Bullinger, C., et al.: Quantifying engagement: measuring player involvement in human–avatar interactions. Comput. Hum. Behav. 34, 1–11 (2014)CrossRefGoogle Scholar
  9. 9.
    Ugarte, A., García, I., Ortiz, A., Oyarzun, D.: User interfaces based on 3D avatars for interactive television. In: Cesar, P., Chorianopoulos, K., Jensen, J.F. (eds.) EuroITV 2007. LNCS, vol. 4471, pp. 107–115. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  10. 10.
    Iglesias, A., Luengo, F.: AI framework for decision modeling in behavioral animation of virtual avatars. In: Shi, Y., van Albada, G.D., Dongarra, J., Sloot, P.M. (eds.) ICCS 2007, Part II. LNCS, vol. 4488, pp. 89–96. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  11. 11.
    Inal, Y., Sancar, H., Cagiltay, K.: Children’s avatar preferences and their personalities. In: Society for Information Technology & Teacher Education International Conference, Florida, USA, 20–25 March 2006 (2006)Google Scholar
  12. 12.
    Carter, E.J., Hyde, J., Williams, D.L., Hodgins, J.K.: Investigating the influence of avatar facial characteristics on the social behaviors of children with autism. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 140–151. ACM, May 2016Google Scholar
  13. 13.
    Johnson, E., Hervás, R., Mondéjar, T., et al.: Improving social communication disorders through human-avatar interaction. In: Ambient Intelligence for Health, Puerto Varas, Chile, 1–4 December 2015, pp. 237–243 (2015)Google Scholar
  14. 14.
    Cambria, E., Hupont, I., Hussain, A., Cerezo, E., Baldassarri, S.A.: Sentic avatar: multimodal affective conversational agent with common sense. In: Esposito, A., Esposito, A.M., Martone, R., Müller, V.C., Scarpetta, G.Y. (eds.) COST 2012. LNCS, vol. 6456, pp. 81–95. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  15. 15.
    Balconi, M., Pozzoli, U.: Face-selective processing and the effect of pleasant and unpleasant emotional expressions on ERP correlates. Int. J. Psychophysiol. 49(1), 67–74 (2003)CrossRefGoogle Scholar
  16. 16.
    Johnson, E., Hervás, H., Gutiérrez López de la Franca, C., Mondéjar, T., Ochoa, S.F., Favela, J.: Assessing empathy and managing emotions through interactions with an affective avatar. Health Inf. J., 1–12, September 2016Google Scholar
  17. 17.
    Mondéjar, T., Hervás, R., Johnson, E., Gutierrez, C., Latorre, J.M.: Correlation between videogame mechanics and executive functions through EEG analysis. J. Biomed. Inform. 63, 131–140 (2016)CrossRefGoogle Scholar
  18. 18.
    de la Franca, G.L., Hervás, R., Bravo, J.: Activity recognition in intelligent assistive environments through video analysis with body-angles algorithm. In: García-Chamizo, J.M., Fortino, G., Martone, R., Müller, V.C., Scarpetta, G.Y. (eds.) UCAmI 2015. LNCS, vol. 9454, pp. 162–173. Springer, Heidelberg (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Esperanza Johnson
    • 1
    Email author
  • Carlos Gutiérrez López de la Franca
    • 1
  • Ramón Hervás
    • 1
  • Tania Mondéjar
    • 1
    • 2
  • José Bravo
    • 1
  1. 1.University of Castilla-la Mancha (MAmI Research Lab)Ciudad RealSpain
  2. 2.eSmile, Psychology for Children and AdolescentsCiudad RealSpain

Personalised recommendations