A Distributed Tool to Perform Dynamic Therapies for Social Cognitive Deficit Through Avatars

  • Mario García-Sánchez
  • Miguel A. TeruelEmail author
  • Elena Navarro
  • Pascual González
  • Antonio Fernández-Caballero
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10586)


Patients suffering from Social Cognition Deficits have difficulties when trying to understand its interlocutor emotional status. In order to contribute to the treatment of this deficit, we have developed a distributed application to offer remote therapies and using the concept of avatars. By using this application, therapist embody avatars that convey their emotions, voices and gestures. Therefore, this application enables patient to recognize the avatars emotions which, in turn, are controlled by the therapist. For this aim, a distributed software has been developed along with different devices such as a Kinect v2 for motion tracking and a facial expression analyzer. Unity has been used for the development of this application to make this type of remote therapy possible.


Social cognitive deficit Therapy Rehabilitation Distributed architecture Emotions 



This work was partially supported by Spanish Ministry of Economy, Industry and Competitiveness, State Research Agency/European Regional Development Fund under EmoBioFeedback (DPI2016-80894-R), HA-SYMBIOSIS (TIN2015-72931-EXP) and Vi-SMARt (TIN2016-79100-R) Grants. We would also want to thank Rumen Filkov for providing us with his Kinect assets for Unity.


  1. 1.
    Harvey, P.D., Penn, D.: Social cognition: the key factor predicting social outcome in people with schizophrenia? Psychiatry (Edgmont) 7, 41–44 (2010)Google Scholar
  2. 2.
    van Zwieten, A., Meyer, J., Hermens, D.F., Hickie, I.B., Hawes, D.J., Glozier, N., Naismith, S.L., Scott, E.M., Lee, R.S.C., Guastella, A.J.: Social cognition deficits and psychopathic traits in young people seeking mental health treatment. PLoS ONE 8, e67753 (2013)CrossRefGoogle Scholar
  3. 3.
    Gottesman, I.I.: Schizophrenia Genesis: The Origins of Madness. WH Freeman/Times Books/Henry Holt & Co, New York (1991)Google Scholar
  4. 4.
    Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., Clubley, E.: The autism-spectrum quotient (AQ): evidence from asperger syndrome/high-functioning autism, malesand females, scientists and mathematicians. J. Autism Dev. Disord. 31, 5–17 (2001)CrossRefGoogle Scholar
  5. 5.
    Teruel, M.A., Navarro, E., Romero, D., García, M., Fernández-Caballero, A., González, P.: An innovative tool to create neurofeedback games for ADHD treatment. In: Ferrández, J.M., Álvarez-Sánchez, J.R., de la Paz, F., Toledo Moreo, J., Adeli, H. (eds.) Natural and Artificial Computation for Biomedicine and Neuroscience, pp. 183–192. Springer International Publishing, Switzerland (2017). doi: 10.1007/978-3-319-59740-9_18 CrossRefGoogle Scholar
  6. 6.
    Rosenberg, H., McDonald, S., Dethier, M., Kessels, R.P.C., Westbrook, R.F.: Facial emotion recognition deficits following moderate-severe traumatic brain injury (TBI): re-examining the valence effect and the role of emotion intensity. J. Int. Neuropsychol. Soc. 20, 994–1003 (2014)CrossRefGoogle Scholar
  7. 7.
    Calvo, M.G., Nummenmaa, L.: Perceptual and affective mechanisms in facial expression recognition: an integrative review. Cogn. Emot. 30, 1081–1106 (2016)CrossRefGoogle Scholar
  8. 8.
    Fernández-Caballero, A., Martínez-Rodrigo, A., Pastor, J.M., Castillo, J.C., Lozano-Monasor, E., López, M.T., Zangróniz, R., Latorre, J.M., Fernández-Sotos, A.: Smart environment architecture for emotion detection and regulation. J. Biomed. Inf. 64, 55–73 (2016)CrossRefGoogle Scholar
  9. 9.
    Calvo, M.G., Nummenmaa, L.: Perceptual and affective mechanisms in facial expression recognition: An integrative review. Cogn. Emot. 30, 1081–1106 (2016)CrossRefGoogle Scholar
  10. 10.
    Garrido, M.V., Lopes, D., Prada, M., Rodrigues, D., Jerónimo, R., Mourão, R.P.: The many faces of a face: comparing stills and videos of facial expressions in eight dimensions (SAVE database). Behav. Res. Methods. (2016)Google Scholar
  11. 11.
    Keltner, D., Cordaro, D.T.: Understanding multimodal emotional expressions: recent advances in basic emotion theory. In: Fernández-Dols, J.-M., Russel, J.A. (eds.) The Science fo Facial Expresion. Oxford University Press, Oxford (2017)Google Scholar
  12. 12.
    Mori, M., MacDorman, K., Kageki, N.: The uncanny valley [from the field]. IEEE Robot. Autom. Mag. 19, 98–100 (2012)CrossRefGoogle Scholar
  13. 13.
    Menard, M., Wagstaff, B.: Game Development with Unity. Cengage Learning, Boston (2014)Google Scholar
  14. 14.
    McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., Kaliouby, R. el: AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. In: 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA 2016), pp. 3723–3726. ACM Press, Santa Clara, USA (2016)Google Scholar
  15. 15.
    Lewis, J.P., Anjyo, K.: Direct manipulation blendshapes. IEEE Comput. Graph. Appl. 30, 42–50 (2010)CrossRefGoogle Scholar
  16. 16.
    Lachat, E., Macher, H., Mittet, M.A., Landes, T., Grussenmeyer, P.: First experiences with Kinect v2 sensor for close range 3D modelling. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 40, 93 (2015)CrossRefGoogle Scholar
  17. 17.
    Taylor, R.M., Hudson, T.C., Seeger, A., Weber, H., Juliano, J., Helser, A.T.: VRPN: a device-independent, network-transparent VR peripheral system. In: ACM Symposium on Virtual Reality Software and Technology (VRST 2001) p. 55. ACM Press, Baniff, Canada (2001)Google Scholar
  18. 18.
    Teruel, M.A., Navarro, E., González, P.: Towards an awareness interpretation for physical and cognitive rehabilitation systems. In: García, C.R., Caballero-Gil, P., Burmester, M., Quesada-Arencibia, A. (eds.) Ubiquitous Computing and Ambient Intelligence, pp. 121–132. Springer International Publishing, Switzerland (2016). doi: 10.1007/978-3-319-48746-5_13 CrossRefGoogle Scholar
  19. 19.
    Teruel, M.A., Navarro, E., González, P.: Exploiting awareness for the development of collaborative rehabilitation systems. Mob. Inf. Syst. J. (2017, in press) Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.LoUISE Research Group, Computing Systems DepartmentUniversity of Castilla – La ManchaAlbaceteSpain

Personalised recommendations