Affective Avatar Interactions: Towards Recognizing Emotions in Verbal Interaction

  • Esperanza JohnsonEmail author
  • Ramón HervásEmail author
  • Carlos Gutiérrez-López-Franca
  • Tania Mondéjar
  • José Bravo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10586)


In current times, virtual agents, also known as avatars, are being used for many different tasks, from helping guide a user experience, to aiding in the diagnosis and treatment of different health issues, both physical and mental. In past work, we have explored the use of an affective avatar that responds to tactile interaction with a pc–tablet, the response from the users and the possible uses for aiding the diagnosis and treatment of Social Communications Disorders. In this paper, we present our proposal as a work in progress, based on our previous work with an affective avatar, but adding verbal interaction. We will also discuss what future work we have thought of doing with this particular interaction of the model.


Cognitive computing Affective computing Affective avatar IBM Watson 


  1. 1.
    Nasoz, F., Lisetti, C.L.: MAUI avatars: mirroring the user’s sensed emotions via expressive multi-ethnic facial avatars. J. Vis. Lang. Comput. 17(5), 430–444 (2006)CrossRefGoogle Scholar
  2. 2.
    Lisetti, C., Yasavur, U., De Leon, C., Amini, R., Rishe, N., Visser, U.: Building an on-demand avatar-based health intervention for behavior change. In: Proceedings of the 25th International Florida Artificial Intelligence Research Society Conference, FLAIRS-25, Marco Island, 23–25 May, pp. 175–180 (2012)Google Scholar
  3. 3.
    Li, M., Kavakli, M., Rudra, T.: eCounsellor: an avatar for student exam stress management. In: Erkan, T.E. (ed.) Proceedings of the 3rd International Conference on Information Management and Evaluation: Atilim University Performance Management and Applications Research Centre, Ankara, Turkey, 16–17 April 2012, pp. 185–193. Academic Publishing International, Nr Reading (2012)Google Scholar
  4. 4.
    Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C.M., Kazemzadeh, A., Lee, S., Neumann, U., Narayanan, S.: Analysis of emotion recognition using facial expressions, speech and multimodal information. In: Proceedings of the 6th International Conference on Multimodal Interfaces, pp. 205–211. ACM, New York (2004)Google Scholar
  5. 5.
    Vogt, Thurid, André, Elisabeth, Bee, Nikolaus: EmoVoice — A Framework for Online Recognition of Emotions from Voice. In: André, Elisabeth, Dybkjær, Laila, Minker, Wolfgang, Neumann, Heiko, Pieraccini, Roberto, Weber, Michael (eds.) PIT 2008. LNCS, vol. 5078, pp. 188–199. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-69369-7_21 CrossRefGoogle Scholar
  6. 6.
    Wang, K., An, N., Li, B.N., Zhang, Y., Li, L.: Speech emotion recognition using Fourier parameters. IEEE Trans. Affect. Comput. 6(1), 69–75 (2015)CrossRefGoogle Scholar
  7. 7.
    Johnson, E., Hervás, R., de la Gutiérrez-López Franca, C., Mondéjar, T., Ochoa, S.F., Favela, J.: Assessing empathy and managing emotions through interactions with an affective avatar. Health Inf. J. (2016). doi: 10.1177/1460458216661864 Google Scholar
  8. 8.
    Johnson, E., Hervás, R., Gutiérrez-López-Franca, C., Mondéjar, T., Bravo, J.: Analyzing and predicting empathy in neurotypical and non-neurotypical users with an affective avatar. Mob. Inf. Syst. (2017). doi: 10.1155/2017/7932529 Google Scholar
  9. 9.
    de la Franca, C.G.L., Hervás, R., Johnson, E., Mondéjar, T., Bravo, J.: Extended body-angles algorithm to recognize activities within intelligent environments. J. Ambient Intell. Hum. Comput. (2017). doi: 10.1007/s12652-017-0463-y Google Scholar
  10. 10.
    Mondéjar, T., Hervás, R., Johnson, E., Gutierrez, C., Latorre, J.M.: Correlation between videogame mechanics and executive functions through EEG analysis. J. Biomed. Inf. 63, 131–140 (2016). doi: 10.1016/j.jbi.2016.08.006 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Esperanza Johnson
    • 1
    Email author
  • Ramón Hervás
    • 1
    Email author
  • Carlos Gutiérrez-López-Franca
    • 1
  • Tania Mondéjar
    • 1
    • 2
  • José Bravo
    • 1
  1. 1.MAmI Research LabUniversity of Castilla-La ManchaCiudad RealSpain
  2. 2.eSmile, Psychology for Children and AdolescentsCiudad RealSpain

Personalised recommendations