Advertisement

An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence

  • Panagiotis D. Bamidis
  • Christos A. Frantzidis
  • Evdokimos I. Konstantinidis
  • Andrej Luneski
  • Chrysa Lithari
  • Manousos A. Klados
  • Charalambos Bratsas
  • Christos L. Papadelis
  • Costas Pappas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5612)

Abstract

Emotion identification is beginning to be considered as an essential feature in human-computer interaction. However, most of the studies are mainly focused on facial expression classifications and speech recognition and not much attention has been paid until recently to physiological pattern recognition. In this paper, an integrative approach is proposed to emotional interaction by fusing multi-modal signals. Subjects are exposed to pictures selected from the International Affective Picture System (IAPS). A feature extraction procedure is used to discriminate between four affective states by means of a Mahalanobis distance classifier. The average classifications rate (74.11%) was encouraging. Thus, the induced affective state is mirrored through an avatar by changing its facial characteristics and generating a voice message sympathising with the user’s mood. It is argued that multi-physiological patterning in combination with anthropomorphic avatars may contribute to the enhancement of affective multi-modal interfaces and the advancement of machine emotional intelligence.

Keywords

Emotion Affective Computing EEG Skin Conductance Avatar Mahalanobis classifier 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bradley, M.M., Codispoti, M., Cuthbert, B.N., Lang, P.J.: Emotion and motivation I: Defensive and appetitive reactions in picture processing. Emotion 1(3), 276–298 (2001)CrossRefGoogle Scholar
  2. 2.
    Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International Affective Picture System (IAPS): Technical Manual and Affective Ratings. NIMH Center for the Study of Emotion and Attention (1997)Google Scholar
  3. 3.
    Picard, R.W., et al.: Affective learning – A Manifesto. BT Technology Journal 22(4), 253–269 (2004)CrossRefGoogle Scholar
  4. 4.
    Reeves, B., Nass, C.: The Media Equation: How People Treat Computers. Cambridge Univ. Press, Cambridge (1996)Google Scholar
  5. 5.
    Cockton, G.: Editorial:From doing to being:bringing emotion into interaction. Interacting with Computers 14(2), 89–92 (2002)CrossRefGoogle Scholar
  6. 6.
    Picard, R.W., Healey, J.: Affective Wearables. Personal and Ubiquitous Computing 1(4), 231–240 (1997)Google Scholar
  7. 7.
    Konstantinidis, E.I., Bamidis, P.D., Koufogiannis, D.: Development of A Generic And Flexible Human Body Wireless Sensor Network. In: 6th European Symposium on Biomedical Engineering (ESBME 2008), Chania, Crete Island, Greece (2008)Google Scholar
  8. 8.
    Healey, J., Picard, R.W.: Digital processing of affective signals. In: International Conference on Acoustics, Speech and Signal Processing, Seattle, USA, vol. 6, pp. 3749–3752 (1998)Google Scholar
  9. 9.
    Picard, R.W., Vyzas, E., Healey, J.: Toward Machine Emotional Intelligence: Analysis of Affective Physiological State. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(10), 1175–1191 (2001)CrossRefGoogle Scholar
  10. 10.
    Haag, A., Goronzy, S., Schaich, P., Williams, J.: Emotion recognition Using Bio-Sensors: First Steps Towards an Automatic System. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS, vol. 3068, pp. 36–48. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Nasoz, F., Lisetti, C.L., Alvarez, K., Finkelstein, N.: Emotion Recognition from Physiological Signals for User Modeling of Affect. In: 3rd Workshop on Affective and Attitude User Modeling, USA (2003)Google Scholar
  12. 12.
    Bamidis, P.D., Luneski, A., Vivas, A., Papadelis, C., Maglaveras, N., Pappas, C.: Multi-channel physiological sensing of human emotion: insights into Emotion-Aware Computing using Affective Protocols, Avatars and Emotion. Stud. Health Technol. Inform. 129(Pt 2), 1068–1072 (2007)Google Scholar
  13. 13.
    Frantzidis, C., Lithari, C., Vivas, A., Papadelis, C., Pappas, C., Bamidis, P.D.: Towards Emotion Aware Computing: a Study of Arousal Modulation with Multichannel Event Related Potentials, Delta Oscillatory Activity and Skin Conductivity Responses. In: 8th IEEE International Conference on BioInformatics and BioEngineering (BIBE 2008), Athens, Greece, pp. 1–6 (2008)Google Scholar
  14. 14.
    Brody, L.R.: Gender differences in emotional development: A review of theories and research. Journal of Personality 53(2), 102–149 (2006)CrossRefGoogle Scholar
  15. 15.
    Luneski, A., Bamidis, P.D.: Towards an Emotion Specification Method: Representing Emotional Physiological Signals. In: Proc. of the 20th IEEE International Symposium on Computer-Based Medical Systems (CBMS 2007), Maribor, Slovenia, pp. 313–320 (2007)Google Scholar
  16. 16.
    The free Haptek Guide, http://haptek.com/ (last access 24/2/2009)
  17. 17.
    Loquendo: global supplier of speech recognition and speech synthesis technology and solutions, http://www.loquendo.com/ (last access 24/2/2009)
  18. 18.
    Cincotti, F., et al.: The use of EEG modifications due to motor imagery for brain-computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering 11(2), 131–133 (2003)CrossRefGoogle Scholar
  19. 19.
    Luneski, A., Konstantinidis, E.I., Hitoglou-Antoniadou, M., Bamidis, P.D.: Affective Computer-Aided Learning for Autistic Children. In: 1st Workshop on Child, Computer and Interaction, ICMI 2008, Chania, Crete, Greece, October 20-23 (2008)Google Scholar
  20. 20.
    Lisetti, C.L., Nasoz, F.: Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals. EURASIP Journal on Applied Signal Processing 2004(11), 1672–1687 (2004)CrossRefGoogle Scholar
  21. 21.
    Boulos, K., Maged, N., Wheeler, S.: The emerging Web 2.0 social software: an enabling suite of sociable technologies in health and health care education. Health Information and Libraries Journal 24(1), 2–23 (2007)CrossRefGoogle Scholar
  22. 22.
    Vilon, O., Lisetti, C.L.: Toward Recognizing Individual’s Subjective Emotion from Physiological Signals in Practical Application. In: Proc. of 20th IEEE International Symposium on Computer-Based Medical Systems (CBMS 2007), Maribor, Slovenia, pp. 357–362 (2007)Google Scholar
  23. 23.
    Luneski, A., Bamidis, P.D., Hitoglou-Antoniadou, M.: Affective computing and medical informatics: state of the art in emotion-aware medical applications. Stud. Health Technol. Inform. 136, 517–522 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Panagiotis D. Bamidis
    • 1
  • Christos A. Frantzidis
    • 1
  • Evdokimos I. Konstantinidis
    • 1
  • Andrej Luneski
    • 1
  • Chrysa Lithari
    • 1
  • Manousos A. Klados
    • 1
  • Charalambos Bratsas
    • 1
  • Christos L. Papadelis
    • 2
  • Costas Pappas
    • 1
  1. 1.Lab of Medical Informatics, Medical SchoolAristotle University of ThessalonikiGreece
  2. 2.Center for Mind/Brain (CIMEC)University of TrentoMattarelloItaly

Personalised recommendations