Single Trial Classification of EEG and Peripheral Physiological Signals for Recognition of Emotions Induced by Music Videos

  • Sander Koelstra
  • Ashkan Yazdani
  • Mohammad Soleymani
  • Christian Mühl
  • Jong-Seok Lee
  • Anton Nijholt
  • Thierry Pun
  • Touradj Ebrahimi
  • Ioannis Patras
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6334)

Abstract

Recently, the field of automatic recognition of users’ affective states has gained a great deal of attention. Automatic, implicit recognition of affective states has many applications, ranging from personalized content recommendation to automatic tutoring systems. In this work, we present some promising results of our research in classification of emotions induced by watching music videos. We show robust correlations between users’ self-assessments of arousal and valence and the frequency powers of their EEG activity. We present methods for single trial classification using both EEG and peripheral physiological signals. For EEG, an average (maximum) classification rate of 55.7% (67.0%) for arousal and 58.8% (76.0%) for valence was obtained. For peripheral physiological signals, the results were 58.9% (85.5%) for arousal and 54.2% (78.5%) for valence.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cacioppo, J., Berntson, G., Larsen, J., Poehlmann, K., Ito, T.: The psychophysiology of emotion. In: Handbook of Emotions, pp. 119–142 (1993)Google Scholar
  2. 2.
    Chanel, G., Kierkels, J., Soleymani, M., Pun, T.: Short-term emotion assessment in a recall paradigm. Int’l. Journal Human-Computer Studies 67(8), 607–627 (2009)CrossRefGoogle Scholar
  3. 3.
    Demaree, H.A., Everhart, E.D., Youngstrom, E.A., Harrison, D.W.: Brain lateralization of emotional processing: Historical roots and a future incorporating“dominance”. Behavioral and Cognitive Neuroscience Reviews 4(1), 3–20 (2005)CrossRefGoogle Scholar
  4. 4.
    Ekman, P., Friesen, W., Osullivan, M., Chan, A., Diacoyannitarlatzis, I., Heider, K., Krause, R., Lecompte, W., Pitcairn, T., Riccibitti, P., Scherer, K., Tomita, M., Tzavaras, A.: Universals and cultural-differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology 53(4), 712–717 (1987)CrossRefGoogle Scholar
  5. 5.
    Kierkels, J., Soleymani, M., Pun, T.: Queries and tags in affect-based multimedia retrieval. In: Int’l. Conf. Multimedia and Expo, Special Session on Implicit Tagging (ICME 2009), New York, United States (2009)Google Scholar
  6. 6.
    Knyazev, G.: Motivation, emotion, and their inhibitory control mirrored in brain oscillations. Neuroscience & Biobehavioral Reviews 31(3), 377–395 (2007)CrossRefGoogle Scholar
  7. 7.
    Ko, K., Yang, H., Sim, K.: Emotion recognition using EEG signals with relative power values and Bayesian network. Int’l. Journal of Control, Automation and Systems 7(5), 865–870 (2009)CrossRefGoogle Scholar
  8. 8.
    Koles, Z.: The quantitative extraction and topographic mapping of the abnormal components in the clinical EEG. Electroencephalography and Clinical Neurophysiology 79(6), 440–447 (1991)CrossRefGoogle Scholar
  9. 9.
    Lang, P., Greenwald, M., Bradely, M., Hamm, A.: Looking at pictures - affective, facial, visceral, and behavioral reactions. Psychophysiology 30(3), 261–273 (1993)CrossRefGoogle Scholar
  10. 10.
    Li, M., Chai, Q., Kaixiang, T., Wahab, A., Abut, H.: EEG Emotion Recognition System. In: Vehicle Corpus and Signal Processing for Driver Behavior, p. 125 (2008)Google Scholar
  11. 11.
    Lisetti, C.L., Nasoz, F.: Using noninvasive wearable computers to recognize human emotions from physiological signals. EURASIP J. Appl. Signal Process. 2004, 1672–1687 (2004)Google Scholar
  12. 12.
    Loughin, T.M.: A systematic comparison of methods for combining p-values from independent tests. Computational Statistics & Data Analysis 47, 467–485 (2004)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    McCraty, R., Atkinson, M., Tiller, W., Rein, G., Watkins, A.: The effects of emotions on short-term power spectrum analysis of heart rate variability. The American Journal of Cardiology 76(14), 1089–1093 (1995)CrossRefGoogle Scholar
  14. 14.
    Murugappan, M., Juhari, M., Nagarajan, R., Yaacob, S.: An investigation on visual and audiovisual stimulus based emotion recognition using EEG. Int’l. Journal of Medical Engineering and Informatics 1(3), 342–356 (2009)CrossRefGoogle Scholar
  15. 15.
    Russell, J.: A circumplex model of affect. Journal of Personality and Social Psychology 39(6), 1161–1178 (1980)CrossRefGoogle Scholar
  16. 16.
    Solis-Escalante, T., Müller-Putz, G., Pfurtscheller, G.: Overt foot movement detection in one single laplacian EEG derivation. Journal of Neuroscience Methods 175(1), 148–153 (2008)CrossRefGoogle Scholar
  17. 17.
    Stemmler, G., Heldmann, M., Pauls, C., Scherer, T.: Constraints for emotion specificity in fear and anger: The context counts. Psychophysiology 38(02), 275–291 (2001)CrossRefGoogle Scholar
  18. 18.
    Wang, J., Gong, Y.: Recognition of multiple drivers emotional state. In: Int’l. Conf. Pattern Recognition, pp. 1–4 (December 2008)Google Scholar
  19. 19.
    Yazdani, A., Lee, J.-S., Ebrahimi, T.: Implicit emotional tagging of multimedia using EEG signals and brain computer interface. In: Proc. SIGMM Workshop on Social Media, pp. 81–88. ACM, New York (2009)Google Scholar
  20. 20.
    Yu, L., Liu, H.: Efficient feature selection via analysis of relevance and redundancy. Journal of Machine Learning Research 5, 1205–1224 (2004)MathSciNetMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Sander Koelstra
    • 1
  • Ashkan Yazdani
    • 2
  • Mohammad Soleymani
    • 3
  • Christian Mühl
    • 4
  • Jong-Seok Lee
    • 2
  • Anton Nijholt
    • 4
  • Thierry Pun
    • 3
  • Touradj Ebrahimi
    • 2
  • Ioannis Patras
    • 1
  1. 1.Department of Electronic EngineeringQueen Mary University of LondonUK
  2. 2.Multimedia Signal Processing GroupEcole Polytechnique Fédérale de LausanneSwitzerland
  3. 3.Computer Vision and Multimedia LaboratoryUniversity of GenevaSwitzerland
  4. 4.Human Media Interaction GroupUniversity of TwenteNetherlands

Personalised recommendations