Skip to main content
SpringerLink
Log in

Real-Time EEG-Based Emotion Recognition and Its Applications

  • Chapter
Transactions on Computational Science XII

Part of the book series: Lecture Notes in Computer Science ((TCOMPUTATSCIE,volume 6670))

Abstract

Since emotions play an important role in the daily life of human beings, the need and importance of automatic emotion recognition has grown with increasing role of human computer interface applications. Emotion recognition could be done from the text, speech, facial expression or gesture. In this paper, we concentrate on recognition of “inner” emotions from electroencephalogram (EEG) signals. We propose real-time fractal dimension based algorithm of quantification of basic emotions using Arousal-Valence emotion model. Two emotion induction experiments with music stimuli and sound stimuli from International Affective Digitized Sounds (IADS) database were proposed and implemented. Finally, the real-time algorithm was proposed, implemented and tested to recognize six emotions such as fear, frustrated, sad, happy, pleasant and satisfied. Real-time applications were proposed and implemented in 3D virtual environments. The user emotions are recognized and visualized in real time on his/her avatar adding one more so-called “emotion dimension” to human computer interfaces. An EEG-enabled music therapy site was proposed and implemented. The music played to the patients helps them deal with problems such as pain and depression. An EEG-based web-enable music player which can display the music according to the user’s current emotion states was designed and implemented.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Emotiv, http://www.emotiv.com

  2. Haptek, http://www.haptek.com

  3. American electroencephalographic society guidelines for standard electrode position nomenclature. Journal of Clinical Neurophysiology 8(2), 200–202 (1991)

    Google Scholar 

  4. Accardo, A., Affinito, M., Carrozzi, M., Bouquet, F.: Use of the fractal dimension for the analysis of electroencephalographic time series. Biological Cybernetics 77(5), 339–350 (1997)

    Article  MATH  Google Scholar 

  5. Block, A., Von Bloh, W., Schellnhuber, H.J.: Efficient box-counting determination of generalized fractal dimensions. Physical Review A 42(4), 1869–1874 (1990)

    Article  MathSciNet  Google Scholar 

  6. Bos, D.O.: EEG-based emotion recognition (2006), http://hmi.ewi.utwente.nl/verslagen/capita-selecta/CS-Oude_Bos-Danny.pdf

  7. Bradley, M.M.: Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25(1), 49–59 (1994)

    Article  Google Scholar 

  8. Bradley, M.M., Lang, P.J.: The international affective digitized sounds (2nd edition; IADS-2): Affective ratings of sounds and instruction manual. Tech. rep., University of Florida, Gainesville (2007)

    Google Scholar 

  9. Canli, T., Desmond, J.E., Zhao, Z., Glover, G., Gabrieli, J.D.E.: Hemispheric asymmetry for emotional stimuli detected with fMRI. NeuroReport 9(14), 3233–3239 (1998)

    Article  Google Scholar 

  10. Chanel, G.: Emotion assessment for affective-computing based on brain and peripheral signals. Ph.D. thesis, University of Geneva, Geneva (2009)

    Google Scholar 

  11. Chanel, G., Kierkels, J.J.M., Soleymani, M., Pun, T.: Short-term emotion assessment in a recall paradigm. International Journal of Human Computer Studies 67(8), 607–627 (2009)

    Article  Google Scholar 

  12. Chanel, G., Kronegg, J., Grandjean, D., Pun, T.: Emotion assessment: Arousal evaluation using EEG’s and peripheral physiological signals (2006)

    Google Scholar 

  13. Ekman, P.: Basic emotions. In: Dalgleish, T., Power, M. (eds.) Handbook of Cognition and Emotion. Wiley, New York (1999)

    Google Scholar 

  14. Grocke, D.E., Wigram, T.: Receptive Methods in Music Therapy: Techniques and Clinical Applications for Music Therapy Clinicians, Educators and Students, 1st edn. Jessica Kingsley Publishers (2007)

    Google Scholar 

  15. Guetin, S., Portet, F., Picot, M.C., Defez, C., Pose, C., Blayac, J.P., Touchon, J.: Impact of music therapy on anxiety and depression for patients with alzheimer’s disease and on the burden felt by the main caregiver (feasibility study). Interets de la musicotherapie sur l’anxiete, la depression des patients atteints de la maladie d’Alzheimer et sur la charge ressentie par l’accompagnant principal 35(1), 57–65 (2009)

    Google Scholar 

  16. Hamann, S., Canli, T.: Individual differences in emotion processing. Current Opinion in Neurobiology 14(2), 233–238 (2004)

    Article  Google Scholar 

  17. Higuchi, T.: Approach to an irregular time series on the basis of the fractal theory. Physica D: Nonlinear Phenomena 31(2), 277–283 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  18. Horlings, R.: Emotion recognition using brain activity. Ph.D. thesis, Delft University of Technology (2008)

    Google Scholar 

  19. IDM-Project: Emotion-based personalized digital media experience in co-spaces (2008), http://www3.ntu.edu.sg/home/eosourina/CHCILab/projects.html

  20. James, W.: What is an emotion. Mind 9(34), 188–205 (1984)

    Google Scholar 

  21. Jones, N.A., Fox, N.A.: Electroencephalogram asymmetry during emotionally evocative films and its relation to positive and negative affectivity. Brain and Cognition 20(2), 280–299 (1992)

    Article  Google Scholar 

  22. Khalili, Z., Moradi, M.H.: Emotion recognition system using brain and peripheral signals: Using correlation dimension to improve the results of eeg. In: Proceedings of the International Joint Conference on Neural Networks, pp. 1571–1575 (2009)

    Google Scholar 

  23. Kulish, V., Sourin, A., Sourina, O.: Analysis and visualization of human electroencephalograms seen as fractal time series. Journal of Mechanics in Medicine and Biology 26(2), 175–188 (2006)

    Article  Google Scholar 

  24. Kulish, V., Sourin, A., Sourina, O.: Human electroencephalograms seen as fractal time series: Mathematical analysis and visualization. Computers in Biology and Medicine 36(3), 291–302 (2006)

    Article  Google Scholar 

  25. Lane, R.D., Reiman, E.M., Bradley, M.M., Lang, P.J., Ahern, G.L., Davidson, R.J., Schwartz, G.E.: Neuroanatomical correlates of pleasant and unpleasant emotion. Neuropsychologia 35(11), 1437–1444 (1997)

    Article  Google Scholar 

  26. Lang, P., Bradley, M., Cuthbert, B.: International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Tech. rep., University of Florida, Gainesville, FL (2008)

    Google Scholar 

  27. Li, M., Chai, Q., Kaixiang, T., Wahab, A.: EEG emotion recognition system. In: In-Vehicle Corpus and Signal Processing for Driver Behavior, pp. 125–135. Springer, US (2009)

    Chapter  Google Scholar 

  28. Lin, Y.P., Wang, C.H., Wu, T.L., Jeng, S.K., Chen, J.H.: EEG-based emotion recognition in music listening: A comparison of schemes for multiclass support vector machine. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, Taipei, pp. 489–492 (2009)

    Google Scholar 

  29. Liu, Y., Sourina, O., Nguyen, M.K.: Real-time EEG-based human emotion recognition and visualization. In: Proc. 2010 Int. Conf. on Cyberworlds, Singapore, pp. 262–269 (2010)

    Google Scholar 

  30. Lutzenberger, W., Elbert, T., Birbaumer, N., Ray, W.J., Schupp, H.: The scalp distribution of the fractal dimension of the EEG and its variation with mental tasks. Brain Topography 5(1), 27–34 (1992)

    Article  Google Scholar 

  31. Maragos, P., Sun, F.-K.: Measuring the fractal dimension of signals: morphological covers and iterative optimization. IEEE Transactions on Signal Processing 41(1), 108–121 (1993)

    Article  MATH  Google Scholar 

  32. Mauss, I.B., Robinson, M.D.: Measures of emotion: A review. Cognition and Emotion 23(2), 209–237 (2009)

    Article  Google Scholar 

  33. Murugappan, M., Rizon, M., Nagarajan, R., Yaacob, S., Zunaidi, I., Hazry, D.: Lifting scheme for human emotion recognition using EEG. In: International Symposium on Information Technology, ITSim 2008, vol. 2 (2008)

    Google Scholar 

  34. Pardo, J.V., Pardo, P.J., Raichle, M.E.: Neural correlates of self-induced dysphoria. American Journal of Psychiatry 150(5), 713–719 (1993)

    Article  Google Scholar 

  35. Petrantonakis, P.C., Hadjileontiadis, L.J.: Emotion recognition from EEG using higher order crossings. IEEE Transactions on Information Technology in Biomedicine 14(2), 186–197 (2010)

    Article  Google Scholar 

  36. Plutchik, R.: Emotions and life: perspectives from psychology, biology, and evolution, 1st edn. American Psychological Association, Washington, DC (2003)

    Google Scholar 

  37. Pradhan, N., Narayana Dutt, D.: Use of running fractal dimension for the analysis of changing patterns in electroencephalograms. Computers in Biology and Medicine 23(5), 381–388 (1993)

    Google Scholar 

  38. Russell, J.A.: Affective space is bipolar. Journal of Personality and Social Psychology 37(3), 345–356 (1979)

    Article  Google Scholar 

  39. Russell, J.A.: A circumplex model of affect. Journal of Personality and Social Psychology 39(6), 1161–1178 (1980)

    Article  Google Scholar 

  40. Sammler, D., Grigutsch, M., Fritz, T., Koelsch, S.: Music and emotion: Electrophysiological correlates of the processing of pleasant and unpleasant music. Psychophysiology 44(2), 293–304 (2007)

    Article  Google Scholar 

  41. Sanei, S., Chambers, J.: EEG signal processing. John Wiley & Sons, Chichester (2007)

    Book  Google Scholar 

  42. Savran, A., Ciftci, K., Chanel, G., Mota, J., Viet, L., Sankur, B., Akarun, L., Caplier, A., Rombaut, M.: Emotion detection in the loop from brain signals and facial images (2006), http://www.enterface.net/results/

  43. Schaaff, K.: EEG-based emotion recognition. Ph.D. thesis, Universitat Karlsruhe (TH) (2008)

    Google Scholar 

  44. Sourina, O., Kulish, V.V., Sourin, A.: Novel tools for quantification of brain responses to music stimuli. In: Proc. of 13th International Conference on Biomedical Engineering, ICBME 2008, pp. 411–414 (2008)

    Google Scholar 

  45. Sourina, O., Liu, Y.: A fractal-based algorithm of emotion recognition from EEG using arousal-valence model. In: Biosignals 2011, Rome, Italy (accepted, 2011)

    Google Scholar 

  46. Sourina, O., Sourin, A., Kulish, V.: EEG data driven animation and its application. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2009. LNCS, vol. 5496, pp. 380–388. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  47. Sourina, O., Wang, Q., Liu, Y., Nguyen, M.K.: A real-time fracal-based brain state recognition from EEG and its application. In: Biosignals 2011, Rome, Italy (accepted, 2011)

    Google Scholar 

  48. Sourina, O., Wang, Q., Nguyen, M.K.: EEG-based ”serious” games and monitoring tools for pain management. In: Proc. MMVR18, Newport Beach, California (accepted, 2011)

    Google Scholar 

  49. Stam, C.J.: Nonlinear dynamical analysis of EEG and MEG: Review of an emerging field. Clinical Neurophysiology 116(10), 2266–2301 (2005)

    Article  Google Scholar 

  50. Stevens, K.: Patients’ perceptions of music during surgery. Journal of Advanced Nursing 15(9), 1045–1051 (1990)

    Article  Google Scholar 

  51. Takahashi, K.: Remarks on emotion recognition from multi-modal bio-potential signals. In: IEEE International Conference on Industrial Technology, IEEE ICIT 2004, vol. 3, pp. 1138–1143 (2004)

    Google Scholar 

  52. Wang, Q., Sourina, O., Nguyen, M.K.: EEG-based ”serious” games design for medical applications. In: Proc. 2010 Int. Conf. on Cyberworlds, Singapore, pp. 270–276 (2010)

    Google Scholar 

  53. Wang, Q., Sourina, O., Nguyen, M.K.: Fractal dimension based algorithm for neurofeedback games. In: Proc. CGI 2010, Singapore, p. SP25 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Nanyang Technological University, Singapore

    Yisi Liu, Olga Sourina & Minh Khoa Nguyen

Authors
  1. Yisi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olga Sourina
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minh Khoa Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Calgary, 2500 University Drive N.W., T2N 1N4, Calgary, AB, Canada

    Marina L. Gavrilova

  2. Exascala Ltd., Unit 9, 97 Rickman Drive, B15 2AL, Birmingham, UK

    C. J. Kenneth Tan

  3. School of Computer Engineering, Division of Computer Science, Nanyang Technological University, Singapore

    Alexei Sourin

  4. Nanyang Technological University, Nanyang Ave., Singapore

    Olga Sourina

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Liu, Y., Sourina, O., Nguyen, M.K. (2011). Real-Time EEG-Based Emotion Recognition and Its Applications. In: Gavrilova, M.L., Tan, C.J.K., Sourin, A., Sourina, O. (eds) Transactions on Computational Science XII. Lecture Notes in Computer Science, vol 6670. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22336-5_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-22336-5_13

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-22335-8

  • Online ISBN: 978-3-642-22336-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation