Advertisement

Cognitive Processing

, Volume 20, Issue 4, pp 405–417 | Cite as

Improving the accuracy of EEG emotion recognition by combining valence lateralization and ensemble learning with tuning parameters

  • Evi Septiana PaneEmail author
  • Adhi Dharma Wibawa
  • Mauridhi Hery Purnomo
Research Article
  • 56 Downloads

Abstract

For emotion recognition using EEG signals, the challenge is improving accuracy. This study proposes strategies that concentrate on incorporating emotion lateralization and ensemble learning approach to enhance the accuracy of EEG-based emotion recognition. In this paper, we obtained EEG signals from an EEG-based public emotion dataset with four classes (i.e. happy, sad, angry and relaxed). The EEG signal is acquired from pair asymmetry channels from left and right hemispheres. EEG features were extracted using a hybrid features extraction from three domains, namely time, frequency and wavelet. To demonstrate the lateralization, we performed a set of four experimental scenarios, i.e. without lateralization, right-/left-dominance lateralization, valence lateralization and others lateralization. For emotion classification, we use random forest (RF), which is known as the best classifier in ensemble learning. Tuning parameters in the RF model were done by grid search optimization. As a comparison of RF, we employed two prevalent algorithms in EEG, namely SVM and LDA. Emotion classification accuracy increased significantly from without lateralization to the valence lateralization using three pairs of asymmetry channel, i.e. T7–T8, C3–C4 and O1–O2. For the classification, the RF method provides the highest accuracy of 75.6% compared to SVM of 69.8% and LDA of 60.4%. In addition, the features of energy–entropy from wavelet are important for EEG emotion recognition. This study yields a significant performance improvement of EEG-based emotion recognition by the valence emotion lateralization. It indicates that happy and relaxed emotions are dominant in the left hemisphere, while angry and sad emotions are better recognized from the right hemisphere.

Keywords

Emotional recognition Brain signals Lateralization Brain asymmetry Random forest 

Notes

Acknowledgements

We would like to thank the Indonesia Endowment Fund for Education (LPDP), the Republic of Indonesia, which has provided funding through the Indonesia Education Scholarship granted to the first author.

Funding

This study was funded by Indonesia Endowment Fund for Education, contract number PRJ-2173/LPDP/2015.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors. The data being used in this article is the public dataset which can be accessed online and also cited as one of the references.

Informed consent

Informed consent was obtained from all individual participants included in the study performed by the dataset creator, not the author of these articles.

References

  1. Ahern GL, Schwartz GE (1985) Differential lateralization for positive and negative emotion in the human brain: EEG spectral analysis. Neuropsychologia 23:745–755CrossRefGoogle Scholar
  2. Altenmüller E, Schürmann K, Lim VK, Parlitz D (2002) Hits to the left, flops to the right: different emotions during listening to music are reflected in cortical lateralisation patterns. Neuropsychologia 40:2242–2256CrossRefGoogle Scholar
  3. Alves NT, Fukusima SS, Aznar-Casanova JA (2008) Models of brain asymmetry in emotional processing. Psychol Neurosci 1:63–66Google Scholar
  4. Atkinson J, Campos D (2016) Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers. Expert Syst Appl 47:35–41.  https://doi.org/10.1016/j.eswa.2015.10.049 CrossRefGoogle Scholar
  5. Berk LS, Felten DL, Tan SA, Bittman BB, Westengard J (2001) Modulation of neuroimmune parameters during the eustress of humor-associated mirthful laughter. Altern Ther Health Med 7:62–76PubMedGoogle Scholar
  6. Bernard S, Adam S, Heutte L (2012) Dynamic random forests. Pattern Recognit Lett 33:1580–1586CrossRefGoogle Scholar
  7. Bhatti AM, Majid M, Anwar SM, Khan B (2016) Human emotion recognition and analysis in response to audio music using brain signals. Comput Hum Behav 65:267–275.  https://doi.org/10.1016/j.chb.2016.08.029 CrossRefGoogle Scholar
  8. Borod JC, Kent J, Koff E, Martin C, Alpert M (1988) Facial asymmetry while posing positive and negative emotions: support for the right hemisphere hypothesis. Neuropsychologia 26:759–764.  https://doi.org/10.1016/0028-3932(88)90013-9 CrossRefPubMedGoogle Scholar
  9. Bos DO (2006) EEG-based emotion recognition. Influ Vis Audit Stimul 56:1–17Google Scholar
  10. Bradley MM, Lang PJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry 25:49–59CrossRefGoogle Scholar
  11. Breiman L (2001) Random forests. Mach Learn 45:5–32CrossRefGoogle Scholar
  12. Brosschot JF, Thayer JF (2003) Heart rate response is longer after negative emotions than after positive emotions. Int J Psychophysiol 50:181–187.  https://doi.org/10.1016/S0167-8760(03)00146-6 CrossRefPubMedGoogle Scholar
  13. Candra H, Yuwono M, Chai R, Handojoseno A, Elamvazuthi I, Nguyen HT, Su S (2015) Investigation of window size in classification of EEG-emotion signal with wavelet entropy and support vector machine. In: 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 7250–7253Google Scholar
  14. Castillo JC, Castro-González Á, Fernández-Caballero A, Latorre JM, Pastor JM, Fernández-Sotos A, Salichs MA (2016) Software architecture for smart emotion recognition and regulation of the ageing adult. Cogn Comput 8:357–367.  https://doi.org/10.1007/s12559-016-9383-y CrossRefGoogle Scholar
  15. Davidson RJ, Schwartz GE, Saron C, Bennett J, Goleman DJ (1979) Frontal vs. parietal EEG asymmetry during positive and negative affect. Pyscophysiology 16:202–203Google Scholar
  16. Demaree HA, Everhart DE, Youngstrom EA, Harrison DW (2005) Brain lateralization of emotional processing: historical roots and a future incorporating “Dominance”. Behav Cogn Neurosci Rev 4:3–20.  https://doi.org/10.1177/1534582305276837 CrossRefPubMedGoogle Scholar
  17. Diykh M, Li Y, Wen P (2016) EEG sleep stages classification based on time domain features and structural graph similarity. IEEE Trans Neural Syst Rehabil Eng 24:1159–1168.  https://doi.org/10.1109/TNSRE.2016.2552539 CrossRefPubMedGoogle Scholar
  18. Higuchi T (1988) Approach to an irregular time series on the basis of the fractal theory. Physica D 31:277–283CrossRefGoogle Scholar
  19. Hjorth B (1970) EEG analysis based on time domain properties. Electroencephalogr Clin Neurophysiol 29:306–310CrossRefGoogle Scholar
  20. Ibanez A, Melloni M, Huepe D, Helgiu E, Rivera-Rei A, Canales-Johnson A, Baker P, Moya A (2012) What event-related potentials (ERPs) bring to social neuroscience? Soc Neurosci 7:632–649.  https://doi.org/10.1080/17470919.2012.691078 CrossRefPubMedGoogle Scholar
  21. Jenke R, Peer A, Buss M (2014) Feature extraction and selection for emotion recognition from EEG. IEEE Trans Affect Comput 5:327–339.  https://doi.org/10.1109/TAFFC.2014.2339834 CrossRefGoogle Scholar
  22. Kaiser JF (1990) On a simple algorithm to calculate the ‘energy’ of a signal. In: 1990 international conference on acoustics, speech, and signal processing, 1990. ICASSP-90. IEEE, pp 381–384Google Scholar
  23. Koelstra S, Muhl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) DEAP: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3:18–31CrossRefGoogle Scholar
  24. Koh KB (1998) Emotion and immunity. J Psychosom Res 45:107–115CrossRefGoogle Scholar
  25. Kononenko I (2001) Machine learning for medical diagnosis: history, state of the art and perspective. Artif Intell Med 23:89–109CrossRefGoogle Scholar
  26. Li J, Zhang Z, He H (2017) Hierarchical convolutional neural networks for EEG-based emotion recognition. Cogn Comput 10:368–380.  https://doi.org/10.1007/s12559-017-9533-x CrossRefGoogle Scholar
  27. Lin Y-P, Wang C-H, Jung T-P, Wu T-L, Jeng S-K, Duann J-R, Chen J-H (2010) EEG-based emotion recognition in music listening. IEEE Trans Biomed Eng 57:1798–1806.  https://doi.org/10.1109/TBME.2010.2048568 CrossRefPubMedGoogle Scholar
  28. Liu Y, Sourina O, Nguyen MK (2011) Real-time EEG-based emotion recognition and its applications. In: Gavrilova ML, Tan CJK, Sourin A, Sourina O (eds) Transactions on computational science XII. Springer, Berlin, Heidelberg, pp 256–277CrossRefGoogle Scholar
  29. Mohammadi Z, Frounchi J, Amiri M (2016) Wavelet-based emotion recognition system using EEG signal. Neural Comput Appl.  https://doi.org/10.1007/s00521-015-2149-8 CrossRefGoogle Scholar
  30. Olofsson JK, Nordin S, Sequeira H, Polich J (2008) Affective picture processing: an integrative review of ERP findings. Biol Psychol 77:247–265.  https://doi.org/10.1016/j.biopsycho.2007.11.006 CrossRefPubMedGoogle Scholar
  31. Özerdem MS, Polat H (2017) Emotion recognition based on EEG features in movie clips with channel selection. Brain Informatics 4:241–252.  https://doi.org/10.1007/s40708-017-0069-3 CrossRefPubMedPubMedCentralGoogle Scholar
  32. Pane ES, Hendrawan MA, Wibawa AD, Purnomo MH (2017) Identifying rules for electroencephalograph (EEG) emotion recognition and classification. In: 2017 5th international conference on instrumentation, communications, information technology, and biomedical engineering (ICICI-BME). IEEE, pp 167–172.  https://doi.org/10.1109/ICICI-BME.2017.8537731
  33. Petrantonakis PC, Hadjileontiadis LJ (2010) Emotion recognition from EEG using higher order crossings. IEEE Trans Inf Technol Biomed 14:186–197.  https://doi.org/10.1109/TITB.2009.2034649 CrossRefPubMedGoogle Scholar
  34. Pires P, Mendes L, Mendes J, Rodrigues R, Pereira A (2015) Integrated e-healthcare system for elderly support. Cogn Comput 8:368–384.  https://doi.org/10.1007/s12559-015-9367-3 CrossRefGoogle Scholar
  35. Pressman SD, Cohen S (2005) Does positive affect influence health? Psychol Bull 131:925–971.  https://doi.org/10.1037/0033-2909.131.6.925 CrossRefPubMedGoogle Scholar
  36. Purves D (ed) (2004) Neuroscience, 3rd edn. Sinauer Associates, Publishers, SunderlandGoogle Scholar
  37. Qi Y (2012) Random forest for bioinformatics. In: Zhang C, Ma Y (eds) Ensemble machine learning. Springer, Boston, MA, pp 307–323CrossRefGoogle Scholar
  38. Schmidt LA, Trainor LJ (2001) Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cogn Emot 15:487–500.  https://doi.org/10.1080/0269993004200187 CrossRefGoogle Scholar
  39. Schwartz GE, Davidson RJ, Maer F (1975) Right hemisphere lateralization for emotion in the human brain: interactions with cognition. Science 190:286–288CrossRefGoogle Scholar
  40. Wager TD, Phan KL, Liberzon I, Taylor SF (2003) Valence, gender, and lateralization of functional brain anatomy in emotion: a meta-analysis of findings from neuroimaging. NeuroImage 19:513–531.  https://doi.org/10.1016/S1053-8119(03)00078-8 CrossRefPubMedGoogle Scholar
  41. Wang X-W, Nie D, Lu B-L (2014) Emotional state classification from EEG data using machine learning approach. Neurocomputing 129:94–106.  https://doi.org/10.1016/j.neucom.2013.06.046 CrossRefGoogle Scholar
  42. Wibawa AD, Purnomo MH, Marzuki A, Rumpa LD (2016) Physiological pattern of human state emotion based on ECG and pulse sensor. J Theor Appl Inf Technol 93(1)Google Scholar
  43. Zhang J, Chen M, Zhao S, Hu S, Shi Z, Cao Y (2016a) ReliefF-based EEG sensor selection methods for emotion recognition. Sensors 16:1558.  https://doi.org/10.3390/s16101558 CrossRefGoogle Scholar
  44. Zhang Y, Ji X, Zhang S (2016b) An approach to EEG-based emotion recognition using combined feature extraction method. Neurosci Lett 633:152–157.  https://doi.org/10.1016/j.neulet.2016.09.037 CrossRefPubMedGoogle Scholar
  45. Zheng W (2017) Multichannel EEG-based emotion recognition via group sparse canonical correlation analysis. IEEE Trans Cogn Dev Syst 9:281–290.  https://doi.org/10.1109/TCDS.2016.2587290 CrossRefGoogle Scholar
  46. Zheng W-L, Lu B-L (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7:162–175.  https://doi.org/10.1109/TAMD.2015.2431497 CrossRefGoogle Scholar

Copyright information

© Marta Olivetti Belardinelli and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Evi Septiana Pane
    • 1
    • 3
    Email author
  • Adhi Dharma Wibawa
    • 1
    • 2
  • Mauridhi Hery Purnomo
    • 1
    • 2
  1. 1.Department of Electrical EngineeringInstitut Teknologi Sepuluh NopemberSurabayaIndonesia
  2. 2.Department of Computer EngineeringInstitut Teknologi Sepuluh NopemberSurabayaIndonesia
  3. 3.Industrial Training and Education of SurabayaMinistry of IndustrySurabayaIndonesia

Personalised recommendations