Skip to main content
SpringerLink
Log in

EEG-based emotion recognition using modified covariance and ensemble classifiers

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

The Electroencephalography (EEG)-based precise emotion identification is one of the most challenging tasks in pattern recognition. In this paper, an innovative EEG signal processing method is devised for an automated emotion identification. The Symlets-4 filters based “Multi Scale Principal Component Analysis” (MSPCA) is used to denoise and reduce the raw signal’s dimension. Onward, the “Modified Covariance” (MCOV) is used as a feature extractor. In the classification step, the ensemble classifiers are used. The proposed method achieved 99.6% classification accuracy by using the ensemble of Bagging and Random Forest (RF). It confirms effectiveness of the devised method in EEG-based emotion recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data availability

The dataset used in this paper is publicly available at: https://bcmi.sjtu.edu.cn/home/seed/.

References

  • Alfaro E, Gámez M, García N (2018) Ensemble classification methods with applications in R. Wiley, Hoboken

    Book  Google Scholar 

  • Alpaydin E (2014) Introduction to machine learning. MIT press, Cambridge

    Google Scholar 

  • Alsberg BK, Woodward AM, Winson MK, Rowland JJ, Kell DB (1998) Variable selection in wavelet regression models. Anal Chim Acta 368(1):29–44

    Article  CAS  Google Scholar 

  • Al Zoubi O, Awad M, Kasabov NK (2018) Anytime multipurpose emotion recognition from EEG data using a liquid state machine based framework. Artif Intell Med 86:1–8

    Article  PubMed  Google Scholar 

  • Bakshi BR (1998) Multiscale PCA with application to multivariate statistical process monitoring. AIChE J 44(7):1596–1610

    Article  ADS  CAS  Google Scholar 

  • Bakshi BR, Bansal P, Nounou MN (1997) Multiscale rectification of random errors without fundamental process models. Comput Chem Eng 21:S1167–S1172

    Article  CAS  Google Scholar 

  • Bauer E, Kohavi R (1999) An empirical comparison of voting classification algorithms: bagging, boosting, and variants. Mach Learn 36(1):105–139

    Article  Google Scholar 

  • Bonat WH, Jørgensen B (2016) Multivariate covariance generalized linear models. J Roy Stat Soc: Ser C (Appl Stat) 65(5):649–675

    MathSciNet  Google Scholar 

  • Chai X, Wang Q, Zhao Y, Li Y, Liu D, Liu X, Bai O (2017) A fast, efficient domain adaptation technique for cross-domain electroencephalography (EEG)-based emotion recognition. Sensors 17(5):1014

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  • Chawla NV, Japkowicz N, Kotcz A (2004) Special issue on learning from imbalanced data sets. ACM SIGKDD Explor Newsl 6(1):1–6

    Article  Google Scholar 

  • Cichosz P (2014) Data mining algorithms: explained using R. Wiley, Hoboken

    Google Scholar 

  • Daimi SN, Saha G (2014) Classification of emotions induced by music videos and correlation with participants’ rating. Expert Syst Appl 41(13):6057–6065

    Article  Google Scholar 

  • Eibe F, Hall MA, Witten IH (2016) The WEKA workbench. Online appendix for data mining: practical machine learning tools and techniques. In: Morgan Kaufmann. Elsevier Amsterdam, The Netherlands

  • Gao Z, Wang X, Yang Y, Li Y, Ma K, Chen G (2020) A channel-fused dense convolutional network for EEG-based emotion recognition. IEEE Trans Cogn Dev Syst 13(4):945–954

    Article  Google Scholar 

  • Hall M, Witten I, Frank E (2011) Data mining: practical machine learning tools and techniques. Kaufmann, Burlington

    Google Scholar 

  • Heidemann L, Schwaiger A, Roscher K (2021) Measuring ensemble diversity and its effects on model robustness. AISafety@ IJCAI, Vienna

    Google Scholar 

  • Ho TK (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20(8):832–844

    Article  Google Scholar 

  • Hwang S, Hong K, Son G, Byun H (2020) Learning CNN features from DE features for EEG-based emotion recognition. Pattern Anal Appl 23(3):1323–1335

    Article  Google Scholar 

  • Kamble KS, Sengupta J (2021) Ensemble machine learning-based affective computing for emotion recognition using dual-decomposed EEG signals. IEEE Sens J 22(3):2496–2507

    Article  ADS  Google Scholar 

  • Kanjo E, Younis EM, Ang CS (2019) Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection. Inform Fusion 49:46–56

    Article  Google Scholar 

  • Katsigiannis S, Ramzan N (2018) Dreamer: a database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices. IEEE J Biomed Health Inf 22(1):98–107

    Article  Google Scholar 

  • Khare SK, Bajaj V, Sinha GR (2020) Adaptive tunable Q wavelet transform-based emotion identification. IEEE Trans Instrum Meas 69(12):9609–9617

    Article  ADS  Google Scholar 

  • Khare S, Nishad A, Upadhyay A, Bajaj V (2020) Classification of emotions from EEG signals using time-order representation based on the S‐transform and convolutional neural network. Electron Lett 56(25):1359–1361

    Article  ADS  Google Scholar 

  • Kim S-K, Kang H-B (2018) An analysis of smartphone overuse recognition in terms of emotions using brainwaves and deep learning. Neurocomputing 275:1393–1406

    Article  Google Scholar 

  • Kumar M, Molinas M (2022) Human emotion recognition from EEG signals: Model evaluation in DEAP and SEED datasets. In: Proceedings of the first workshop on artificial intelligence for human-machine interaction (AIxHMI 2022) co-located with the 21th international conference of the Italian association for artificial intelligence (AI* IA 2022), CEUR workshop proceedings, CEUR-WS. org

  • Liu W, Zhang L, Tao D, Cheng J (2018) Reinforcement online learning for emotion prediction by using physiological signals. Pattern Recognit Lett 107:123–130

    Article  ADS  Google Scholar 

  • Liu Y-J, Yu M, Zhao G, Song J, Ge Y, Shi Y (2018) Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Trans Affect Comput 9(4):550–562

    Article  Google Scholar 

  • Malinowski ER (2002) Factor analysis in chemistry. Wiley, New York

    Google Scholar 

  • Mert A, Akan A (2018) Emotion recognition based on time–frequency distribution of EEG signals using multivariate synchrosqueezing transform. Digit Signal Proc 81:106–115

    Article  Google Scholar 

  • Mian Qaisar S, Subasi A (2020) Effective epileptic seizure detection based on the event-driven processing and machine learning for mobile healthcare. J Ambient Intell Humaniz Comput 13:3619–3631

    Article  Google Scholar 

  • Mian Qaisar S, Hussain SF (2021) An effective arrhythmia classification via ECG signal subsampling and mutual information based subbands statistical features selection. J Ambient Intell Humaniz Comput 14(3):1473–1487

    Article  Google Scholar 

  • Nakisa B, Rastgoo MN, Tjondronegoro D, Chandran V (2018) Evolutionary computation algorithms for feature selection of EEG-based emotion recognition using mobile sensors. Expert Syst Appl 93:143–155

    Article  Google Scholar 

  • Nandi A, Xhafa F, Subirats L, Fort S (2021) Real-time emotion classification using eeg data stream in e-learning contexts. Sensors 21(5):1589

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  • Peng Y, Kong W, Qin F, Nie F, Fang J, Lu BL, Cichocki A (2021) Self-weighted semi-supervised classification for joint eeg-based emotion recognition and affective activation patterns mining. IEEE Trans Instrum Meas 70:1–11

    Google Scholar 

  • Rodriguez JJ, Kuncheva LI, Alonso CJ (2006) Rotation forest: a new classifier ensemble method. IEEE Trans Pattern Anal Mach Intell 28(10):1619–1630

    Article  PubMed  Google Scholar 

  • Salzberg SL (1997) On comparing classifiers: pitfalls to avoid and a recommended approach. Data Min Knowl Disc 1(3):3

    Article  Google Scholar 

  • Selesnick IW (2011) Wavelet transform with tunable Q-factor. IEEE Trans Signal Process 59(8):3560–3575

    Article  ADS  MathSciNet  Google Scholar 

  • Seth D, Chakraborty D, Ghosal P, Sanyal SK (2017) Brain computer interfacing: a spectrum estimation based neurophysiological signal interpretation. In: 2017 4th international conference on signal processing and integrated networks (SPIN). IEEE, pp 534–539. https://doi.org/10.1109/SPIN.2017.8050008

  • Skurichina M, Duin RP (2002) Bagging, boosting and the random subspace method for linear classifiers. Pattern Anal Appl 5(2):121–135

    Article  MathSciNet  Google Scholar 

  • Sokolova M, Japkowicz N, Szpakowicz S (2006) Beyond accuracy, F-score and ROC: a family of discriminant measures for performance evaluation. In: Australasian joint conference on artificial intelligence. Springer, Berlin, Heidelberg, pp 1015–1021. https://doi.org/10.1007/11941439_114

  • Su Y, Chen P, Liu X, Li W, Lv Z (2018) A spatial filtering approach to environmental emotion perception based on electroencephalography. Med Eng Phys 60:77–85

    Article  PubMed  Google Scholar 

  • Subasi A, Qaisar SM (2020) Heartbeat classification using parametric and time–frequency methods. In: Modelling and analysis of active biopotential signals in healthcare, vol 2. IOP Publishing, Bristol, UK. https://doi.org/10.1088/978-0-7503-3411-2ch11

  • Subasi A, Bandic L, Qaisar SM (2020) Cloud-based health monitoring framework using smart sensors and smartphone. In: Innovation in health informatics. Elsevier, pp 217–243

  • Subasi A, Tuncer T, Dogan S, Tanko D, Sakoglu U (2021) EEG-based emotion recognition using tunable Q wavelet transform and rotation forest ensemble classifier. Biomed Signal Process Control 68:102648

    Article  Google Scholar 

  • Tripathi S, Muhr D, Brunner M, Jodlbauer H, Dehmer M, Emmert-Streib F (2021) Ensuring the robustness and reliability of data-driven knowledge discovery models in production and manufacturing. Front Artif Intell 4:22

    Article  Google Scholar 

  • Trygg J, Wold S (1998) PLS regression on wavelet compressed NIR spectra. Chemometr Intell Lab Syst 42(1–2):209–220. https://doi.org/10.1016/S0169-7439(98)00013-6

    Article  CAS  Google Scholar 

  • Tuncer T, Dogan S, Subasi A (2021) A new fractal pattern feature generation function based emotion recognition method using EEG. Chaos Solitons Fractals 144:110671. https://doi.org/10.1016/j.chaos.2021.110671

    Article  MathSciNet  Google Scholar 

  • Tuncer T, Dogan S, Subasi A (2021) LEDPatNet19: automated emotion recognition model based on nonlinear LED pattern feature extraction function using EEG signals. Cogn Neurodyn. https://doi.org/10.1007/s11571-021-09748-0

    Article  PubMed  PubMed Central  Google Scholar 

  • Wang F, Wu S, Zhang W, Xu Z, Zhang Y, Wu C, Coleman S (2020) Emotion recognition with convolutional neural network and EEG-based EFDMs. Neuropsychologia 146:107506

    Article  PubMed  Google Scholar 

  • Webb GI (2000) Multiboosting: a technique for combining boosting and wagging. Mach Learn 40(2):159–196

    Article  Google Scholar 

  • Witten IH, Frank E, Hall MA, Pal CJ (2016) Data Mining: practical machine learning tools and techniques. Morgan Kaufmann, Burlington

    Google Scholar 

  • Wu X, Zheng W-L, Li Z, Lu B-L (2022) Investigating EEG-based functional connectivity patterns for multimodal emotion recognition. J Neural Eng 19(1):016012

    Article  Google Scholar 

  • Yang WY, Cao W, Kim J, Park KW, Park H-H, Joung J, Ro J-S, Lee HL, Hong C-H, Im T (2020) Applied numerical methods using MATLAB. Wiley, Hoboken

    Book  Google Scholar 

  • Yin Z, Zhao M, Wang Y, Yang J, Zhang J (2017) Recognition of emotions using multimodal physiological signals and an ensemble deep learning model. Comput Methods Programs Biomed 140:93–110

    Article  PubMed  Google Scholar 

  • Zhao G, Ge Y, Shen B, Wei X, Wang H (2018) Emotion analysis for personality inference from EEG signals. IEEE Trans Affect Comput 9(3):362–371

    Article  Google Scholar 

  • Zhao X, Huang W, Banks A, Cox V, Flynn D, Schewe S, Huang X (2021) Assessing the reliability of deep learning classifiers through robustness evaluation and operational profiles. Preprint at https://arxiv.org/abs/2106.01258

  • Zheng W-L, Lu B-L (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7(3):162–175

    Article  Google Scholar 

  • Zhong P, Wang D, Miao C (2020) EEG-based emotion recognition using regularized graph neural networks. IEEE Trans Affect Comput 13:1290–1301. https://doi.org/10.1109/TAFFC.2020.2994159

    Article  Google Scholar 

Download references

Funding

This work was supported by Effat University with the Decision Number of UC#9/29 April.2020/7.1–22(2)5, Jeddah, Saudi Arabia.

Author information

Authors and Affiliations

  1. Institute of Biomedicine, Faculty of Medicine, University of Turku, 20520, Turku, Finland

    Abdulhamit Subasi

  2. CESI LINEACT, 69100, Lyon, France

    Saeed Mian Qaisar

  3. College of Engineering, EFfat University, Jeddah, 21478, Saudi Arabia

    Abdulhamit Subasi & Saeed Mian Qaisar

Authors
  1. Abdulhamit Subasi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saeed Mian Qaisar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Abdulhamit Subasi or Saeed Mian Qaisar.

Ethics declarations

Conflict of interest

Authors declare no conflict of interest. The second author (S. M. Qaisar) has recently moved to the  CESI LINEACT, France and certain parts of the manuscript were written or revised while he was with the Effat University.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Subasi, A., Mian Qaisar, S. EEG-based emotion recognition using modified covariance and ensemble classifiers. J Ambient Intell Human Comput 15, 575–591 (2024). https://doi.org/10.1007/s12652-023-04715-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-023-04715-5

Keywords

Search

Navigation