Skip to main content
SpringerLink
Log in

A hybrid ensemble voting-based residual attention network for motor imagery EEG Classification

  • Published:
Analog Integrated Circuits and Signal Processing Aims and scope Submit manuscript

Abstract

Multi-class motor imagery Electroencephalography (EEG) activity decoding has always been challenging for the development of Brain Computer Interface (BCI) system. Deep learning has recently emerged as a powerful approach for BCI system development using EEG activity. However, the EEG activity analysis and classification should be robust, automated and accurate. Currently, available BCI systems perform well for binary task identification whereas, multi-class classification of EEG activity for BCI applications is still a challenging task. In this work, a hybrid residual attention ensemble voting classifier model is developed for EEG-based Motor Imagery-Brain Computer Interface (MI-BCI) task classification. The Time–Frequency Representation (TFR) of the multi-class EEG activity is generated using Transient Extracting Transform. The TFR spectrogram images are input to the designed residual attention ensemble voting classifier model for training and classification purposes. The model is evaluated using different fusion strategies viz. feature-level and score-level fusion of layers. The proposed model is evaluated on two MI-BCI datasets, BCI competition IV 2a and BCI competition III 3a, yielding the highest classification accuracies of 88.14% and 93.13%, respectively. The results obtained on a large multi-class MI-BCI dataset confirm that the proposed hybrid residual attention ensemble voting classifier model significantly outperforms the conventional algorithm and achieves significantly high classification accuracy for the feature-level fusion of layers. The developed framework aids in identifying different tasks for multi-class MI-BCI EEG activity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Data availability

This article does not contain any studies with human participants or animals performed by any of the authors. However, authors have collected BCI datasets from various publicly available databases for analysis purpose. The references to the publicly available databases have been cited appropriately in the manuscript.

References

  1. Khorshidtalab, A., Salami, M. J. E., & Akmeliawati, R. (2017). Motor imagery task classification using transformation based features. Biomedical Signal Processing and Control, 33, 213–219. https://doi.org/10.1016/j.bspc.2016.12.006

    Article  Google Scholar 

  2. Jindal, K., & Upadhyay, R. (2017, July). Epileptic seizure detection from EEG signal using Flexible Analytical Wavelet Transform. In 2017 International Conference on Computer, Communications and Electronics (Comptelix) (pp. 67-72). https://doi.org/10.1109/COMPTELIX.2017.8003940

  3. Blanco, J. A., Vanleer, A. C., Calibo, T. K., & Firebaugh, S. L. (2019). Single-trial cognitive stress classification using portable wireless electroencephalography. Sensors. https://doi.org/10.3390/s19030499

    Article  Google Scholar 

  4. Khare, S., & Bajaj, V. (2020). Adaptive Tunable Q Wavelet Transform based Emotion Identification. IEEE Transactions on Instrumentation and Measurement. Vol. 9456(c), (pp.1–1). https://doi.org/10.1109/TIM.2020.3006611

  5. He, L., Liu, B., Hu, D., Wen, Y., Wan, M., & Long, J. (2016). Motor imagery EEG signals analysis based on Bayesian network with Gaussian distribution. Neurocomputing, 188, 217–224. https://doi.org/10.1016/j.neucom.2015.05.133

    Article  Google Scholar 

  6. Atul, D. Verma. (2018). EEGNet: A Compact Convolutional NN for EEG-based BCI. On certain distance and degree based topological indices of Zeolite LTA frameworks. (11–14).

  7. Nguyen, T., Hettiarachchi, I., Khatami, A., Gordon-Brown, L., Lim, C. P., & Nahavandi, S. (2018). Classification of multi-class BCI data by common spatial pattern and fuzzy system. IEEE Access, 6, 27873–27884. https://doi.org/10.1109/ACCESS.2018.2841051

    Article  Google Scholar 

  8. Ang, K. K., Chin, Z. Y., Zhang, H., & Guan, C. (2008). Filter Bank Common Spatial Pattern (FBCSP) in brain-computer interface. In Proceedings of the International Joint Conference on Neural Networks. (pp. 2390–2397). https://doi.org/10.1109/IJCNN.2008.4634130

  9. Lotte, F., & Guan, C. (2011). Regularizing common spatial patterns to improve BCI designs: Unified theory and new algorithms. IEEE Transactions on Biomedical Engineering, 58(2), 355–362. https://doi.org/10.1109/TBME.2010.2082539

    Article  Google Scholar 

  10. Robinson, N., Vinod, A. P., Guan, C., Ang, K. K., & Peng, T. K. (2011). A Wavelet-CSP method to classify hand movement directions in EEG based BCI system. In ICICS 2011-8th International Conference on Information, Communications and Signal Processing. (pp. 11–15). https://doi.org/10.1109/ICICS.2011.6174210

  11. Tang, Z. C., Li, C., Wu, J. F., Liu, P. C., & Cheng, S. W. (2019). Classification of EEG-based single-trial motor imagery tasks using a B-CSP method for BCI. Frontiers of Information Technology & Electronic Engineering, 20(8), 1087–1098. https://doi.org/10.1631/FITEE.1800083

    Article  Google Scholar 

  12. Reddy, T. K., Arora, V., Behera, L., Wang, Y. K., & Lin, C. T. (2019). Multiclass fuzzy time-delay common spatio-spectral patterns with fuzzy information theoretic optimization for EEG-based regression problems in brain-computer interface (BCI). IEEE Transactions on Fuzzy Systems, 27(10), 1943–1951. https://doi.org/10.1109/TFUZZ.2019.2892921

    Article  Google Scholar 

  13. Deng, X., Zhang, B., Yu, N., Liu, K., & Sun, K. (2021). Advanced TSGL-EEGNet for motor imagery EEG-based brain-computer interfaces. IEEE Access, 9, 25118–25130. https://doi.org/10.1109/ACCESS.2021.3056088

    Article  Google Scholar 

  14. Bajaj, V., Taran, S., Khare, S. K., & Sengur, A. (2020). Feature extraction method for classification of alertness and drowsiness states EEG signals. Applied Acoustics, 163, 107224. https://doi.org/10.1016/j.apacoust.2020.107224

    Article  Google Scholar 

  15. Ghosh-Dastidar, S., Adeli, H., & Dadmehr, N. (2007). Mixed-band wavelet-chaos-neural network methodology for epilepsy and epileptic seizure detection. IEEE Transactions on Biomedical Engineering, 54(9), 1545–1551. https://doi.org/10.1109/TBME.2007.891945

    Article  Google Scholar 

  16. Sadiq, M. T., Yu, X., Yuan, Z., Aziz, M. Z., Siuly, S., & Ding, W. (2021). Toward the development of versatile brain-computer interfaces. IEEE Transactions on Artificial Intelligence, 2(4), 314–328. https://doi.org/10.1109/tai.2021.3097307

    Article  Google Scholar 

  17. Taran, S., Bajaj, V., Sharma, D., Siuly, S., & Sengur, A. (2018). Features based on analytic IMF for classifying motor imagery EEG signals in BCI applications. Measurement: Journal of the International Measurement Confederation, 116, 68–76. https://doi.org/10.1016/j.measurement.2017.10.067

    Article  Google Scholar 

  18. Hassan, A. R., & Bhuiyan, M. I. H. (2016). A decision support system for automatic sleep staging from EEG signals using tunable Q-factor wavelet transform and spectral features. Journal of Neuroscience Methods, 271, 107–118. https://doi.org/10.1016/j.jneumeth.2016.07.012

    Article  Google Scholar 

  19. Yu, X., Aziz, M. Z., Sadiq, M. T., Fan, Z., & Xiao, G. (2021). A new framework for automatic detection of motor and mental imagery EEG signals for robust BCI systems. IEEE Transactions on Instrumentation and Measurement. https://doi.org/10.1109/TIM.2021.3069026

    Article  Google Scholar 

  20. Sadiq, M. T., Yu, X., Yuan, Z., Fan, Z., Rehman, A. U., Li, G., & Xiao, G. (2019). Motor imagery EEG signals classification based on mode amplitude and frequency components using empirical wavelet transform. IEEE Access, 7, 127678–127692.

    Article  Google Scholar 

  21. Sadiq, M. T., Yu, X., Yuan, Z., Aziz, M. Z., ur Rehman, N., Ding, W., & Xiao, G. (2022). Motor imagery BCI classification based on multivariate variational mode decomposition. IEEE Transactions on Emerging Topics in Computational Intelligence, 6(5), 1177–1189. https://doi.org/10.1109/TETCI.2022.3147030

    Article  Google Scholar 

  22. Sadiq, M. T., Yu, X., Yuan, Z., Zeming, F., Rehman, A. U., Ullah, I., & Xiao, G. (2019). Motor imagery EEG signals decoding by multivariate empirical wavelet transform-based framework for robust brain-computer interfaces. IEEE Access, 7, 171431–171451. https://doi.org/10.1109/ACCESS.2019.2956018

    Article  Google Scholar 

  23. You, Y., Chen, W., & Zhang, T. (2020). Biomedical signal processing and control motor imagery EEG classification based on flexible analytic wavelet transform. Biomedical Signal Processing and Control, 62, 102069. https://doi.org/10.1016/j.bspc.2020.102069

    Article  Google Scholar 

  24. Taran, S., & Bajaj, V. (2019). Motor imagery tasks-based EEG signals classification using tunable-Q wavelet transform. Neural Computing and Applications, 31(11), 6925–6932. https://doi.org/10.1007/s00521-018-3531-0

    Article  Google Scholar 

  25. Yuan, L., Yang, B., & Ma, S. (2009). Combination of Wavelet Packet Transform and Hilbert-Huang Transform for Recognition of Continuous EEG in BCIs. (pp. 5).

  26. Akbari, H., Sadiq, M. T., Jafari, N., Too, J., Mikaeilvand, N., Cicone, A., & Serra-Capizzano, S. (2023). Recognizing seizure using Poincaré plot of EEG signals and graphical features in DWT domain. NeuroRehabilitation, 124(1), 12–24. https://doi.org/10.4149/BLL_2023_002

    Article  Google Scholar 

  27. Akbari, H., Sadiq, M. T., Payan, M., Esmaili, S. S., Baghri, H., & Bagheri, H. (2021). Depression detection based on geometrical features extracted from sodp shape of EEG signals and binary PSO. Traitement du Signal., 38(1), 13–26. https://doi.org/10.18280/TS.380102

    Article  Google Scholar 

  28. Zhang, R., Zong, Q., & Zhao, X. (2019, July). A new convolutional neural network for motor imagery classification. In  Chinese Control Conference CCC, 2019-July(10) (pp. 8428-8432). https://doi.org/10.23919/ChiCC.2019.8865152

  29. Ha, K. W., & Jeong, J. W. (2019). Motor imagery EEG classification using capsule networks. Sensors (Switzerland). https://doi.org/10.3390/s19132854

    Article  Google Scholar 

  30. Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., & Tangermann, M. (2017). Deep learning with convolutional neural networks for EEG decoding and visualization. Human Brain Mapping, 38(11), 5391–5420. https://doi.org/10.1002/hbm.23730

    Article  Google Scholar 

  31. Sadiq, M. T., Akbari, H., Siuly, S., Li, Y., & Wen, P. (2022). Alcoholic EEG signals recognition based on phase space dynamic and geometrical features. Chaos, Solitons and Fractals,. https://doi.org/10.1016/j.chaos.2022.112036

    Article  MathSciNet  Google Scholar 

  32. Yu, G. (2020). A concentrated time-frequency analysis tool for bearing fault diagnosis. IEEE Transactions on Instrumentation and Measurement, 69(2), 371–381. https://doi.org/10.1109/TIM.2019.2901514

    Article  Google Scholar 

  33. Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H. Tang, X. (n.d.). Residual Attention Network for Image Classification.

  34. Yao, R., Xue, J., Yang, P., Wang, Q., Gao, P., Yang, X., & Smith, E. (2021). Residual attention graph convolutional network for web services classification. Neurocomputing, 440(1), 45–57. https://doi.org/10.1038/s41598-021-83350-6

    Article  Google Scholar 

  35. Sravani, C., Bajaj, V., Taran, S., & Sengur, A. (2020). Flexible analytic wavelet transform based features for physical action identification using sEMG signals. Irbm, 41(1), 18–22. https://doi.org/10.1016/j.irbm.2019.07.002

    Article  Google Scholar 

  36. Saqlain, M., Jargalsaikhan, B., & Lee, J. Y. (2019). A voting ensemble classifier for wafer map defect patterns identification in semiconductor manufacturing. IEEE Transactions on Semiconductor Manufacturing, 32(2), 171–182. https://doi.org/10.1109/TSM.2019.2904306

    Article  Google Scholar 

  37. You, Y., Chen, W., Li, M., Zhang, T., Jiang, Y., & Zheng, X. (2020). Automatic focal and non-focal EEG detection using entropy-based features from flexible analytic wavelet transform. Biomedical Signal Processing and Control, 57, 101761. https://doi.org/10.1016/j.bspc.2019.101761

    Article  Google Scholar 

  38. Göksu, H. (2018). BCI oriented EEG analysis using log energy entropy of wavelet packets. Biomedical Signal Processing and Control, 44, 101–109. https://doi.org/10.1016/j.bspc.2018.04.002

    Article  Google Scholar 

  39. Żabiński, G., Gramacki, J., Gramacki, A., Miśta-Jakubowska, E., Birch, T., & Disser, A. (2020). Multi-classifier majority voting analyses in provenance studies on iron artefacts. Journal of Archaeological Science. https://doi.org/10.1016/j.jas.2019.105055

    Article  Google Scholar 

  40. Rahmawati, D., Chasanah, N. R. U., & Sarno, R. (2017). Classify epilepsy and normal Electroencephalogram (EEG) signal using wavelet transform and K-nearest neighbor. In Proceeding-2017 3rd International conference on science in information technology: Theory and application of IT for education, industry and society in big data era, ICSITech 2017, 2018-Janua, (pp. 110–114). https://doi.org/10.1109/ICSITech.2017.8257094

  41. Budak, U., Bajaj, V., Akbulut, Y., Atila, O., & Sengur, A. (2019). An effective hybrid model for EEG-based drowsiness detection. IEEE Sensors Journal, 19(17), 7624–7631. https://doi.org/10.1109/JSEN.2019.2917850

    Article  Google Scholar 

  42. Das, D., Bhattacharjee, T., Datta, S., Choudhury, A. D., Das, P., & Pal, A. (2018). Classification and quantitative estimation of cognitive stress from in-game keystroke analysis using EEG and GSR. 2017 IEEE Life Sciences Conference, LSC 2017, 2018-Janua, (pp. 286–291). https://doi.org/10.1109/LSC.2017.8268199

  43. Cieslak, M. C., Castelfranco, A. M., Roncalli, V., Lenz, P. H., & Hartline, D. K. (2020). t-distributed stochastic neighbor embedding (t-SNE): A tool for eco-physiological transcriptomic analysis. Marine Genomics, 51(November), 100723. https://doi.org/10.1016/j.margen.2019.100723

    Article  Google Scholar 

  44. Raza, H., Chowdhury, A., Bhattacharyya, S., & Samothrakis, S. (2020). Single-Trial EEG Classification with EEGNet and Neural Structured Learning for Improving BCI Performance.

  45. Xu, M., Yao, J., Zhang, Z., Li, R., Yang, B., Li, C., & Zhang, J. (2020). Learning EEG topographical representation for classification via convolutional neural network. Pattern Recognition. https://doi.org/10.1016/j.patcog.2020.107390

    Article  Google Scholar 

  46. Olias, J., Martin-Clemente, R., Sarmiento-Vega, M. A., & Cruces, S. (2019). EEG signal processing in mi-bci applications with improved covariance matrix estimators. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 27(5), 895–904. https://doi.org/10.1109/TNSRE.2019.2905894

    Article  Google Scholar 

  47. Salazar-Ramirez, A., Martin, J. I., Martinez, R., Arruti, A., Muguerza, J., & Sierra, B. (2019). A hierarchical architecture for recognising intentionality in mental tasks on a brain-computer interface. PLoS ONE, 14(6), 1–18. https://doi.org/10.1371/journal.pone.0218181

    Article  Google Scholar 

  48. Bagh, N., & Reddy, M. R. (2019). Improving the performance of motor imagery based brain-computer interface using phase space reconstruction. In Proceedings of the annual international conference of the IEEE engineering in medicine and biology society, EMBS, (Mi), (pp. 3075–3078). https://doi.org/10.1109/EMBC.2019.8857066

Download references

Funding

Authors are thankful to the Thapar Institute of Engineering and Technology, Patiala, Punjab, India for financial support in form of seed money project “Analysis of Electroencephalogram Signals for implementation of P300-based Brain Computer Interface”.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Thapar Institute of Engineering and Technology, Patiala, Punjab, 147002, India

    K. Jindal, R. Upadhyay & H. S. Singh

  2. Neuroimaging and Neurospectroscopy (NINS) Laboratory, National Brain Research Centre, Manesar, Haryana, 122051, India

    K. Jindal

Authors
  1. K. Jindal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Upadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H. S. Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KJ: Conceptualization and design of study, Methodology, Software, Validation, Formal analysis, Investigation, Writing—original draft. RU: Conceptualization, Methodology, Investigation, Writing—review & editing, Supervision, Project administration, Funding acquisition. HSS: Writing—review & editing, Supervision.

Corresponding author

Correspondence to K. Jindal.

Ethics declarations

Conflict of interest

This manuscript by K. Jindal*, R. Upadhyay, H. S. Singh titled “A Novel EEG Channel Selection and Classification Methodology for Multi-Class Motor Imagery-based BCI System Design” is an original unpublished work and the manuscript or any variation of it has not been submitted to any other publication previously. All of the authors have agreed with the submission.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jindal, K., Upadhyay, R. & Singh, H.S. A hybrid ensemble voting-based residual attention network for motor imagery EEG Classification. Analog Integr Circ Sig Process 119, 165–184 (2024). https://doi.org/10.1007/s10470-023-02240-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10470-023-02240-1

Keywords

Search

Navigation