Skip to main content
SpringerLink
Log in

Toward CNN-Based Motor-Imagery EEG Classification with Fuzzy Fusion

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

Abstract

Recently, decoding human electroencephalographic (EEG) data using convolutional neural network (CNN) has driven the state-of-the-art recognition of motor-imagery EEG patterns for brain–computer interfacing (BCI). While a variety of CNN models have been used to classify motor-imagery EEG data, it is unclear if aggregating an ensemble of heterogeneous CNN models could further enhance the classification performance. To integrate the outputs of ensemble classifiers, this work utilizes fuzzy integral with particle swarm optimization (PSO) to estimate optimal confidence levels assigned to classifiers. The proposed framework aggregates CNN classifiers and fuzzy integral with PSO, achieving robust performance in single-trial classification of motor-imagery EEG data across various CNN model training schemes depending on the scenarios of BCI usage. This proof-of-concept study demonstrates the feasibility of applying fuzzy fusion techniques to enhance CNN-based EEG decoding and benefit practical applications of BCI.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Wolpaw, J.R., Birbaumer, N., Heetderks, W.J., McFarland, D.J., Peckham, P.H., Schalk, G., Donchin, E., Quatrano, L.A., Robinson, C.J., Vaughan, T.M., et al.: Brain-computer interface technology: a review of the first international meeting. IEEE Trans. Rehabil. Eng. 8(2), 164–173 (2000)

    Article  Google Scholar 

  2. Pfurtscheller, G., Neuper, C., Muller, G., Obermaier, B., Krausz, G., Schlogl, A., Scherer, R., Graimann, B., Keinrath, C., Skliris, D., et al.: Graz-bci: state of the art and clinical applications. IEEE Trans. Neural Syst. Rehabil. Eng. 11(2), 1–4 (2003)

    Article  Google Scholar 

  3. Fabiani, G.E., McFarland, D.J., Wolpaw, J.R., Pfurtscheller, G.: Conversion of eeg activity into cursor movement by a brain-computer interface (BCI). IEEE Trans. Neural Syst. Rehabil. Eng. 12(3), 331–338 (2004)

    Article  Google Scholar 

  4. Mulder, T.: Motor imagery and action observation: cognitive tools for rehabilitation. J. Neural Transm. 114(10), 1265–1278 (2007)

    Article  Google Scholar 

  5. Johnson, D.H.: Signal-to-noise ratio. Scholarpedia 1(12), 2088 (2006)

    Article  Google Scholar 

  6. Bast, T., Boppel, T., Rupp, A., Harting, I., Hoechstetter, K., Fauser, S., Schulze-Bonhage, A., Scherg, M., et al.: Noninvasive source localization of interictal eeg spikes: effects of signal-to-noise ratio and averaging. J. Clin. Neurophysiol. 23(6), 487–497 (2006)

    Article  Google Scholar 

  7. Bostanov, V.: Bci competition 2003-data sets ib and iib: feature extraction from event-related brain potentials with the continuous wavelet transform and the t-value scalogram. IEEE Trans. Biomed. Eng. 51(6), 1057–1061 (2004)

    Article  Google Scholar 

  8. Hsu, W.-Y., Sun, Y.-N.: Eeg-based motor imagery analysis using weighted wavelet transform features. J. Neurosci. Methods 176(2), 310–318 (2009)

    Article  Google Scholar 

  9. Obermaier, B., Neuper, C., Guger, C., Pfurtscheller, G.: Information transfer rate in a five-classes brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 9(3), 283–288 (2001)

    Article  Google Scholar 

  10. Burke, D.P., Kelly, S.P., De Chazal, P., Reilly, R.B., Finucane, C.: A parametric feature extraction and classification strategy for brain-computer interfacing. IEEE Trans. Neural Syst. Rehabil. Eng. 13(1), 12–17 (2005)

    Article  Google Scholar 

  11. O’Shea, K., Nash, R.: An introduction to convolutional neural networks (2015). arXiv:1511.08458.

  12. Lawhern, V.J., Solon, A.J., Waytowich, N.R., Gordon, S.M., Hung, C.P., Lance, B.J.: Eegnet: a compact convolutional neural network for eeg-based brain-computer interfaces. J. Neural Eng. 15(5), 056013 (2018)

    Article  Google Scholar 

  13. Schirrmeister, R.T., Springenberg, J.T., Fiederer, L.D.J., Glasstetter, M., Eggensperger, K., Tangermann, M., Hutter, F., Burgard, W., Ball, T.: Deep learning with convolutional neural networks for eeg decoding and visualization. Hum. Brain Map. 38(11), 5391–5420 (2017)

    Article  Google Scholar 

  14. Wei, C.-S., Koike-Akino, T., Wang, Y.: Spatial component-wise convolutional network (sccnet) for motor-imagery eeg classification. In: 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE, pp. 328–331 (2019).

  15. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)

    Google Scholar 

  16. Anderson, M.F., Anderson, D.T., Wescott, D.J.: Estimation of adult skeletal age-at-death using the sugeno fuzzy integral. Am. J. Phys. Anthropol. 142(1), 30–41 (2010)

    Google Scholar 

  17. Agahi, H., Mesiar, R., Ouyang, Y.: On some advanced type inequalities for sugeno integral and t-(s-) evaluators. Inf. Sci. 190, 64–75 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  18. Singh, M., Madasu, V.K., Srivastava, S., Hanmandlu, M.: Choquet fuzzy integral based verification of handwritten signatures. J. Intell. Fuzzy Syst. 24(1), 145–161 (2013)

    Article  MATH  Google Scholar 

  19. Salimi-Khorshidi, G., Nasrabadi, A.M., Golpayegani, M.H.: Fusion of classic p300 detection methods’ inferences in a framework of fuzzy labels. Artif. Intell. Med. 44(3), 247–259 (2008)

    Article  Google Scholar 

  20. Luqiang, X., Guangcan, X.: Study on power spectrum signal fuzzy fusion for motor imagery. Comput. Eng. 41, 306–309 (2015)

    Google Scholar 

  21. Yoo, B.-S., Kim, J.-H.: Fuzzy integral-based gaze control of a robotic head for human robot interaction. IEEE Trans. Cybern. 45(9), 1769–1783 (2014)

    Google Scholar 

  22. Ko, L.-W., Lu, Y.-C., Bustince, H., Chang, Y.-C., Chang, Y., Ferandez, J., Wang, Y.-K., Sanz, J.A., Dimuro, G.P., Lin, C.-T.: Multimodal fuzzy fusion for enhancing the motor-imagery-based brain computer interface. IEEE Comput. Intell. Mag. 14(1), 96–106 (2019)

    Article  Google Scholar 

  23. Banerjee, A., Singh, P.K., Sarkar, R.: Fuzzy integral based cnn classifier fusion for 3d skeleton action recognition. IEEE Trans. Circuits Syst. Video Technol (2020)

  24. Liu, C., Du, W.-B., Wang, W.-X.: Particle swarm optimization with scale-free interactions. PLoS ONE 9(5), e97822 (2014)

    Article  Google Scholar 

  25. Brunner, C., Leeb, R., Müller-Putz, G., Schlögl, A., Pfurtscheller, G.: “Bci competition 2008-graz data set a,’’ Institute for Knowledge Discovery (Laboratory of Brain-Computer Interfaces). Graz University of Technology 16, 1–6 (2008)

    Google Scholar 

  26. Grabisch, M., Nguyen, H.T., Walker, E.A.: Fundamentals of Uncertainty Calculi with Applications to Fuzzy Inference, vol. 30. Springer, Berlin (2013)

  27. Waytowich, N., Lawhern, V.J., Garcia, J.O., Cummings, J., Faller, J., Sajda, P., Vettel, J.M.: Compact convolutional neural networks for classification of asynchronous steady-state visual evoked potentials. J. Neural Eng. 15(6), 066031 (2018)

    Article  Google Scholar 

  28. Wu, S.-L., Liu, Y.-T., Hsieh, T.-Y., Lin, Y.-Y., Chen, C.-Y., Chuang, C.-H., Lin, C.-T.: Fuzzy integral with particle swarm optimization for a motor-imagery-based brain-computer interface. IEEE Trans. Fuzzy Syst. 25(1), 21–28 (2016)

    Article  Google Scholar 

  29. Hsieh, T.-Y., Lin, Y.-Y., Liu, Y.-T., Fang, C.-N., Lin, C.-T.: Developing a novel multi-fusion brain-computer interface (bci) system with particle swarm optimization for motor imagery task. In: 2015 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE). IEEE, 2015, pp. 1–4

  30. Ayub, M.: Choquet and sugeno integrals (2009)

  31. Altalhi, A.H., Forcén, J.I., Pagola, M., Barrenechea, E., Bustince, H., Takáč, Z.: Moderate deviation and restricted equivalence functions for measuring similarity between data. Inf. Sci. 501, 19–29 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  32. Bustince, H., Beliakov, G., Dimuro, G.P., Bedregal, B., Mesiar, R.: On the definition of penalty functions in data aggregation. Fuzzy Sets Syst. 323, 1–18 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  33. Jurio, A., Pagola, M., Mesiar, R., Beliakov, G., Bustince, H.: Image magnification using interval information. IEEE Trans. Image Process. 20(11), 3112–3123 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  34. Dimuro, G.P., Lucca, G., Bedregal, B., Mesiar, R., Sanz, J.A., Lin, C.-T., Bustince, H.: Generalized cf1f2-integrals: from choquet-like aggregation to ordered directionally monotone functions. Fuzzy Sets Syst. 378, 44–67 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  35. Beliakov, G., Sola, H.B., Sánchez, T.C.: A Practical Guide to Averaging Functions. Springer, New York (2016)

  36. Miranda, L.J.V.: PySwarms, a research-toolkit for particle swarm optimization in python. J. Open Source Softw. https://doi.org/10.21105/joss.00433

  37. Bonyadi, M.R., Michalewicz, Z.: Particle swarm optimization for single objective continuous space problems: a review. Evolut. Comput. 25(1), 1–54 (2017)

    Article  Google Scholar 

  38. Zambrano-Bigiarini, M., Clerc, M., Rojas, R.:Standard particle swarm optimisation, at cec-2013: a baseline for future pso improvements. In: 2013 IEEE Congress on Evolutionary Computation. IEEE 2013, 2337–2344 (2011)

  39. Woolson, R.: Wilcoxon signed-rank test. In: Wiley Encyclopedia of Clinical Trials, pp. 1–3 (2007).

  40. Chu, M.-T., Shyu, J.Z., Tzeng, G.-H., Khosla, R.: Using nonadditive fuzzy integral to assess performances of organizational transformation via communities of practice. IEEE Trans. Eng. Manag. 54(2), 327–339 (2007)

    Article  Google Scholar 

  41. Pinar, A.J., Havens, T.C., Islam, M.A., Anderson, D.T.: Visualization and learning of the choquet integral with limited training data. In: 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE). IEEE, pp. 1–6 (2017)

Download references

Acknowledgements

This work was supported in part by the Ministry of Science and Technology under Contracts 109-2222-E-009-006-MY3, 110-2221-E-A49-130-MY2, and 110-2314-B-037-061; and in part by the Higher Education Sprout Project of the National Chiao Tung University and Ministry of Education of Taiwan. The authors would also like to thank Xin-Yao Huang for his support in implementing CNN models.

Author information

Authors and Affiliations

  1. Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwan

    Jian-Xue Huang, Chia-Ying Hsieh, Ya-Lin Huang & Chun-Shu Wei

  2. Institute of Education, National Yang Ming Chiao Tung University, Hsinchu, Taiwan

    Chun-Shu Wei

  3. Institute of Biomedical Engineering, National Yang Ming Chiao Tung University, Hsinchu, Taiwan

    Chun-Shu Wei

  4. Institute of Electrical and Control Engineering, National Yang Ming Chiao Tung University, Hsinchu, Taiwan

    Chun-Shu Wei

Authors
  1. Jian-Xue Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chia-Ying Hsieh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ya-Lin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chun-Shu Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian-Xue Huang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, JX., Hsieh, CY., Huang, YL. et al. Toward CNN-Based Motor-Imagery EEG Classification with Fuzzy Fusion. Int. J. Fuzzy Syst. 24, 3812–3823 (2022). https://doi.org/10.1007/s40815-022-01307-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-022-01307-x

Keywords

Search

Navigation