Skip to main content
SpringerLink
Log in

EEG-based imagined words classification using Hilbert transform and deep networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The completely paralyzed and quadriplegic patients cannot communicate with others. However, the imagined thoughts of these patients can be used to drive assistive devices by brain-computer interfacing (BCI), the success of which relies on better classification accuracies. In this paper, we have performed an experiment for the classification of imagined words, which can provide an alternative neural path of speech communication for deprived people. A 32-channel industry-standard physiological signal system is used to measure imagined electroencephalogram (EEG) signals of five words (sos, stop, medicine, washroom, comehere) from 13 subjects. We have used the Hilbert transform to calculate time and joint time–frequency features from the imagined EEG signals. The above features are extracted individually in electrodes corresponding to nine brain regions. Each region of the brain is further analyzed in seven EEG frequency bands. The imagined speech features from each of the 63 combinations of brain region and frequency band are classified by the proposed deep architectures like long short term memory (LSTM), gated recurrent unit, and convolutional neural network (CNN). Some combinations are also classified by six traditional machine learning classifiers for performance comparison. In a five-class classification framework, we achieved the average and maximum accuracy of 71.75% and 94.29%. CNN gave high accuracy, but LSTM gave less network prediction time. Our results show that the alpha band can classify imagined speech better than other frequency bands. We have implemented subject-independent BCI, and the results are better than the state-of-the-art methods present in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Agarwal P, Kumar S (2021) Transforming Imagined Thoughts into Speech Using a Covariance-Based Subset Selection Method. Indian J Pure Appl Phys; 59:180–3. http://nopr.niscair.res.in/handle/123456789/56517. Accessed 5 Jan 2022

  2. Agarwal P, Kumar S (2022) Electroencephalography based imagined alphabets classification using spatial and time-domain features. Int J Imaging Syst Technol 32:111–122. https://doi.org/10.1002/ima.22655

    Article  Google Scholar 

  3. Asghari Bejestani MR, Mohammad Khani GhR, Nafisi VR, Darakeh F (2022) EEG-Based Multiword Imagined Speech Classification for Persian Words. BioMed Res Int 2022:8333084. https://doi.org/10.1155/2022/8333084

    Article  Google Scholar 

  4. Bakhshali MA, Khademi M, Ebrahimi-Moghadam A, Moghimi S (2020) EEG signal classification of imagined speech based on Riemannian distance of correntropy spectral density. Biomed Signal Process Control 59:101899. https://doi.org/10.1016/j.bspc.2020.101899

    Article  Google Scholar 

  5. Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, et al. (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. ArXiv:14061078 v3[CsCL]

  6. D’Zmura M, Deng S, Lappas T, Thorpe S, Srinivasan R. (2009) Toward EEG Sensing of Imagined Speech. In: Jacko JA, editor. Human-Computer Interaction. New Trends, vol. 5610, Berlin, Heidelberg: Springer; p. 40–8

  7. DaSalla CS, Kambara H, Sato M, Koike Y (2009) Single-trial classification of vowel speech imagery using common spatial patterns. Neural Netw 22:1334–1339. https://doi.org/10.1016/j.neunet.2009.05.008

    Article  Google Scholar 

  8. Deng S, Srinivasan R, Lappas T, D’Zmura M (2010) EEG classification of imagined syllable rhythm using Hilbert spectrum methods. J Neural Eng 7:046006. https://doi.org/10.1088/1741-2560/7/4/046006

    Article  Google Scholar 

  9. Dewan EM (1967) Occipital Alpha Rhythm Eye Position and Lens Accommodation. Nature 214:975–7. https://doi.org/10.1038/214975a0

    Article  Google Scholar 

  10. Esfahani ET, Sundararajan V (2012) Classification of primitive shapes using brain-computer interfaces. Comput Aided Des 44:1011–1019. https://doi.org/10.1016/j.cad.2011.04.008

    Article  Google Scholar 

  11. Fujimaki N, Takeuchi F, Kobayashi T, Kuriki S, Hasuo S (1994) Event-related potentials in silent speech. Brain Topogr 6:259–267. https://doi.org/10.1007/BF01211171

    Article  Google Scholar 

  12. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS) 2010, vol. 9, Chia Laguna Resort, Sardinia, Italy: JMLR: W&CP 9; p. 249–56

  13. Hahn SL (1996) Hilbert transforms in signal processing. eBook. Boston. Artech House, USA

    Google Scholar 

  14. Hochreiter S, Schmidhuber J (1997) Long Short-Term Memory. Neural Comput 9:1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  15. Huang NE, Attoh-Okine (Eds.) NO (2005) The Hilbert-Huang Transform in Engineering. 1st ed. Boca Raton, Florida, USA: CRC Press; https://doi.org/10.1201/9781420027532

  16. Huang NE, Samuel SPS (2014) Hilbert-Huang transform and its applications. vol. 16. 2nd ed. World Scientific. Singapore

  17. Huang NE, Shen Z, Long SR, Wu MC, Shih HH, Zheng Q et al (1998) The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc Math Phys Eng Sci 454:903–995. https://doi.org/10.1098/rspa.1998.0193

    Article  MathSciNet  Google Scholar 

  18. Kaushik P, Gupta A, Roy PP, Dogra DP (2019) EEG-Based Age and Gender Prediction Using Deep BLSTM-LSTM Network Model. IEEE Sens J 19:2634–2641. https://doi.org/10.1109/JSEN.2018.2885582

    Article  Google Scholar 

  19. Khademi S, Neghabi M, Farahi M, Shirzadi M, Marateb HR. 2 - A comprehensive review of the movement imaginary brain-computer interface methods: Challenges and future directions. In: Bajaj V, Sinha GR, editors. Artificial Intelligence-Based Brain-Computer Interface, Academic Press; 2022, p. 23–74. https://doi.org/10.1016/B978-0-323-91197-9.00004-7

  20. Kingma DP, Ba J (2014) Adam: A Method for Stochastic Optimization. ArXiv:14126980 [CsLG]

  21. Klem GH, Lüders HO, Jasper HH, Elger C (1999) The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. Electroencephalogr Clin Neurophysiol Suppl 52:3–6. https://doi.org/10.1080/00029238.1961.11080571

    Article  Google Scholar 

  22. Kristensen AB, Subhi Y, Puthusserypady S (2020) Vocal Imagery vs Intention: Viability of Vocal-Based EEG-BCI Paradigms. IEEE Trans Neural Syst Rehabilitation Eng 28:1750–1759. https://doi.org/10.1109/TNSRE.2020.3004924

    Article  Google Scholar 

  23. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet Classification with Deep Convolutional Neural Networks. Commun ACM 60:84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  24. Kumar P, Saini R, Roy PP, Sahu PK, Dogra DP (2018) Envisioned speech recognition using EEG sensors. Pers Ubiquitous Comput 22:185–199. https://doi.org/10.1007/s00779-017-1083-4

    Article  Google Scholar 

  25. La Vaque TJ (1999) The History of EEG Hans Berger: Psychophysiologist. A Historical Vignette. J Neurother 3:1–9. https://doi.org/10.1300/J184v03n02_01

    Article  Google Scholar 

  26. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W et al (1989) Backpropagation Applied to Handwritten Zip Code Recognition. Neural Comput 1:541–551. https://doi.org/10.1162/neco.1989.1.4.541

    Article  Google Scholar 

  27. Martin S, Brunner P, Iturrate I, Millán J del R, Schalk G, Knight RT, et al. (2016) Word pair classification during imagined speech using direct brain recordings. Sci Rep; 6. https://doi.org/10.1038/srep25803

  28. Nguyen CH, Karavas GK, Artemiadis P (2017) Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features. J Neural Eng 15:016002. https://doi.org/10.1088/1741-2552/aa8235

    Article  Google Scholar 

  29. Nie K, Barco A, Zeng F-G (2006) Spectral and temporal cues in cochlear implant speech perception. Ear Hear 27:208–217. https://doi.org/10.1097/01.aud.0000202312.31837.25

    Article  Google Scholar 

  30. Panachakel JT, Ramakrishnan AG, Ananthapadmanabha TV (2019) Decoding Imagined Speech using Wavelet Features and Deep Neural Networks. 2019 IEEE 16th India Council International Conference (INDICON), Rajkot, India: IEEE; p. 1–4. https://doi.org/10.1109/INDICON47234.2019.9028925

  31. Porbadnigk A, Wester M, Calliess J, Schultz T. EEG-based speech recognition- impact of temporal effects. Proceedings of the International Conference on Bio-inspired Systems and Signal Processing - BIOSIGNALS, (BIOSTEC 2009), Porto, Portugal: 2009, p. 376–81. https://doi.org/10.5220/0001554303760381

  32. Qureshi MNI, Min B, Park H, Cho D, Choi W, Lee B (2018) Multiclass Classification of Word Imagination Speech With Hybrid Connectivity Features. IEEE Trans Biomed Eng 65:2168–2177. https://doi.org/10.1109/TBME.2017.2786251

    Article  Google Scholar 

  33. Ramadan RA, Vasilakos AV (2017) Brain computer interface: control signals review. Neurocomputing 223:26–44. https://doi.org/10.1016/j.neucom.2016.10.024

    Article  Google Scholar 

  34. Recio-Spinoso A, Fan Y-H, Ruggero MA (2011) Basilar-Membrane Responses to Broadband Noise Modeled Using Linear Filters With Rational Transfer Functions. IEEE Trans Biomed Eng 58:1456–1465. https://doi.org/10.1109/TBME.2010.2052254

    Article  Google Scholar 

  35. Rezazadeh Sereshkeh A, Trott R, Bricout A, Chau T (2017) EEG Classification of Covert Speech Using Regularized Neural Networks. IEEE/ACM Trans Audio, Speech, Language Process 25:2292–2300. https://doi.org/10.1109/TASLP.2017.2758164

    Article  Google Scholar 

  36. Roy AM (2022) An efficient multi-scale CNN model with intrinsic feature integration for motor imagery EEG subject classification in brain-machine interfaces. Biomed Signal Process Control 74:103496. https://doi.org/10.1016/j.bspc.2022.103496

    Article  Google Scholar 

  37. Roy Y, Banville H, Albuquerque I, Gramfort A, Falk TH, Faubert J (2019) Deep learning-based electroencephalography analysis: a systematic review. J Neural Eng 16:051001. https://doi.org/10.1088/1741-2552/ab260c. Accessed 5 Jan 2022 

  38. Saxe AM, McClelland JL, Ganguli S (2013) Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. ArXiv:13126120 [CsNE]

  39. Sereshkeh AR, Trott R, Bricout A, Chau T (2017) Online EEG Classification of Covert Speech for Brain-Computer Interfacing. Int J Neural Syst 27:1750033. https://doi.org/10.1142/S0129065717500332

    Article  Google Scholar 

  40. Sreeja SR, Himanshu, Samanta D (2020) Distance-based weighted sparse representation to classify motor imagery EEG signals for BCI applications. Multimed Tools Appl 79:13775–93. https://doi.org/10.1007/s11042-019-08602-0

    Article  Google Scholar 

  41. Torres-García AA, Reyes-García CA, Villaseñor-Pineda L, García-Aguilar G (2016) Implementing a fuzzy inference system in a multi-objective EEG channel selection model for imagined speech classification. Expert Syst Appl 59:1–12. https://doi.org/10.1016/j.eswa.2016.04.011

    Article  Google Scholar 

  42. Wang L, Liu X, Liang Z, Yang Z, Hu X (2019) Analysis and classification of hybrid BCI based on motor imagery and speech imagery. Measurement 147:106842. https://doi.org/10.1016/j.measurement.2019.07.070

    Article  Google Scholar 

  43. Xu F, Xu X, Sun Y, Li J, Dong G, Wang Y et al (2022) A framework for motor imagery with LSTM neural network. Comput Methods Programs Biomed 218:106692. https://doi.org/10.1016/j.cmpb.2022.106692

    Article  Google Scholar 

  44. Zhang Y, Zhang S, Ji X (2018) EEG-based classification of emotions using empirical mode decomposition and autoregressive model. Multimed Tools Appl 77:26697–26710. https://doi.org/10.1007/s11042-018-5885-9

    Article  Google Scholar 

  45. Zhao S, Rudzicz F (2015) Classifying phonological categories in imagined and articulated speech. 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia: IEEE; p. 992–6. https://doi.org/10.1109/ICASSP.2015.7178118

Download references

Author information

Authors and Affiliations

  1. Electronics & Communication Engineering, National Institute of Technology Delhi, Plot No. FA7, Zone P1, GT Karnal Road, North West Delhi, Delhi, 110036, India

    Prabhakar Agarwal & Sandeep Kumar

Authors
  1. Prabhakar Agarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sandeep Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

1) Prabhakar Agarwal: Conceptualization, Formal Analysis, Investigation, Methodology, Software, Visualization, Writing- Original Draft

2) Sandeep Kumar: Data Curation, Investigation, Project Administration, Resources, Supervision, Writing- Reviewing and Editing

Corresponding author

Correspondence to Sandeep Kumar.

Ethics declarations

Research involving Human Participants and/or Animals

The subjects voluntarily participated and submitted written consent for the experiment. Their identity was maintained confidential throughout the work.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Conflicts of interest

There is no conflict of interest between the authors regarding the manuscript preparation and submission.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Agarwal, P., Kumar, S. EEG-based imagined words classification using Hilbert transform and deep networks. Multimed Tools Appl 83, 2725–2748 (2024). https://doi.org/10.1007/s11042-023-15664-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15664-8

Keywords

Search

Navigation