Skip to main content

Advertisement

SpringerLink
Log in

EEG classification of driver mental states by deep learning

  • Research Article
  • Published:
Cognitive Neurodynamics Aims and scope Submit manuscript

Abstract

Driver fatigue is attracting more and more attention, as it is the main cause of traffic accidents, which bring great harm to society and families. This paper proposes to use deep convolutional neural networks, and deep residual learning, to predict the mental states of drivers from electroencephalography (EEG) signals. Accordingly we have developed two mental state classification models called EEG-Conv and EEG-Conv-R. Tested on intra- and inter-subject, our results show that both models outperform the traditional LSTM- and SVM-based classifiers. Our major findings include (1) Both EEG-Conv and EEG-Conv-R yield very good classification performance for mental state prediction; (2) EEG-Conv-R is more suitable for inter-subject mental state prediction; (3) EEG-Conv-R converges more quickly than EEG-Conv. In summary, our proposed classifiers have better predictive power and are promising for application in practical brain-computer interaction .

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Abdel-Hamid O, Mohamed AR, Jiang H, Deng L, Penn G, Yu D (2014) Convolutional neural networks for speech recognition. IEEE/ACM Trans Audio Speech Lang Pprocess 22(10):1533–1545

    Article  Google Scholar 

  • Ahn S, Nguyen T, Jang H, Kim JG, Jun SC (2016) Exploring neuro-physiological correlates of drivers’ mental fatigue caused by sleep deprivation using simultaneous EEG, ECG, and FNIRS data. Front Hum Neurosci 10:219. https://doi.org/10.3389/fnhum.2016.00219

    Article  PubMed  PubMed Central  Google Scholar 

  • Bornas X, Fiolveny A, Balle M, Morillasromero A, Tortellafeliu M (2015) Long range temporal correlations in eeg oscillations of subclinically depressed individuals: their association with brooding and suppression. Cognit Neurodyn 9(1):53–62

    Article  Google Scholar 

  • Brookhuis KA, De WD (1993) The use of psychophysiology to assess driver status. Ergonomics 36(9):1099

    Article  CAS  Google Scholar 

  • Cecotti H, Graser A (2011) Convolutional neural networks for p300 detection with application to brain-computer interfaces. IEEE Trans Pattern Anal Mach Intell 33(3):433–445

    Article  Google Scholar 

  • Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–27

    Article  Google Scholar 

  • Chen LL, Zhao Y, Ye PF, Zhang J, Zou JZ (2017) Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers. Expert Syst Appl 85(C):279–291

    Article  Google Scholar 

  • Correa AG, Orosco L, Laciar E (2014) Automatic detection of drowsiness in eeg records based on multimodal analysis. Med Eng Phys 36(2):244

    Article  Google Scholar 

  • Domhan T, Springenberg JT, Hutter F (2015) Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In: IJCAI, pp 3460–3468

  • Fu R, Wang H (2014) Detection of driving fatigue by using noncontact emg and ecg signals measurement system. Int J Neural Syst 24(03):1450006

    Article  Google Scholar 

  • Fu RR, Wang H, Zhao WB (2016) Dynamic driver fatigue detection using hidden markov model in real driving condition. Expert Syst Appl 63(C):397–411

    Article  Google Scholar 

  • Gravier A, Quek C, Duch W, Wahab A, Gravier-Rymaszewska J (2016) Neural network modelling of the influence of channelopathies on reflex visual attention. Cognit Neurodyn 10(1):49–72

    Article  Google Scholar 

  • Hajinoroozi M, Mao Z, Huang Y (2016) Prediction of driver’s drowsy and alert states from eeg signals with deep learning. In: IEEE international workshop on computational advances in multi-sensor adaptive processing, pp 493–496

  • He KM, Zhang XY, Ren SQ, Sun J (2016) Deep residual learning for image recognition. In: Computer vision and pattern recognition, pp 770–778

  • Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  CAS  Google Scholar 

  • Hu JF (2017) Automated detection of driver fatigue based on adaboost classifier with eeg signals. Front Comput Neurosci 11:72

    Article  Google Scholar 

  • Hu SH, Zheng GT (2009) Driver drowsiness detection with eyelid related parameters by support vector machine. Expert Syst Appl 36(4):7651–7658

    Article  Google Scholar 

  • Idogawa K (2006) On the brain wave activity of professional drivers during monotonous work. Behaviormetrika 18(30):23–34

    Article  Google Scholar 

  • Jap BT, Lal S, Fischer P, Bekiaris E (2009) Using eeg spectral components to assess algorithms for detecting fatigue. Expert Syst Appl 36(2):2352–2359

    Article  Google Scholar 

  • Jeong IC, Lee DH, Park SW, Ko JI, Yoon HR (2007) Automobile driver’s stress index provision system that utilizes electrocardiogram. In: Intelligent vehicles symposium, 2007 IEEE. IEEE, pp 652–656

  • Jung TP, Makeig S, Humphries C, Lee TW, Mckeown MJ, Iragui V, Sejnowski TJ (2000) Removing electroencephalographic artifacts by blind source separation. Psychophysiology 37(2):163–178

    Article  CAS  Google Scholar 

  • Kar S, Bhagat M, Routray A (2010) Eeg signal analysis for the assessment and quantification of drivers fatigue. Transp Res Part F Traffic Psychol Behav 13(5):297–306

    Article  Google Scholar 

  • Khushaba RN, Kodagoda S, Lal S, Dissanayake G (2010) Driver drowsiness classification using fuzzy wavelet-packet-based feature-extraction algorithm. IEEE Trans Biomed Eng 58(1):121–131

    Article  Google Scholar 

  • Kong WZ, Zhou ZP, Jiang B, Babiloni F, Borghini G (2017) Assessment of driving fatigue based on intra/inter-region phase synchronization. Neurocomputing 219(5):474–482

    Article  Google Scholar 

  • Lal SK, Craig A (2001) A critical review of the psychophysiology of driver fatigue. Biol Psychol 55(3):173–194

    Article  CAS  Google Scholar 

  • Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ (2016) Eegnet: a compact convolutional network for EEG-based brain-computer interfaces. arXiv preprint arXiv:1611.08024

  • LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  CAS  Google Scholar 

  • Liang SF, Wang HC, Chang WL (2010) Combination of eeg complexity and spectral analysis for epilepsy diagnosis and seizure detection. Eurasip J Adv Signal Process 2010(1):1–15

    Google Scholar 

  • Lin CT, Huang KC, Chao CF, Chen JA, Chiu TW, Ko LW, Jung TP (2010) Tonic and phasic eeg and behavioral changes induced by arousing feedback. NeuroImage 52(2):633–642

    Article  Google Scholar 

  • Lin CT, Wang YK, Chen SA (2014) An eeg-based brain-computer interface for dual task driving detection. Neurocomputing 129(4):85–93

    Google Scholar 

  • Manor R, Geva AB (2015) Convolutional neural network for multi-category rapid serial visual presentation BCI. Front Comput Neurosci 9:146

    Article  Google Scholar 

  • Mu ZD, Hu JF, Min JL (2017) Driver fatigue detection system using electroencephalography signals based on combined entropy features. Appl Sci 7(2):150

    Article  Google Scholar 

  • Page A, Shea C, Mohsenin T (2016) Wearable seizure detection using convolutional neural networks with transfer learning. In: IEEE international symposium on circuits and systems (ISCAS). IEEE, pp 1086–1089

  • Pal NR, Chuang CY, Ko LW, Chao CF, Jung TP, Liang SF, Lin CT (2008) Eeg-based subject-and session-independent drowsiness detection: an unsupervised approach. EURASIP J Adv Signal Process 2008(1):519480

    Article  Google Scholar 

  • Puanhvuan D, Khemmachotikun S, Wechakarn P, Wijarn B, Wongsawat Y (2017) Navigation-synchronized multimodal control wheelchair from brain to alternative assistive technologies for persons with severe disabilities. Cognit Neurodyn 11(2):117–134

    Article  Google Scholar 

  • Qin FW, Gao NN, Peng Y, Wu ZZ, Shen SY, Grudtsin A (2018) Fine-grained leukocyte classification with deep residual learning for microscopic images. Comput Methods Programs Biomed 162(8):243–252

    Article  Google Scholar 

  • Raghu S, Sriraam N, Kumar GP (2017) Classification of epileptic seizures using wavelet packet log energy and norm entropies with recurrent elman neural network classifier. Cognit Neurodyn 11(1):51–66

    Article  CAS  Google Scholar 

  • Sahayadhas A, Sundaraj K, Murugappan M (2012) Detecting driver drowsiness based on sensors: a review. Sensors 12(12):16937

    Article  Google Scholar 

  • Sakhavi S, Guan CT, Yan SC (2015) Parallel convolutional-linear neural network for motor imagery classification. In: Signal processing conference (EUSIPCO). IEEE, pp 2736–2740

  • Schirrmeister RT, Springenberg JT, Fiederer LDJ, Glasstetter M, Eggensperger K, Tangermann M, Hutter F, Burgard W, Ball T (2017) Deep learning with convolutional neural networks for brain mapping and decoding of movement-related information from the human eeg. arXiv preprint arXiv:1703.05051

  • Schoenberg PLA, Speckens AEM (2015) Multi-dimensional modulations of alpha and gamma cortical dynamics following mindfulness-based cognitive therapy in major depressive disorder. Cognit Neurodyn 9(1):13–29

    Article  Google Scholar 

  • Stein D, Orbach ISM, Har ED, Yaruslasky A, Roth D, Meged S, Apter A (2013) Eeg alpha band synchrony predicts cognitive and motor performance in patients with ischemic stroke. Behav Neurol 26(3):187

    Article  Google Scholar 

  • Tang ZC, Li C, Sun SQ (2017) Single-trial eeg classification of motor imagery using deep convolutional neural networks. Opt Int J Light Electron Opt 130:11–18

    Article  Google Scholar 

  • Thodoroff P, Pineau J, Lim A (2016) Learning robust features using deep learning for automatic seizure detection. In: Machine learning for healthcare conference, pp 178–190

  • Tsuchida A, Bhuiyan M, Oguri K (2009) Estimation of drowsiness level based on eyelid closure and heart rate variability. In: EMBC 2009 international conference of the IEEE Engineering in Medicine and Biology Society, 2009, pp 2543–2546

  • Wali MK, Murugappan M, Ahmmad B (2013) Wavelet packet transform based driver distraction level classification using eeg. Math Probl Eng 2013(3):841–860

    Google Scholar 

  • Zeng H, Dai GJ, Kong WZ, Chen FY, Wang LY (2017) A novel nonlinear dynamic method for stroke rehabilitation effect evaluation using eeg. IEEE Trans Neural Syst Rehabil Eng 25(12):2488–2497

    Article  Google Scholar 

  • Zhang JH, Li SN, Wang RB (2017) Pattern recognition of momentary mental workload based on multi-channel electrophysiological data and ensemble convolutional neural networks. Front Neurosci 11:310

    Article  Google Scholar 

  • Zhang JH, Cui XQ, Li JR, Wang RB (2017) Imbalanced classification of mental workload using a cost-sensitive majority weighted minority oversampling strategy. Cognit Technol Work 19(4):633–653

    Article  Google Scholar 

  • Zhao CL, Zheng CX, Zhao M, Liu JP (2010) Physiological assessment of driving mental fatigue using wavelet packet energy and random forests. Am J Biomed Sci 2(3):262–274

    Article  Google Scholar 

Download references

Acknowledgements

The authors would also like to thank the anonymous referees for their valuable comments and helpful suggestions. The work is supported by the National Natural Science Foundation of China under Grant Nos. {61671193, 61633010, 61473110, 61502129}, Key Research and Development Plan of Zhejiang Province under Grant No. 2018C04012, Zhejiang Provincial Natural Science Foundation of China under Grant No. LQ16F020004. Science and technology platform construction project of Fujian science and Technology Department No. 2015Y2001.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China

    Hong Zeng, Chen Yang, Guojun Dai, Feiwei Qin, Jianhai Zhang & Wanzeng Kong

Authors
  1. Hong Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guojun Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feiwei Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianhai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wanzeng Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wanzeng Kong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeng, H., Yang, C., Dai, G. et al. EEG classification of driver mental states by deep learning. Cogn Neurodyn 12, 597–606 (2018). https://doi.org/10.1007/s11571-018-9496-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11571-018-9496-y

Keywords

Search

Navigation