Skip to main content
SpringerLink
Log in

Deep learning for sleep stages classification: modified rectified linear unit activation function and modified orthogonal weight initialisation

Multimedia Tools and Applications Aims and scope Submit manuscript

Cite this article

Abstract

Each stage of sleep can affect human health, and not getting enough sleep at any stage may lead to sleep disorder like parasomnia, apnea, insomnia, etc. Sleep-related diseases could be diagnosed using Convolutional Neural Network Classifier. However, this classifier has not been successfully implemented into sleep stage classification systems due to high complexity and low accuracy of classification. The aim of this research is to increase the accuracy and reduce the learning time of Convolutional Neural Network Classifier. The proposed system used a modified Orthogonal Convolutional Neural Network and a modified Adam optimisation technique to improve the sleep stage classification accuracy and reduce the gradient saturation problem that occurs due to sigmoid activation function. The proposed system uses Leaky Rectified Linear Unit (ReLU) instead of sigmoid activation function as an activation function. The proposed system called Enhanced Sleep Stage Classification system (ESSC) used six different databases for training and testing the proposed model on the different sleep stages. These databases are University College Dublin database (UCD), Beth Israel Deaconess Medical Center MIT database (MIT-BIH), Sleep European Data Format (EDF), Sleep EDF Extended, Montreal Archive of Sleep Studies (MASS), and Sleep Heart Health Study (SHHS). Our results show that the gradient saturation problem does not exist anymore. The modified Adam optimiser helps to reduce the noise which in turn result in faster convergence time. The convergence speed of ESSC is increased along with better classification accuracy compared to the state of art solution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Abbreviations

CNN:

Convolutional Neural Network

RNN:

Recurrent Neural Network

LSTM:

Long Short Term Memory

FFT:

Fast Fourier Transform

STFT:

Short-Time Fourier Transform

HHT:

Hilbert Huang Transform

EEG:

Electroencephalogram

ECG:

Electrocardiogram

W:

Wake state

S1:

Sleep Stage 1

S2:

Sleep Stage 2

SWS:

Sleep Wake State

REM:

Rapid Eye Movement

ESSC:

Enhanced Sleep Stage Classification

ReLU:

Rectified Linear Unit

References

  1. Erdenebayar U, Kim YJ, Park JU, Joo EY, Lee KJ (2019 Oct) Deep learning approaches for automatic detection of sleep apnea events from an electrocardiogram. Comput Methods Prog Biomed 180:105001. https://doi.org/10.1016/j.cmpb.2019.105001

    Article  Google Scholar 

  2. Fernandez-Blanco E, Rivero D, Pazos A (2020) Convolutional neural networks for sleep stage scoring on a two-channel EEG signal. Soft Comput 24:4067–4079. https://doi.org/10.1007/s00500-019-04174-1

    Article  Google Scholar 

  3. Hu J, Shen L, Albanie S, Sun G, Wu E (2019) Squeeze-and-excitation networks. IEEE Trans Patt Anal Mach Intell https://arxiv.org/abs/1709.01507

  4. Kang CH, Erdenebayar U, Park JU, Lee KJ (2019 Dec) Multi-class classification of sleep apnea/hypopnea events based on long short-term memory using a Photoplethysmography signal. J Med Syst 44(1):14. https://doi.org/10.1007/s10916-019-1485-0

    Article  Google Scholar 

  5. Kemp B, Zwinderman AH, Tuk B, Kamphuisen HAC, Oberye JJL (2000) Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG. IEEE Trans Biomed Eng 47(9):1185–1194

    Article  Google Scholar 

  6. Liu D., “A practical guide to Relu,” 30 November 2017. [Online]. Available: https://medium.com/@danqing/a-practical-guide-to-relu-b83ca804f1f7.

  7. Mousavi Z, Rezaii TY, Sheykhivand S, Farzamnia A, Razavi S (2019) Deep convolutional neural network for classification of sleep stages from single-channel EEG signals. J Neurosci Methods 324:108312. https://doi.org/10.1016/j.jneumeth.2019.108312

    Article  Google Scholar 

  8. O'Reilly C, Gosselin N, Carrier J, Nielsen T (2014 Dec) Montreal archive of sleep studies: an open-access resource for instrument benchmarking and exploratory research. J Sleep Res 23(6):628–635. https://doi.org/10.1111/jsr.12169 Epub 2014 Jun 9

  9. Phan H, Andreotti F, Cooray N, Chen OY, Vos MD (2019) Joint classification and prediction CNN framework for automatic sleep stage classification. IEEE Trans Biomed Eng 66(5):1285–1296

    Article  Google Scholar 

  10. Serengil S.I. (2018) The Insider’s Guide to Adam Optimization Algorithm for Deep Learning [Online]. Available: https://sefiks.com/2018/06/23/the-insiders-guide-to-adam-optimization-algorithm-for-deep-learning/.

  11. Sleep-EDF Database ( 2002). [Online]. Available: https://physionet.org/content/sleep-edf/1.0.0/.

  12. Sors A, Bonnet S, Mirek S, Vercueil L, Payen JF (2018) A convolutional neural network for sleep stage scoring from raw single-channel EEG. Biomed Signal Process Control 42:107–114

    Article  Google Scholar 

  13. Sun C, Fan J, Chen C, Li W, Chen W (2019) A two-stage neural network for sleep stage classification based on feature learning, sequence learning, and data augmentation. IEEE Access 7:109386–109397. https://doi.org/10.1109/ACCESS.2019.2933814

    Article  Google Scholar 

  14. Werth J, Radha M, Andriessen P, Arts RM, Long X (2020) Deep learning approach for ECG-based automatic sleep state classification in preterm infants. Biomed Signal Process Control 56:101663. https://doi.org/10.1016/j.bspc.2019.101663

    Article  Google Scholar 

  15. Xu Z, Yang X, Sun J, Liu P, Qin W (January 2020) Sleep stage classification using time-frequency spectra from consecutive multi-time points. Front Neurosci 14:14. https://doi.org/10.3389/fnins.2020.00014

    Article  Google Scholar 

  16. Zhang J, Yao R, Ge W, Gao J (2020 Jan) Orthogonal convolutional neural networks for automatic sleep stage classification based on single-channel EEG. Comput Methods Prog Biomed 183:105089. https://doi.org/10.1016/j.cmpb.2019.105089

    Article  Google Scholar 

  17. Zhang L, Fabbri D, Upender R, Kent D. Automated sleep stage scoring of the Sleep Heart Health Study using deep neural networks. Sleep. 2019;42(11):zsz159. https://doi.org/10.1093/sleep/zsz159.

  18. Zhang X, Xu M, Li Y, et al. Automated multi-model deep neural network for sleep stage scoring with unfiltered clinical data. Sleep Breathing = Schlaf Atmung. 2020 24(2):581–590. https://doi.org/10.1007/s11325-019-02008-w.

Download references

Author information

Authors and Affiliations

  1. School of Computing Mathematics and Engineering, Charles Sturt University (CSU), Sydney, Australia

    Akriti Bhusal, Abeer Alsadoon & P. W. C. Prasad

  2. School of Computer Data and Mathematical Sciences, Western Sydney University (WSU), Sydney, Australia

    Abeer Alsadoon & P. W. C. Prasad

  3. Kent Institute Australia, Sydney, Australia

    Abeer Alsadoon & P. W. C. Prasad

  4. Asia Pacific International College (APIC), Sydney, Australia

    Abeer Alsadoon

  5. Computer Science Department, Worcester State University, Worcester, MA, USA

    Nada Alsalami

  6. Computer Science and Engineering, University of Kurdistan Hewler, Erbil, KRG, Iraq

    Tarik A. Rashid

Authors
  1. Akriti Bhusal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abeer Alsadoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. W. C. Prasad
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nada Alsalami
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tarik A. Rashid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abeer Alsadoon.

Ethics declarations

Conflict of interest

No Funding for this work and no Conflicts of interests as well.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhusal, A., Alsadoon, A., Prasad, P.W.C. et al. Deep learning for sleep stages classification: modified rectified linear unit activation function and modified orthogonal weight initialisation. Multimed Tools Appl 81, 9855–9874 (2022). https://doi.org/10.1007/s11042-022-12372-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12372-7

Keywords

search

Navigation