Skip to main content
Log in

Deep learning for sleep stages classification: modified rectified linear unit activation function and modified orthogonal weight initialisation

Multimedia Tools and Applications Aims and scope Submit manuscript

Cite this article


Each stage of sleep can affect human health, and not getting enough sleep at any stage may lead to sleep disorder like parasomnia, apnea, insomnia, etc. Sleep-related diseases could be diagnosed using Convolutional Neural Network Classifier. However, this classifier has not been successfully implemented into sleep stage classification systems due to high complexity and low accuracy of classification. The aim of this research is to increase the accuracy and reduce the learning time of Convolutional Neural Network Classifier. The proposed system used a modified Orthogonal Convolutional Neural Network and a modified Adam optimisation technique to improve the sleep stage classification accuracy and reduce the gradient saturation problem that occurs due to sigmoid activation function. The proposed system uses Leaky Rectified Linear Unit (ReLU) instead of sigmoid activation function as an activation function. The proposed system called Enhanced Sleep Stage Classification system (ESSC) used six different databases for training and testing the proposed model on the different sleep stages. These databases are University College Dublin database (UCD), Beth Israel Deaconess Medical Center MIT database (MIT-BIH), Sleep European Data Format (EDF), Sleep EDF Extended, Montreal Archive of Sleep Studies (MASS), and Sleep Heart Health Study (SHHS). Our results show that the gradient saturation problem does not exist anymore. The modified Adam optimiser helps to reduce the noise which in turn result in faster convergence time. The convergence speed of ESSC is increased along with better classification accuracy compared to the state of art solution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7



Convolutional Neural Network


Recurrent Neural Network


Long Short Term Memory


Fast Fourier Transform


Short-Time Fourier Transform


Hilbert Huang Transform






Wake state


Sleep Stage 1


Sleep Stage 2


Sleep Wake State


Rapid Eye Movement


Enhanced Sleep Stage Classification


Rectified Linear Unit


  1. Erdenebayar U, Kim YJ, Park JU, Joo EY, Lee KJ (2019 Oct) Deep learning approaches for automatic detection of sleep apnea events from an electrocardiogram. Comput Methods Prog Biomed 180:105001.

    Article  Google Scholar 

  2. Fernandez-Blanco E, Rivero D, Pazos A (2020) Convolutional neural networks for sleep stage scoring on a two-channel EEG signal. Soft Comput 24:4067–4079.

    Article  Google Scholar 

  3. Hu J, Shen L, Albanie S, Sun G, Wu E (2019) Squeeze-and-excitation networks. IEEE Trans Patt Anal Mach Intell

  4. Kang CH, Erdenebayar U, Park JU, Lee KJ (2019 Dec) Multi-class classification of sleep apnea/hypopnea events based on long short-term memory using a Photoplethysmography signal. J Med Syst 44(1):14.

    Article  Google Scholar 

  5. Kemp B, Zwinderman AH, Tuk B, Kamphuisen HAC, Oberye JJL (2000) Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG. IEEE Trans Biomed Eng 47(9):1185–1194

    Article  Google Scholar 

  6. Liu D., “A practical guide to Relu,” 30 November 2017. [Online]. Available:

  7. Mousavi Z, Rezaii TY, Sheykhivand S, Farzamnia A, Razavi S (2019) Deep convolutional neural network for classification of sleep stages from single-channel EEG signals. J Neurosci Methods 324:108312.

    Article  Google Scholar 

  8. O'Reilly C, Gosselin N, Carrier J, Nielsen T (2014 Dec) Montreal archive of sleep studies: an open-access resource for instrument benchmarking and exploratory research. J Sleep Res 23(6):628–635. Epub 2014 Jun 9

  9. Phan H, Andreotti F, Cooray N, Chen OY, Vos MD (2019) Joint classification and prediction CNN framework for automatic sleep stage classification. IEEE Trans Biomed Eng 66(5):1285–1296

    Article  Google Scholar 

  10. Serengil S.I. (2018) The Insider’s Guide to Adam Optimization Algorithm for Deep Learning [Online]. Available:

  11. Sleep-EDF Database ( 2002). [Online]. Available:

  12. Sors A, Bonnet S, Mirek S, Vercueil L, Payen JF (2018) A convolutional neural network for sleep stage scoring from raw single-channel EEG. Biomed Signal Process Control 42:107–114

    Article  Google Scholar 

  13. Sun C, Fan J, Chen C, Li W, Chen W (2019) A two-stage neural network for sleep stage classification based on feature learning, sequence learning, and data augmentation. IEEE Access 7:109386–109397.

    Article  Google Scholar 

  14. Werth J, Radha M, Andriessen P, Arts RM, Long X (2020) Deep learning approach for ECG-based automatic sleep state classification in preterm infants. Biomed Signal Process Control 56:101663.

    Article  Google Scholar 

  15. Xu Z, Yang X, Sun J, Liu P, Qin W (January 2020) Sleep stage classification using time-frequency spectra from consecutive multi-time points. Front Neurosci 14:14.

    Article  Google Scholar 

  16. Zhang J, Yao R, Ge W, Gao J (2020 Jan) Orthogonal convolutional neural networks for automatic sleep stage classification based on single-channel EEG. Comput Methods Prog Biomed 183:105089.

    Article  Google Scholar 

  17. Zhang L, Fabbri D, Upender R, Kent D. Automated sleep stage scoring of the Sleep Heart Health Study using deep neural networks. Sleep. 2019;42(11):zsz159.

  18. Zhang X, Xu M, Li Y, et al. Automated multi-model deep neural network for sleep stage scoring with unfiltered clinical data. Sleep Breathing = Schlaf Atmung. 2020 24(2):581–590.

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Abeer Alsadoon.

Ethics declarations

Conflict of interest

No Funding for this work and no Conflicts of interests as well.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhusal, A., Alsadoon, A., Prasad, P.W.C. et al. Deep learning for sleep stages classification: modified rectified linear unit activation function and modified orthogonal weight initialisation. Multimed Tools Appl 81, 9855–9874 (2022).

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: