Skip to main content
SpringerLink
Log in

Insights from the 2nd China intelligent sleep staging competition

  • Sleep Breathing Physiology and Disorders • Original Article
  • Published:
Sleep and Breathing Aims and scope Submit manuscript

Abstract

Study Objectives

Artificial intelligence (AI) is quickly advancing in the field of sleep medicine, which bodes well for the potential of actual clinical use. In this study, an analysis of the 2nd China Intelligent Sleep Staging Competition was conducted to gain insights into the general level and constraints of AI-assisted sleep staging in China.

Methods

The outcomes of 10 teams from the children‘s track and 13 teams from the adult track were investigated in this study. The analysis included overall performance, differences between five different sleep stages, variations across subjects, and performance during stage transitions.

Results

The adult track's accuracy peaked at 80.46%, while the children's track's accuracy peaked at 88.96%. On average, accuracy rates stood at 71.43% for children and 68.40% for adults. All results were produced within a mere 5-min timeframe. The N1 stage was prone to misclassification as W, N2, and R stages. In the adult track, significant differences were apparent among subjects (p < 0.05), whereas in the children’s track, such differences were not observed. Nonetheless, both tracks experienced a performance decline during stage transitions.

Conclusions

The computational speed of AI is remarkably fast, simultaneously holding the potential to surpass the accuracy of physicians. Improving the machine learning model's classification of the N1 stage and transitional periods between stages, along with bolstering its robustness to individual subject variations, is imperative for maximizing its ability in assisting clinical scoring.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The datasets analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Berry RB et al (2017) AASM scoring manual updates for 2017 (Version 2.4). J Clin Sleep Med 13(5):665–666

    Article  PubMed  PubMed Central  Google Scholar 

  2. Foulkes D (1982) A cognitive-psychological model of REM dream production. Sleep 5(2):169–187

    Article  CAS  PubMed  Google Scholar 

  3. Tamaki M et al (2020) Complementary contributions of non-REM and REM sleep to visual learning. Nat Neurosci 23(9):1150–1156

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Diekelmann S, Büchel C, Born J, Rasch B (2011) Labile or stable: opposing consequences for memory when reactivated during waking and sleep. Nat Neurosci 14(3):381–386

    Article  CAS  PubMed  Google Scholar 

  5. I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. MIT press, 2016.

  6. He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K (2019) The practical implementation of artificial intelligence technologies in medicine. Nat Med 25(1):30–36

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Dreyer KJ, Geis JR (2017) When machines think: radiology’s next frontier. Radiology 285(3):713–718

    Article  PubMed  Google Scholar 

  8. Li Y, Luo S, Zhang H, Zhang Y, Zhang Y, Lo B (2023) MtCLSS: Multi-task contrastive learning for semi-supervised pediatric sleep staging. IEEE J Biomed Health Inform 27(6):2647–2655

    Article  PubMed  Google Scholar 

  9. S. Nasiri and G. D. Clifford, "Boosting automated sleep staging performance in big datasets using population subgrouping," Sleep, vol. 44, no. 7, 2021, Art. no. zsab027.

  10. Baumert M, Hartmann S, Phan H (2023) Automatic sleep staging for the young and the old-Evaluating age bias in deep learning. Sleep Med 107:18–25

    Article  PubMed  Google Scholar 

  11. L. Cheng, S. Luo, X. Yu, H. Ghayvat, H. Zhang, and Y. Zhang, "EEG-CLNet: Collaborative learning for simultaneous measurement of sleep stages and OSA events based on single EEG signal," IEEE Transactions on Instrumentation and Measurement, vol. 72, 2023, Art. no. 2503910.

  12. Perez-Pozuelo I et al (2020) The future of sleep health: a data-driven revolution in sleep science and medicine. NPJ Digital Medicine 3:42

    Article  PubMed  PubMed Central  Google Scholar 

  13. H. Korkalainen et al., "Deep learning enables sleep staging from photoplethysmogram for patients with suspected sleep apnea," Sleep, vol. 43, no. 11, 2020, Art. no. zsaa098.

  14. H. Q. Sun et al., "Sleep staging from electrocardiography and respiration with deep learning," Sleep, vol. 43, no. 7, 2020, Art. no. zsz306.

  15. Eldele E et al (2021) An attention-based deep learning approach for sleep stage classification with single-channel EEG. IEEE Trans Neural Syst Rehabil Eng 29:809–818

    Article  PubMed  Google Scholar 

  16. Horne J (2013) Why REM sleep? Clues beyond the laboratory in a more challenging world. Biol Psychol 92(2):152–168

    Article  PubMed  Google Scholar 

  17. Hassan AR, Bhuiyan MIH (2017) An automated method for sleep staging from EEG signals using normal inverse Gaussian parameters and adaptive boosting. Neurocomputing 219:76–87

    Article  Google Scholar 

  18. Huy P, Andreotti F, Cooray N, Chen OY, De Vos M (2019) Joint classification and prediction CNN framework for automatic sleep stage classification. IEEE Trans Biomed Eng 66(5):1285–1296

    Article  Google Scholar 

  19. D. Zhou et al., "LightSleepNet: A lightweight deep model for rapid sleep stage classification with spectrograms," in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2021, pp. 43–46: IEEE.

  20. Zhou D et al (2022) SingleChannelNet: A model for automatic sleep stage classification with raw single-channel EEG. Biomedical Signal Processing Control 75:103592

    Article  Google Scholar 

  21. Penzel T, Zhang X, Fietze I (2013) Inter-scorer reliability between sleep centers can teach us what to improve in the scoring rules. J Clin Sleep Med 9(1):89–91

    Article  PubMed  PubMed Central  Google Scholar 

  22. P. E. McKight and J. Najab, "Kruskal‐wallis test," The Corsini Encyclopedia of Psychology, pp. 1–1, 2010.

  23. Rosenberg RS, Van Hout S (2013) The American Academy of Sleep Medicine inter-scorer reliability program: sleep stage scoring. J Clin Sleep Med 9(1):81–87

    Article  PubMed  PubMed Central  Google Scholar 

  24. Goldstein CA et al (2020) Artificial intelligence in sleep medicine: an American Academy of Sleep Medicine position statement. J Clin Sleep Med 16(4):605–607

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 62172340, and in part by the Natural Science Foundation of Chongqing under Grant cstc2021jcyjmsxmX0041.

Author information

Authors and Affiliations

  1. College of Electronic and Information Engineering, Southwest University, Chongqing, 400715, China

    Yamei Li, Zhiqiang Chen & Yuan Zhang

  2. Department of Respiratory Medicine, Beijing Children’s Hospital, Capital Medical University, Beijing, 100045, China

    Zhifei Xu

  3. Department of Psychiatry, Nanfang Hospital, Southern Medical University, Guangzhou, 510515, China

    Bin Zhang

Authors
  1. Yamei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhifei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiqiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuan Zhang.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Xu, Z., Chen, Z. et al. Insights from the 2nd China intelligent sleep staging competition. Sleep Breath (2024). https://doi.org/10.1007/s11325-024-03055-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11325-024-03055-8

Keywords

Search

Navigation