Skip to main content
SpringerLink
Log in

Developing probabilistic ensemble machine learning models for home-based sleep apnea screening using overnight SpO2 data at varying data granularity

  • Sleep Breathing Physiology and Disorders • Original Article
  • Published:
Sleep and Breathing Aims and scope Submit manuscript

Abstract

Purpose

This study aims to develop sleep apnea screening models with overnight SpO2 data, and to investigate the impact of the SpO2 data granularity on model performance.

Methods

A total of 7,718 SpO2 recordings from the SHHS and MESA datasets were used. Probabilistic ensemble machine learning was employed to predict sleep apnea status at three AHI cutoff points: ≥ 5, ≥ 15, and ≥ 30 events/hour. To investigate the impact of data granularity, SpO2 data were aggregated at 30, 60, and 300 s.

Results

Our models demonstrated good to excellent performance on internal test, with average area under the curve (AUC) values of 0.91, 0.93, and 0.96 for cutoffs ≥ 5, ≥ 15, and ≥ 30 at data granularity of 1 s, respectively. Both sensitivity (0.76, 0.84, 0.89) and specificity (0.87, 0.86, 0.90) ranged from good to excellent across three cutoffs. Positive predictive values (PPV) ranged from excellent to fair (0.97, 0.83, 0.66), and negative predictive values (NPV) ranged from low to excellent (0.43, 0.87, 0.98). Model performance on external test slightly dropped compared to internal test, but still achieved good to excellent AUC above 0.80 across all data granularity and all the three cutoffs. Data granularity of 300 s led to a reduction in performance metrics across all cutoffs.

Conclusion

Our models demonstrated superior performance across all three AHI cutoff thresholds compared to existing large sleep apnea screening models, even when considering varying SpO2 data granularity. However, lower data granularity was associated with decreased screening performance, indicating a need for further research in this area.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The SHHS and MESA datasets used in this study are available at https://sleepdata.org/datasets with the permission of NSRR.

References

  1. Senaratna CV et al (2017) Prevalence of obstructive sleep apnea in the general population: a systematic review. Sleep Med Rev 34:70–81

    Article  PubMed  Google Scholar 

  2. Adam VB et al (2019) Estimation of the global prevalence and burden of obstructive sleep apnoea: a literature-based analysis. Lancet Respir Med 7(8):687–698

    Article  Google Scholar 

  3. Pivetta B, Chen L, Nagappa M et al (2021) Use and performance of the STOP-Bang questionnaire for obstructive sleep apnea screening across geographic regions: a systematic review and meta-analysis. JAMA Netw Open 4(3):e211009. https://doi.org/10.1001/jamanetworkopen.2021.1009

    Article  PubMed  PubMed Central  Google Scholar 

  4. Mendonça F, Mostafa SS, Ravelo-García AG, Morgado-Dias F (2018) Penzel T (2018) Devices for home detection of obstructive sleep apnea: a review. Sleep Med Rev 41:149–160

    Article  PubMed  Google Scholar 

  5. Rodrigues J, Pepin JL, Goeuriot L, Amer-Yahia S (2020) An extensive investigation of machine learning techniques for sleep apnea screening. In: Proceedings of the 29th ACM International Conference on Information and Knowledge Management, Virtual Event Ireland, France

  6. Wang S, Xuan W, Chen D et al (2023) Machine learning assisted wearable wireless device for sleep apnea syndrome diagnosis. Biosensors 13(4):483. https://doi.org/10.3390/bios13040483

    Article  PubMed  PubMed Central  Google Scholar 

  7. Li Z, Li Y, Zhao G, Zhang X et al (2021) A model for obstructive sleep apnea detection using a multi-layer feed-forward neural network based on electrocardiogram, pulse oxygen saturation, and body mass index. Sleep Breath 25(4):2065–2072. https://doi.org/10.1007/s11325-021-02302-6

    Article  PubMed  Google Scholar 

  8. Wei K, Zou L, Liu G, Wang C (2023) MS-Net: sleep apnea detection in PPG using multi-scale block and shadow module one-dimensional convolutional neural network. Comput Biol Med 155:106469. https://doi.org/10.1016/j.compbiomed.2022.106469

    Article  PubMed  Google Scholar 

  9. Liang Z (2023) Novel method combining multiscale attention entropy of overnight blood oxygen level and machine learning for easy sleep apnea screening. Digit Health 2023:9. https://doi.org/10.1177/20552076231211550

    Article  Google Scholar 

  10. Xie B, Minn H (2012) Real-time sleep apnea detection by classifier combination. IEEE Trans Inf Technol Biomed 16(3):469–477. https://doi.org/10.1109/TITB.2012.2188299

    Article  PubMed  Google Scholar 

  11. Lin CY, Wang YW, Setiawan F et al (2021) Sleep apnea classification algorithm development using a machine-learning framework and bag-of-features derived from electrocardiogram spectrograms. J Clin Med 11(1). https://doi.org/10.3390/jcm11010192

  12. Bhattacharjee A, Saha S, Fattah SA et al (2019) Sleep apnea detection based on Rician modeling of feature variation in multiband EEG signal. IEEE J Biomed Health Inform 23(3):1066–1074. https://doi.org/10.1109/JBHI.2018.2845303

    Article  PubMed  Google Scholar 

  13. Bahrami M, Forouzanfar M (2022) Sleep apnea detection from single-lead ECG: a comprehensive analysis of machine learning and deep learning algorithms. IEEE Trans Instrum Meas 71:1–11. https://doi.org/10.1109/TIM.2022.3151947

    Article  Google Scholar 

  14. Liang Z, Ploderer B (2020) “How does Fitbit measure brainwaves”: a qualitative study into the credibility of sleep-tracking technologies. Proc ACM Interact Mob Wearable Ubiquitous Technol 4(1):1–29. https://doi.org/10.1145/3380994

    Article  Google Scholar 

  15. Liang Z, Chapa-Martell MA (2021) A multi-level classification approach for sleep stage prediction with processed data derived from consumer wearable activity trackers. Front Digit Health 3:665946. https://doi.org/10.3389/fdgth.2021.665946

    Article  PubMed  PubMed Central  Google Scholar 

  16. Liang Z, Chapa-Martell MA (2019) Accuracy of Fitbit wristbands in measuring sleep stage transitions and the effect of user-specific factors. JMIR mhealth uhealth 7(6):e13384. https://doi.org/10.2196/13384

    Article  PubMed  PubMed Central  Google Scholar 

  17. Lin HC, Su CL, Ong JH et al (2020) Pulse oximetry monitor feasible for early screening of obstructive sleep apnea (OSA). J Med Biol Eng 40:62–70. https://doi.org/10.1007/s40846-019-00479-6

    Article  Google Scholar 

  18. Rodrigues Filho JC, Neves DD, Velasque L et al (2020) Diagnostic performance of nocturnal oximetry in the detection of obstructive sleep apnea syndrome: a Brazilian study. Sleep Breath Physiol Disord 24:1487–1494. https://doi.org/10.1007/s11325-019-02000-4

    Article  Google Scholar 

  19. Quan SF, Howard BV, Iber C et al (1997) The Sleep Heart Health Study: design, rationale, and methods. Sleep 20(12):1077–1085

    CAS  PubMed  Google Scholar 

  20. Chen X, Wang R, Zee P, Lutsey PL, Javaheri S, Alcántara C, Jackson CL, Williams MA, Redline S (2015) Racial/ethnic differences in sleep disturbances: the Multi-Ethnic Study of Atherosclerosis (MESA). Sleep 38(6):877–888

    PubMed  PubMed Central  Google Scholar 

  21. Zhang GQ, Cui L, Mueller R et al (2018) The National Sleep Research Resource: towards a sleep data commons. J Am Med Inform Assoc 25(10):1351–1358. https://doi.org/10.1093/jamia/ocy064

    Article  PubMed  PubMed Central  Google Scholar 

  22. Bernardini A, Brunello A, Gigli GL et al (2022) OSASUD: a dataset of stroke unit recordings for the detection of obstructive sleep apnea syndrome. Sci Data 9:177. https://doi.org/10.1038/s41597-022-01272-y

    Article  PubMed  PubMed Central  Google Scholar 

  23. Berry R, Brooks R, Gamaldo C et al (2017) The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications, version 2.4 edn. American Academy of Sleep Medicine, Darien

  24. Kwak N, Choi CH (2003) Feature extraction based on ICA for binary classification problems. IEEE Trans Knowl Data Eng 15(6):1374–1388. https://doi.org/10.1109/TKDE.2003.1245279

    Article  Google Scholar 

  25. van Rijn J, Hutter F (2018) Hyperparameter importance across datasets. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD '18). pp 2367–2376. https://doi.org/10.1145/3219819.3220058

  26. Brodersen KH, Ong CS, Stephan KE, Buhmann JM (2010) The balanced accuracy and its posterior distribution. In: Proceedings of the 20th International Conference on Pattern Recognition, Istanbul, Turkey, 2010

  27. Levy J, Alvarez D, del Campo F, Behar J (2023) Deep learning for obstructive sleep apnea diagnosis based on single channel oximetry. Nat Commun 14(1):4881. https://doi.org/10.1038/s41467-023-40604-3

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Holfinger SJ, Lyons MM, Keenan BT et al (2022) Diagnostic performance of machine learning-derived OSA prediction tools in large clinical and community-based samples. Chest 161(3):807–817. https://doi.org/10.1016/j.chest.2021.10.023

    Article  PubMed  Google Scholar 

  29. Shi Y, Zhang Y, Cao Z et al (2023) Application and interpretation of machine learning models in predicting the risk of severe obstructive sleep apnea in adults. BMC Med Inform Decis Mak 23:1–15. https://doi.org/10.1186/s12911-023-02331-z

    Article  Google Scholar 

  30. Banerjee I, Gensheimer MF, Wood DJ et al (2018) Probabilistic prognostic estimates of survival in metastatic cancer patients (PPES-Met) utilizing free-text clinical narratives. Sci Rep 8(1):10037. https://doi.org/10.1038/s41598-018-27946-5

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Pevernagie DA, Gnidovec-Strazisar B, Grote L et al (2020) On the rise and fall of the apnea-hypopnea index: a historical review and critical appraisal. J Sleep Res 29(4):e13066. https://doi.org/10.1111/jsr.13066

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The author would like to express gratitude to the National Sleep Research Resources (NSRR) for granting access to the SHHS and MESA datasets.

Funding

Japan Society for the Promotion of Science (JSPS) provided financial support in the form of KAKENHI (Grant Number 21K17670) funding. The sponsor had no role in the design or conduct of this research.

Author information

Authors and Affiliations

  1. Ubiquitous and Personal Computing Lab, Kyoto University of Advanced Science (KUAS), 18 Yamanouchi Gotanda-cho, Ukyo-ku, Kyoto, Japan

    Zilu Liang

Authors
  1. Zilu Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zilu Liang.

Ethics declarations

Ethical approval

This article does not contain any studies with human participants performed by the author. Access to the SHHS dataset was granted by the NSRR.

Conflict of interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers' bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, Z. Developing probabilistic ensemble machine learning models for home-based sleep apnea screening using overnight SpO2 data at varying data granularity. Sleep Breath (2024). https://doi.org/10.1007/s11325-024-03141-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11325-024-03141-x

Keywords

Search

Navigation