Skip to main content

Advertisement

SpringerLink
Log in

Home sleep apnea testing: comparison of manual and automated scoring across international sleep centers

  • Sleep Breathing Physiology and Disorders • Original Article
  • Published:
Sleep and Breathing Aims and scope Submit manuscript

Abstract

Purpose

To determine the agreement between the manual scoring of home sleep apnea tests (HSATs) by international sleep technologists and automated scoring systems.

Methods

Fifteen HSATs, previously recorded using a type 3 monitor, were saved in European Data Format. The studies were scored by nine experienced technologists from the sleep centers of the Sleep Apnea Global Interdisciplinary Consortium (SAGIC) using the locally available software. Each study was scored separately by human scorers using the nasal pressure (NP), flow derived from the NP signal (transformed NP), or respiratory inductive plethysmography (RIP) flow. The same procedure was followed using two automated scoring systems: Remlogic (RLG) and Noxturnal (NOX).

Results

The intra-class correlation coefficients (ICCs) of the apnea-hypopnea index (AHI) scoring using the NP, transformed NP, and RIP flow were 0.96 [95% CI 0.93–0.99], 0.98 [0.96–0.99], and 0.97 [0.95–0.99], respectively. Using the NP signal, the mean differences in AHI between the average of the manual scoring and the automated systems were − 0.9 ± 3.1/h (AHIRLG vs AHIMANUAL) and − 1.3 ± 2.6/h (AHINOX vs AHIMANUAL). Using the transformed NP, the mean differences in AHI were − 1.9 ± 3.3/h (AHIRLG vs AHIMANUAL) and 1.6 ± 3.0/h (AHINOX vs AHIMANUAL). Using the RIP flow, the mean differences in AHI were − 2.7 ± 4.5/h (AHIRLG vs AHIMANUAL) and 2.3 ± 3.4/h (AHINOX vs AHIMANUAL).

Conclusions

There is very strong agreement in the scoring of the AHI for HSATs between the automated systems and experienced international technologists. Automated scoring of HSATs using commercially available software may be useful to standardize scoring in future endeavors involving international sleep centers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Abbreviations

AHI:

apnea-hypopnea Index

EDF:

European data format

HSAT:

home sleep apnea testing

ICC:

intra-class correlation coefficient

MEANdiff :

mean difference

NOX:

Noxturnal

NP:

nasal pressure

PSG:

polysomnography

RIP:

respiratory inductive plethysmography

RLG:

Remlogic

SAGIC:

Sleep Apnea Global Interdisciplinary Consortium

References

  1. Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernandez-Blasco L, Monasterio C, Alonso A, Chiner E, Rubio M, Garcia-Ledesma E, Cacelo L, Carpizo R, Sacristan L, Salord N, Carrera M, Sancho-Chust JN, Embid C, Vazquez-Polo FJ, Negrin MA, Montserrat JM (2011) Effectiveness of home respiratory polygraphy for the diagnosis of sleep apnoea and hypopnoea syndrome. Thorax 66(7):567–573

    Article  PubMed  Google Scholar 

  2. Whittle AT, Finch SP, Mortimore IL, MacKay TW, Douglas NJ (1997) Use of home sleep studies for diagnosis of the sleep apnoea/hypopnoea syndrome. Thorax 52(12):1068–1073

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Kuna ST, Badr MS, Kimoff RJ, Kushida C, Lee-Chiong T, Levy P, McNicholas WT, Strollo PJ, on behalf of the ATS/AASM/ACCP/ERS Committee on Ambulatory Management of Adults with OSA (2011) An official ATS/AASM/ACCP/ERS workshop report: research priorities in ambulatory management of adults with obstructive sleep apnea. Proc Am Thorac Soc 8(1):1–16

    Article  PubMed  Google Scholar 

  4. Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernández-Blasco L, Monasterio C, Alonso A, Chiner E, Zamorano J, Aizpuru F, Montserrat JM, and the Spanish Sleep Network (2011) Therapeutic decision-making for sleep apnea and hypopnea syndrome using home respiratory polygraphy: a large multicentric study. Am J Respir Crit Care Med 184(8):964–971

    Article  PubMed  Google Scholar 

  5. Flemons WW, Littner MR, Rowley JA, Gay P, Anderson WMD, Hudgel DW, McEvoy RD, Loube DI (2003) Home diagnosis of sleep apnea: a systematic review of the literature. An evidence review cosponsored by the American Academy of Sleep Medicine, the American College of Chest Physicians, and the American Thoracic Society. Chest 124(4):1543–1579

    Article  PubMed  Google Scholar 

  6. Berry RB, Budhiraja R, Gottlieb DJ, Gozal D, Iber C, Kapur VK, Marcus CL, Mehra R, Parthasarathy S, Quan SF, Redline S, Strohl KP, Davidson Ward SL, Tangredi MM, American Academy of Sleep Medicine (2012) Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events. Deliberations of the sleep apnea definitions task force of the American Academy of Sleep Medicine. J Clin Sleep Med 8(5):597–619

    PubMed  PubMed Central  Google Scholar 

  7. Magalang UJ, Arnardottir ES, Chen NH, Cistulli PA, Gíslason T, Lim D, Penzel T, Schwab R, Tufik S, Pack AI, SAGIC Investigators (2016) Agreement in the scoring of respiratory events among international sleep centers for home sleep testing. J Clin Sleep Med 12(1):71–77

    Article  PubMed  PubMed Central  Google Scholar 

  8. Aurora RN, Swartz R, Punjabi NM (2015) Misclassification of OSA severity with automated scoring of home sleep recordings. Chest 147(3):719–727

    Article  PubMed  Google Scholar 

  9. Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernández-Blasco L, Monasterio C, Alonso-Fernandez A, Chiner E, Vázquez-Polo FJ, Montserrat JM, the Spanish Sleep Group (2013) Effectiveness of sequential automatic-manual home respiratory polygraphy scoring. Eur Respir J 41(4):879–887

    Article  PubMed  Google Scholar 

  10. Hedner J, Grote L, Bonsignore M, McNicholas W, Lavie P, Parati G, Sliwinski P, Barbe F, de Backer W, Escourrou P, Fietze I, Kvamme JA, Lombardi C, Marrone O, Masa JF, Montserrat JM, Penzel T, Pretl M, Riha R, Rodenstein D, Saaresranta T, Schulz R, Tkacova R, Varoneckas G, Vitols A, Vrints H, Zielinski J (2011) The European Sleep Apnoea Database (ESADA): report from 22 European sleep laboratories. Eur Respir J 38(3):635–642

    Article  CAS  PubMed  Google Scholar 

  11. McEvoy RD, Antic NA, Heeley E, Luo Y, Ou Q, Zhang X, Mediano O, Chen R, Drager LF, Liu Z, Chen G, du B, McArdle N, Mukherjee S, Tripathi M, Billot L, Li Q, Lorenzi-Filho G, Barbe F, Redline S, Wang J, Arima H, Neal B, White DP, Grunstein RR, Zhong N, Anderson CS, SAVE Investigators and Coordinators (2016) CPAP for prevention of cardiovascular events in obstructive sleep apnea. N Engl J Med 375(10):919–931

    Article  PubMed  Google Scholar 

  12. Kemp B, Varri A, Rosa AC, Nielsen KD, Gade J (1992) A simple format for exchange of digitized polygraphic recordings. Electroencephalogr Clin Neurophysiol 82(5):391–393

    Article  CAS  PubMed  Google Scholar 

  13. Berry RB, Brooks R, Gamaldo C, et al. The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications. Version 2.4. In: Darien. Version 2.4. In: Darien, IL: American Academy of Sleep Medicine; 2017

  14. Magalang UJ, Chen N-H, Cistulli PA et al (2013) Agreement in the scoring of respiratory events and sleep among international sleep centers. Sleep 36(4):591–596

    Article  PubMed  PubMed Central  Google Scholar 

  15. Dingli K, Coleman EL, Vennelle M, Finch SP, Wraith PK, Mackay TW, Douglas NJ (2003) Evaluation of a portable device for diagnosing the sleep apnoea/hypopnoea syndrome. Eur Respir J 21(2):253–259

    Article  CAS  PubMed  Google Scholar 

  16. Smith LA, Chong DW, Vennelle M, Denvir MA, Newby DE, Douglas NJ (2007) Diagnosis of sleep-disordered breathing in patients with chronic heart failure: evaluation of a portable limited sleep study system. J Sleep Res 16(4):428–435

    Article  PubMed  Google Scholar 

  17. Munro B. Statistical methods for health care research. 5th ed. Philadelphia: Lippincott Williams Wilkins; 2005

  18. Cheng JW, Tsai WC, Yu TY, Huang KY Reproducibility of sonographic measurement of thickness and echogenicity of the plantar fascia. J Clin Ultrasound 40(1):14–19

  19. Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Stat Methods Med Res 8(2):135–160

    Article  CAS  PubMed  Google Scholar 

  20. Pittman SD, MacDonald MM, Fogel RB et al (2004) Assessment of automated scoring of polysomnographic recordings in a population with suspected sleep-disordered breathing. Sleep 27(7):1394–1403

    Article  PubMed  Google Scholar 

  21. Malhotra A, Younes M, Kuna ST, Benca R, Kushida CA, Walsh J, Hanlon A, Staley B, Pack AI, Pien GW (2013) Performance of an automated polysomnography scoring system versus computer-assisted manual scoring. Sleep 36(4):573–582

    Article  PubMed  PubMed Central  Google Scholar 

  22. Punjabi NM, Shifa N, Dorffner G, Patil S, Pien G, Aurora RN (2015) Computer-assisted automated scoring of polysomnograms using the Somnolyzer system. Sleep 38(10):1555–1566

    Article  PubMed  PubMed Central  Google Scholar 

  23. Thurnheer R, Xie X, Bloch KE (2001) Accuracy of nasal cannula pressure recordings for assessment of ventilation during sleep. Am J Respir Crit Care Med 164(10 Pt 1):1914–1919

    Article  CAS  PubMed  Google Scholar 

  24. Farre R, Rigau J, Montserrat JM, Ballester E, Navajas D (2001) Relevance of linearizing nasal prongs for assessing hypopneas and flow limitation during sleep. Am J Respir Crit Care Med 163(2):494–497

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the following individuals who helped in this project: Mohammad Ahmadi, Alexander Blau, Petra Cornell, Silverio Garbuio, Su-Lan Liu, João Reinfelderon, Beth Staley, Magdalena Ósk Sigurgunnarsdóttir, and Sandra Zimmermann.

Funding

Supported by NHLBI award P01 HL094307 (AIP), Conselho Nacional de Desenvolvimento Cientffico e Tecnologico (CNPq) grant 309336/2017–1 (LRB), Conselho Nacional de Desenvolvimento Cientffico e Tecno1ogico (CNPq), Grant 401569/2016–0 (LRB), and Award grant number UL1TR001070 from the National Center for Advancing Translational Sciences. The sponsor had no role in the design or conduct of the research.

Author information

Authors and Affiliations

  1. Division of Pulmonary, Allergy, Critical Care, and Sleep Medicine, The Ohio State University Wexner Medical Center, 201 Davis Heart and Lung Research Institute, 473 West 12th Avenue, Columbus, OH, 43210, USA

    Ulysses J. Magalang, Jennica N. Johns, Katherine A. Wood & Jesse W. Mindel

  2. Neuroscience Research Institute, The Ohio State University Wexner Medical Center, Columbus, OH, USA

    Ulysses J. Magalang

  3. Center for Sleep and Circadian Neurobiology, Division of Sleep Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA

    Diane C. Lim & Allan I. Pack

  4. Departamento de Psicobiologia, Universidade Federal de São Paulo, São Paulo, Brazil

    Lia R. Bittencourt & Sergio Tufik

  5. Division of Pulmonary, Critical Care, and Sleep Medicine, Chang Gung Memorial Hospital, Taoyuan, Taiwan

    Ning-Hung Chen

  6. Charles Perkins Centre, University of Sydney, Camperdown, Australia

    Peter A. Cistulli

  7. Department of Respiratory and Sleep Medicine, Royal North Shore Hospital, Sydney, Australia

    Peter A. Cistulli

  8. Department of Sleep Medicine, Landspitali University Hospital, Reykjavik, Iceland

    Thorarinn Gíslason & Erna S. Arnardottir

  9. Medical Faculty, University of Iceland, Reykjavik, Iceland

    Thorarinn Gíslason & Erna S. Arnardottir

  10. Interdisciplinary Center of Sleep Medicine, Charité University Hospital, Berlin, Germany

    Thomas Penzel

Authors
  1. Ulysses J. Magalang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jennica N. Johns
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Katherine A. Wood
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jesse W. Mindel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Diane C. Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lia R. Bittencourt
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ning-Hung Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Peter A. Cistulli
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Thorarinn Gíslason
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Erna S. Arnardottir
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Thomas Penzel
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Sergio Tufik
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Allan I. Pack
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ulysses J. Magalang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Formal consent is not required for this type of study.

Electronic supplementary material

ESM 1

(DOCX 14 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Magalang, U.J., Johns, J.N., Wood, K.A. et al. Home sleep apnea testing: comparison of manual and automated scoring across international sleep centers. Sleep Breath 23, 25–31 (2019). https://doi.org/10.1007/s11325-018-1715-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11325-018-1715-6

Keywords

Search

Navigation