To determine the agreement between the manual scoring of home sleep apnea tests (HSATs) by international sleep technologists and automated scoring systems.
Fifteen HSATs, previously recorded using a type 3 monitor, were saved in European Data Format. The studies were scored by nine experienced technologists from the sleep centers of the Sleep Apnea Global Interdisciplinary Consortium (SAGIC) using the locally available software. Each study was scored separately by human scorers using the nasal pressure (NP), flow derived from the NP signal (transformed NP), or respiratory inductive plethysmography (RIP) flow. The same procedure was followed using two automated scoring systems: Remlogic (RLG) and Noxturnal (NOX).
The intra-class correlation coefficients (ICCs) of the apnea-hypopnea index (AHI) scoring using the NP, transformed NP, and RIP flow were 0.96 [95% CI 0.93–0.99], 0.98 [0.96–0.99], and 0.97 [0.95–0.99], respectively. Using the NP signal, the mean differences in AHI between the average of the manual scoring and the automated systems were − 0.9 ± 3.1/h (AHIRLG vs AHIMANUAL) and − 1.3 ± 2.6/h (AHINOX vs AHIMANUAL). Using the transformed NP, the mean differences in AHI were − 1.9 ± 3.3/h (AHIRLG vs AHIMANUAL) and 1.6 ± 3.0/h (AHINOX vs AHIMANUAL). Using the RIP flow, the mean differences in AHI were − 2.7 ± 4.5/h (AHIRLG vs AHIMANUAL) and 2.3 ± 3.4/h (AHINOX vs AHIMANUAL).
There is very strong agreement in the scoring of the AHI for HSATs between the automated systems and experienced international technologists. Automated scoring of HSATs using commercially available software may be useful to standardize scoring in future endeavors involving international sleep centers.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
European data format
home sleep apnea testing
intra-class correlation coefficient
- MEANdiff :
respiratory inductive plethysmography
Sleep Apnea Global Interdisciplinary Consortium
Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernandez-Blasco L, Monasterio C, Alonso A, Chiner E, Rubio M, Garcia-Ledesma E, Cacelo L, Carpizo R, Sacristan L, Salord N, Carrera M, Sancho-Chust JN, Embid C, Vazquez-Polo FJ, Negrin MA, Montserrat JM (2011) Effectiveness of home respiratory polygraphy for the diagnosis of sleep apnoea and hypopnoea syndrome. Thorax 66(7):567–573
Whittle AT, Finch SP, Mortimore IL, MacKay TW, Douglas NJ (1997) Use of home sleep studies for diagnosis of the sleep apnoea/hypopnoea syndrome. Thorax 52(12):1068–1073
Kuna ST, Badr MS, Kimoff RJ, Kushida C, Lee-Chiong T, Levy P, McNicholas WT, Strollo PJ, on behalf of the ATS/AASM/ACCP/ERS Committee on Ambulatory Management of Adults with OSA (2011) An official ATS/AASM/ACCP/ERS workshop report: research priorities in ambulatory management of adults with obstructive sleep apnea. Proc Am Thorac Soc 8(1):1–16
Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernández-Blasco L, Monasterio C, Alonso A, Chiner E, Zamorano J, Aizpuru F, Montserrat JM, and the Spanish Sleep Network (2011) Therapeutic decision-making for sleep apnea and hypopnea syndrome using home respiratory polygraphy: a large multicentric study. Am J Respir Crit Care Med 184(8):964–971
Flemons WW, Littner MR, Rowley JA, Gay P, Anderson WMD, Hudgel DW, McEvoy RD, Loube DI (2003) Home diagnosis of sleep apnea: a systematic review of the literature. An evidence review cosponsored by the American Academy of Sleep Medicine, the American College of Chest Physicians, and the American Thoracic Society. Chest 124(4):1543–1579
Berry RB, Budhiraja R, Gottlieb DJ, Gozal D, Iber C, Kapur VK, Marcus CL, Mehra R, Parthasarathy S, Quan SF, Redline S, Strohl KP, Davidson Ward SL, Tangredi MM, American Academy of Sleep Medicine (2012) Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events. Deliberations of the sleep apnea definitions task force of the American Academy of Sleep Medicine. J Clin Sleep Med 8(5):597–619
Magalang UJ, Arnardottir ES, Chen NH, Cistulli PA, Gíslason T, Lim D, Penzel T, Schwab R, Tufik S, Pack AI, SAGIC Investigators (2016) Agreement in the scoring of respiratory events among international sleep centers for home sleep testing. J Clin Sleep Med 12(1):71–77
Aurora RN, Swartz R, Punjabi NM (2015) Misclassification of OSA severity with automated scoring of home sleep recordings. Chest 147(3):719–727
Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernández-Blasco L, Monasterio C, Alonso-Fernandez A, Chiner E, Vázquez-Polo FJ, Montserrat JM, the Spanish Sleep Group (2013) Effectiveness of sequential automatic-manual home respiratory polygraphy scoring. Eur Respir J 41(4):879–887
Hedner J, Grote L, Bonsignore M, McNicholas W, Lavie P, Parati G, Sliwinski P, Barbe F, de Backer W, Escourrou P, Fietze I, Kvamme JA, Lombardi C, Marrone O, Masa JF, Montserrat JM, Penzel T, Pretl M, Riha R, Rodenstein D, Saaresranta T, Schulz R, Tkacova R, Varoneckas G, Vitols A, Vrints H, Zielinski J (2011) The European Sleep Apnoea Database (ESADA): report from 22 European sleep laboratories. Eur Respir J 38(3):635–642
McEvoy RD, Antic NA, Heeley E, Luo Y, Ou Q, Zhang X, Mediano O, Chen R, Drager LF, Liu Z, Chen G, du B, McArdle N, Mukherjee S, Tripathi M, Billot L, Li Q, Lorenzi-Filho G, Barbe F, Redline S, Wang J, Arima H, Neal B, White DP, Grunstein RR, Zhong N, Anderson CS, SAVE Investigators and Coordinators (2016) CPAP for prevention of cardiovascular events in obstructive sleep apnea. N Engl J Med 375(10):919–931
Kemp B, Varri A, Rosa AC, Nielsen KD, Gade J (1992) A simple format for exchange of digitized polygraphic recordings. Electroencephalogr Clin Neurophysiol 82(5):391–393
Berry RB, Brooks R, Gamaldo C, et al. The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications. Version 2.4. In: Darien. Version 2.4. In: Darien, IL: American Academy of Sleep Medicine; 2017
Magalang UJ, Chen N-H, Cistulli PA et al (2013) Agreement in the scoring of respiratory events and sleep among international sleep centers. Sleep 36(4):591–596
Dingli K, Coleman EL, Vennelle M, Finch SP, Wraith PK, Mackay TW, Douglas NJ (2003) Evaluation of a portable device for diagnosing the sleep apnoea/hypopnoea syndrome. Eur Respir J 21(2):253–259
Smith LA, Chong DW, Vennelle M, Denvir MA, Newby DE, Douglas NJ (2007) Diagnosis of sleep-disordered breathing in patients with chronic heart failure: evaluation of a portable limited sleep study system. J Sleep Res 16(4):428–435
Munro B. Statistical methods for health care research. 5th ed. Philadelphia: Lippincott Williams Wilkins; 2005
Cheng JW, Tsai WC, Yu TY, Huang KY Reproducibility of sonographic measurement of thickness and echogenicity of the plantar fascia. J Clin Ultrasound 40(1):14–19
Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Stat Methods Med Res 8(2):135–160
Pittman SD, MacDonald MM, Fogel RB et al (2004) Assessment of automated scoring of polysomnographic recordings in a population with suspected sleep-disordered breathing. Sleep 27(7):1394–1403
Malhotra A, Younes M, Kuna ST, Benca R, Kushida CA, Walsh J, Hanlon A, Staley B, Pack AI, Pien GW (2013) Performance of an automated polysomnography scoring system versus computer-assisted manual scoring. Sleep 36(4):573–582
Punjabi NM, Shifa N, Dorffner G, Patil S, Pien G, Aurora RN (2015) Computer-assisted automated scoring of polysomnograms using the Somnolyzer system. Sleep 38(10):1555–1566
Thurnheer R, Xie X, Bloch KE (2001) Accuracy of nasal cannula pressure recordings for assessment of ventilation during sleep. Am J Respir Crit Care Med 164(10 Pt 1):1914–1919
Farre R, Rigau J, Montserrat JM, Ballester E, Navajas D (2001) Relevance of linearizing nasal prongs for assessing hypopneas and flow limitation during sleep. Am J Respir Crit Care Med 163(2):494–497
The authors would like to thank the following individuals who helped in this project: Mohammad Ahmadi, Alexander Blau, Petra Cornell, Silverio Garbuio, Su-Lan Liu, João Reinfelderon, Beth Staley, Magdalena Ósk Sigurgunnarsdóttir, and Sandra Zimmermann.
Supported by NHLBI award P01 HL094307 (AIP), Conselho Nacional de Desenvolvimento Cientffico e Tecnologico (CNPq) grant 309336/2017–1 (LRB), Conselho Nacional de Desenvolvimento Cientffico e Tecno1ogico (CNPq), Grant 401569/2016–0 (LRB), and Award grant number UL1TR001070 from the National Center for Advancing Translational Sciences. The sponsor had no role in the design or conduct of the research.
Conflict of interest
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Formal consent is not required for this type of study.
Electronic supplementary material
About this article
Cite this article
Magalang, U.J., Johns, J.N., Wood, K.A. et al. Home sleep apnea testing: comparison of manual and automated scoring across international sleep centers. Sleep Breath 23, 25–31 (2019). https://doi.org/10.1007/s11325-018-1715-6
- Sleep apnea
- Computer-assisted diagnosis