Home sleep apnea testing: comparison of manual and automated scoring across international sleep centers
To determine the agreement between the manual scoring of home sleep apnea tests (HSATs) by international sleep technologists and automated scoring systems.
Fifteen HSATs, previously recorded using a type 3 monitor, were saved in European Data Format. The studies were scored by nine experienced technologists from the sleep centers of the Sleep Apnea Global Interdisciplinary Consortium (SAGIC) using the locally available software. Each study was scored separately by human scorers using the nasal pressure (NP), flow derived from the NP signal (transformed NP), or respiratory inductive plethysmography (RIP) flow. The same procedure was followed using two automated scoring systems: Remlogic (RLG) and Noxturnal (NOX).
The intra-class correlation coefficients (ICCs) of the apnea-hypopnea index (AHI) scoring using the NP, transformed NP, and RIP flow were 0.96 [95% CI 0.93–0.99], 0.98 [0.96–0.99], and 0.97 [0.95–0.99], respectively. Using the NP signal, the mean differences in AHI between the average of the manual scoring and the automated systems were − 0.9 ± 3.1/h (AHIRLG vs AHIMANUAL) and − 1.3 ± 2.6/h (AHINOX vs AHIMANUAL). Using the transformed NP, the mean differences in AHI were − 1.9 ± 3.3/h (AHIRLG vs AHIMANUAL) and 1.6 ± 3.0/h (AHINOX vs AHIMANUAL). Using the RIP flow, the mean differences in AHI were − 2.7 ± 4.5/h (AHIRLG vs AHIMANUAL) and 2.3 ± 3.4/h (AHINOX vs AHIMANUAL).
There is very strong agreement in the scoring of the AHI for HSATs between the automated systems and experienced international technologists. Automated scoring of HSATs using commercially available software may be useful to standardize scoring in future endeavors involving international sleep centers.
KeywordsSleep apnea Automation Computer-assisted diagnosis
European data format
home sleep apnea testing
intra-class correlation coefficient
respiratory inductive plethysmography
Sleep Apnea Global Interdisciplinary Consortium
The authors would like to thank the following individuals who helped in this project: Mohammad Ahmadi, Alexander Blau, Petra Cornell, Silverio Garbuio, Su-Lan Liu, João Reinfelderon, Beth Staley, Magdalena Ósk Sigurgunnarsdóttir, and Sandra Zimmermann.
Supported by NHLBI award P01 HL094307 (AIP), Conselho Nacional de Desenvolvimento Cientffico e Tecnologico (CNPq) grant 309336/2017–1 (LRB), Conselho Nacional de Desenvolvimento Cientffico e Tecno1ogico (CNPq), Grant 401569/2016–0 (LRB), and Award grant number UL1TR001070 from the National Center for Advancing Translational Sciences. The sponsor had no role in the design or conduct of the research.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Formal consent is not required for this type of study.
- 1.Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernandez-Blasco L, Monasterio C, Alonso A, Chiner E, Rubio M, Garcia-Ledesma E, Cacelo L, Carpizo R, Sacristan L, Salord N, Carrera M, Sancho-Chust JN, Embid C, Vazquez-Polo FJ, Negrin MA, Montserrat JM (2011) Effectiveness of home respiratory polygraphy for the diagnosis of sleep apnoea and hypopnoea syndrome. Thorax 66(7):567–573CrossRefGoogle Scholar
- 3.Kuna ST, Badr MS, Kimoff RJ, Kushida C, Lee-Chiong T, Levy P, McNicholas WT, Strollo PJ, on behalf of the ATS/AASM/ACCP/ERS Committee on Ambulatory Management of Adults with OSA (2011) An official ATS/AASM/ACCP/ERS workshop report: research priorities in ambulatory management of adults with obstructive sleep apnea. Proc Am Thorac Soc 8(1):1–16CrossRefGoogle Scholar
- 4.Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernández-Blasco L, Monasterio C, Alonso A, Chiner E, Zamorano J, Aizpuru F, Montserrat JM, and the Spanish Sleep Network (2011) Therapeutic decision-making for sleep apnea and hypopnea syndrome using home respiratory polygraphy: a large multicentric study. Am J Respir Crit Care Med 184(8):964–971CrossRefGoogle Scholar
- 5.Flemons WW, Littner MR, Rowley JA, Gay P, Anderson WMD, Hudgel DW, McEvoy RD, Loube DI (2003) Home diagnosis of sleep apnea: a systematic review of the literature. An evidence review cosponsored by the American Academy of Sleep Medicine, the American College of Chest Physicians, and the American Thoracic Society. Chest 124(4):1543–1579CrossRefGoogle Scholar
- 6.Berry RB, Budhiraja R, Gottlieb DJ, Gozal D, Iber C, Kapur VK, Marcus CL, Mehra R, Parthasarathy S, Quan SF, Redline S, Strohl KP, Davidson Ward SL, Tangredi MM, American Academy of Sleep Medicine (2012) Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events. Deliberations of the sleep apnea definitions task force of the American Academy of Sleep Medicine. J Clin Sleep Med 8(5):597–619Google Scholar
- 9.Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernández-Blasco L, Monasterio C, Alonso-Fernandez A, Chiner E, Vázquez-Polo FJ, Montserrat JM, the Spanish Sleep Group (2013) Effectiveness of sequential automatic-manual home respiratory polygraphy scoring. Eur Respir J 41(4):879–887CrossRefGoogle Scholar
- 10.Hedner J, Grote L, Bonsignore M, McNicholas W, Lavie P, Parati G, Sliwinski P, Barbe F, de Backer W, Escourrou P, Fietze I, Kvamme JA, Lombardi C, Marrone O, Masa JF, Montserrat JM, Penzel T, Pretl M, Riha R, Rodenstein D, Saaresranta T, Schulz R, Tkacova R, Varoneckas G, Vitols A, Vrints H, Zielinski J (2011) The European Sleep Apnoea Database (ESADA): report from 22 European sleep laboratories. Eur Respir J 38(3):635–642CrossRefGoogle Scholar
- 11.McEvoy RD, Antic NA, Heeley E, Luo Y, Ou Q, Zhang X, Mediano O, Chen R, Drager LF, Liu Z, Chen G, du B, McArdle N, Mukherjee S, Tripathi M, Billot L, Li Q, Lorenzi-Filho G, Barbe F, Redline S, Wang J, Arima H, Neal B, White DP, Grunstein RR, Zhong N, Anderson CS, SAVE Investigators and Coordinators (2016) CPAP for prevention of cardiovascular events in obstructive sleep apnea. N Engl J Med 375(10):919–931CrossRefGoogle Scholar
- 13.Berry RB, Brooks R, Gamaldo C, et al. The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications. Version 2.4. In: Darien. Version 2.4. In: Darien, IL: American Academy of Sleep Medicine; 2017Google Scholar
- 17.Munro B. Statistical methods for health care research. 5th ed. Philadelphia: Lippincott Williams Wilkins; 2005Google Scholar
- 18.Cheng JW, Tsai WC, Yu TY, Huang KY Reproducibility of sonographic measurement of thickness and echogenicity of the plantar fascia. J Clin Ultrasound 40(1):14–19Google Scholar