Sleep and Breathing

, Volume 23, Issue 1, pp 25–31 | Cite as

Home sleep apnea testing: comparison of manual and automated scoring across international sleep centers

  • Ulysses J. MagalangEmail author
  • Jennica N. Johns
  • Katherine A. Wood
  • Jesse W. Mindel
  • Diane C. Lim
  • Lia R. Bittencourt
  • Ning-Hung Chen
  • Peter A. Cistulli
  • Thorarinn Gíslason
  • Erna S. Arnardottir
  • Thomas Penzel
  • Sergio Tufik
  • Allan I. Pack
Sleep Breathing Physiology and Disorders • Original Article



To determine the agreement between the manual scoring of home sleep apnea tests (HSATs) by international sleep technologists and automated scoring systems.


Fifteen HSATs, previously recorded using a type 3 monitor, were saved in European Data Format. The studies were scored by nine experienced technologists from the sleep centers of the Sleep Apnea Global Interdisciplinary Consortium (SAGIC) using the locally available software. Each study was scored separately by human scorers using the nasal pressure (NP), flow derived from the NP signal (transformed NP), or respiratory inductive plethysmography (RIP) flow. The same procedure was followed using two automated scoring systems: Remlogic (RLG) and Noxturnal (NOX).


The intra-class correlation coefficients (ICCs) of the apnea-hypopnea index (AHI) scoring using the NP, transformed NP, and RIP flow were 0.96 [95% CI 0.93–0.99], 0.98 [0.96–0.99], and 0.97 [0.95–0.99], respectively. Using the NP signal, the mean differences in AHI between the average of the manual scoring and the automated systems were − 0.9 ± 3.1/h (AHIRLG vs AHIMANUAL) and − 1.3 ± 2.6/h (AHINOX vs AHIMANUAL). Using the transformed NP, the mean differences in AHI were − 1.9 ± 3.3/h (AHIRLG vs AHIMANUAL) and 1.6 ± 3.0/h (AHINOX vs AHIMANUAL). Using the RIP flow, the mean differences in AHI were − 2.7 ± 4.5/h (AHIRLG vs AHIMANUAL) and 2.3 ± 3.4/h (AHINOX vs AHIMANUAL).


There is very strong agreement in the scoring of the AHI for HSATs between the automated systems and experienced international technologists. Automated scoring of HSATs using commercially available software may be useful to standardize scoring in future endeavors involving international sleep centers.


Sleep apnea Automation Computer-assisted diagnosis 



apnea-hypopnea Index


European data format


home sleep apnea testing


intra-class correlation coefficient


mean difference




nasal pressure




respiratory inductive plethysmography




Sleep Apnea Global Interdisciplinary Consortium



The authors would like to thank the following individuals who helped in this project: Mohammad Ahmadi, Alexander Blau, Petra Cornell, Silverio Garbuio, Su-Lan Liu, João Reinfelderon, Beth Staley, Magdalena Ósk Sigurgunnarsdóttir, and Sandra Zimmermann.


Supported by NHLBI award P01 HL094307 (AIP), Conselho Nacional de Desenvolvimento Cientffico e Tecnologico (CNPq) grant 309336/2017–1 (LRB), Conselho Nacional de Desenvolvimento Cientffico e Tecno1ogico (CNPq), Grant 401569/2016–0 (LRB), and Award grant number UL1TR001070 from the National Center for Advancing Translational Sciences. The sponsor had no role in the design or conduct of the research.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Formal consent is not required for this type of study.

Supplementary material

11325_2018_1715_MOESM1_ESM.docx (14 kb)
ESM 1 (DOCX 14 kb)


  1. 1.
    Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernandez-Blasco L, Monasterio C, Alonso A, Chiner E, Rubio M, Garcia-Ledesma E, Cacelo L, Carpizo R, Sacristan L, Salord N, Carrera M, Sancho-Chust JN, Embid C, Vazquez-Polo FJ, Negrin MA, Montserrat JM (2011) Effectiveness of home respiratory polygraphy for the diagnosis of sleep apnoea and hypopnoea syndrome. Thorax 66(7):567–573CrossRefGoogle Scholar
  2. 2.
    Whittle AT, Finch SP, Mortimore IL, MacKay TW, Douglas NJ (1997) Use of home sleep studies for diagnosis of the sleep apnoea/hypopnoea syndrome. Thorax 52(12):1068–1073CrossRefGoogle Scholar
  3. 3.
    Kuna ST, Badr MS, Kimoff RJ, Kushida C, Lee-Chiong T, Levy P, McNicholas WT, Strollo PJ, on behalf of the ATS/AASM/ACCP/ERS Committee on Ambulatory Management of Adults with OSA (2011) An official ATS/AASM/ACCP/ERS workshop report: research priorities in ambulatory management of adults with obstructive sleep apnea. Proc Am Thorac Soc 8(1):1–16CrossRefGoogle Scholar
  4. 4.
    Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernández-Blasco L, Monasterio C, Alonso A, Chiner E, Zamorano J, Aizpuru F, Montserrat JM, and the Spanish Sleep Network (2011) Therapeutic decision-making for sleep apnea and hypopnea syndrome using home respiratory polygraphy: a large multicentric study. Am J Respir Crit Care Med 184(8):964–971CrossRefGoogle Scholar
  5. 5.
    Flemons WW, Littner MR, Rowley JA, Gay P, Anderson WMD, Hudgel DW, McEvoy RD, Loube DI (2003) Home diagnosis of sleep apnea: a systematic review of the literature. An evidence review cosponsored by the American Academy of Sleep Medicine, the American College of Chest Physicians, and the American Thoracic Society. Chest 124(4):1543–1579CrossRefGoogle Scholar
  6. 6.
    Berry RB, Budhiraja R, Gottlieb DJ, Gozal D, Iber C, Kapur VK, Marcus CL, Mehra R, Parthasarathy S, Quan SF, Redline S, Strohl KP, Davidson Ward SL, Tangredi MM, American Academy of Sleep Medicine (2012) Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events. Deliberations of the sleep apnea definitions task force of the American Academy of Sleep Medicine. J Clin Sleep Med 8(5):597–619Google Scholar
  7. 7.
    Magalang UJ, Arnardottir ES, Chen NH, Cistulli PA, Gíslason T, Lim D, Penzel T, Schwab R, Tufik S, Pack AI, SAGIC Investigators (2016) Agreement in the scoring of respiratory events among international sleep centers for home sleep testing. J Clin Sleep Med 12(1):71–77CrossRefGoogle Scholar
  8. 8.
    Aurora RN, Swartz R, Punjabi NM (2015) Misclassification of OSA severity with automated scoring of home sleep recordings. Chest 147(3):719–727CrossRefGoogle Scholar
  9. 9.
    Masa JF, Corral J, Pereira R, Duran-Cantolla J, Cabello M, Hernández-Blasco L, Monasterio C, Alonso-Fernandez A, Chiner E, Vázquez-Polo FJ, Montserrat JM, the Spanish Sleep Group (2013) Effectiveness of sequential automatic-manual home respiratory polygraphy scoring. Eur Respir J 41(4):879–887CrossRefGoogle Scholar
  10. 10.
    Hedner J, Grote L, Bonsignore M, McNicholas W, Lavie P, Parati G, Sliwinski P, Barbe F, de Backer W, Escourrou P, Fietze I, Kvamme JA, Lombardi C, Marrone O, Masa JF, Montserrat JM, Penzel T, Pretl M, Riha R, Rodenstein D, Saaresranta T, Schulz R, Tkacova R, Varoneckas G, Vitols A, Vrints H, Zielinski J (2011) The European Sleep Apnoea Database (ESADA): report from 22 European sleep laboratories. Eur Respir J 38(3):635–642CrossRefGoogle Scholar
  11. 11.
    McEvoy RD, Antic NA, Heeley E, Luo Y, Ou Q, Zhang X, Mediano O, Chen R, Drager LF, Liu Z, Chen G, du B, McArdle N, Mukherjee S, Tripathi M, Billot L, Li Q, Lorenzi-Filho G, Barbe F, Redline S, Wang J, Arima H, Neal B, White DP, Grunstein RR, Zhong N, Anderson CS, SAVE Investigators and Coordinators (2016) CPAP for prevention of cardiovascular events in obstructive sleep apnea. N Engl J Med 375(10):919–931CrossRefGoogle Scholar
  12. 12.
    Kemp B, Varri A, Rosa AC, Nielsen KD, Gade J (1992) A simple format for exchange of digitized polygraphic recordings. Electroencephalogr Clin Neurophysiol 82(5):391–393CrossRefGoogle Scholar
  13. 13.
    Berry RB, Brooks R, Gamaldo C, et al. The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications. Version 2.4. In: Darien. Version 2.4. In: Darien, IL: American Academy of Sleep Medicine; 2017Google Scholar
  14. 14.
    Magalang UJ, Chen N-H, Cistulli PA et al (2013) Agreement in the scoring of respiratory events and sleep among international sleep centers. Sleep 36(4):591–596CrossRefGoogle Scholar
  15. 15.
    Dingli K, Coleman EL, Vennelle M, Finch SP, Wraith PK, Mackay TW, Douglas NJ (2003) Evaluation of a portable device for diagnosing the sleep apnoea/hypopnoea syndrome. Eur Respir J 21(2):253–259CrossRefGoogle Scholar
  16. 16.
    Smith LA, Chong DW, Vennelle M, Denvir MA, Newby DE, Douglas NJ (2007) Diagnosis of sleep-disordered breathing in patients with chronic heart failure: evaluation of a portable limited sleep study system. J Sleep Res 16(4):428–435CrossRefGoogle Scholar
  17. 17.
    Munro B. Statistical methods for health care research. 5th ed. Philadelphia: Lippincott Williams Wilkins; 2005Google Scholar
  18. 18.
    Cheng JW, Tsai WC, Yu TY, Huang KY Reproducibility of sonographic measurement of thickness and echogenicity of the plantar fascia. J Clin Ultrasound 40(1):14–19Google Scholar
  19. 19.
    Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Stat Methods Med Res 8(2):135–160CrossRefGoogle Scholar
  20. 20.
    Pittman SD, MacDonald MM, Fogel RB et al (2004) Assessment of automated scoring of polysomnographic recordings in a population with suspected sleep-disordered breathing. Sleep 27(7):1394–1403CrossRefGoogle Scholar
  21. 21.
    Malhotra A, Younes M, Kuna ST, Benca R, Kushida CA, Walsh J, Hanlon A, Staley B, Pack AI, Pien GW (2013) Performance of an automated polysomnography scoring system versus computer-assisted manual scoring. Sleep 36(4):573–582CrossRefGoogle Scholar
  22. 22.
    Punjabi NM, Shifa N, Dorffner G, Patil S, Pien G, Aurora RN (2015) Computer-assisted automated scoring of polysomnograms using the Somnolyzer system. Sleep 38(10):1555–1566CrossRefGoogle Scholar
  23. 23.
    Thurnheer R, Xie X, Bloch KE (2001) Accuracy of nasal cannula pressure recordings for assessment of ventilation during sleep. Am J Respir Crit Care Med 164(10 Pt 1):1914–1919CrossRefGoogle Scholar
  24. 24.
    Farre R, Rigau J, Montserrat JM, Ballester E, Navajas D (2001) Relevance of linearizing nasal prongs for assessing hypopneas and flow limitation during sleep. Am J Respir Crit Care Med 163(2):494–497CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Ulysses J. Magalang
    • 1
    • 2
    Email author
  • Jennica N. Johns
    • 1
  • Katherine A. Wood
    • 1
  • Jesse W. Mindel
    • 1
  • Diane C. Lim
    • 3
  • Lia R. Bittencourt
    • 4
  • Ning-Hung Chen
    • 5
  • Peter A. Cistulli
    • 6
    • 7
  • Thorarinn Gíslason
    • 8
    • 9
  • Erna S. Arnardottir
    • 8
    • 9
  • Thomas Penzel
    • 10
  • Sergio Tufik
    • 4
  • Allan I. Pack
    • 3
  1. 1.Division of Pulmonary, Allergy, Critical Care, and Sleep MedicineThe Ohio State University Wexner Medical Center, 201 Davis Heart and Lung Research InstituteColumbusUSA
  2. 2.Neuroscience Research InstituteThe Ohio State University Wexner Medical CenterColumbusUSA
  3. 3.Center for Sleep and Circadian Neurobiology, Division of Sleep MedicineUniversity of Pennsylvania Perelman School of MedicinePhiladelphiaUSA
  4. 4.Departamento de PsicobiologiaUniversidade Federal de São PauloSão PauloBrazil
  5. 5.Division of Pulmonary, Critical Care, and Sleep MedicineChang Gung Memorial HospitalTaoyuanTaiwan
  6. 6.Charles Perkins CentreUniversity of SydneyCamperdownAustralia
  7. 7.Department of Respiratory and Sleep MedicineRoyal North Shore HospitalSydneyAustralia
  8. 8.Department of Sleep MedicineLandspitali University HospitalReykjavikIceland
  9. 9.Medical FacultyUniversity of IcelandReykjavikIceland
  10. 10.Interdisciplinary Center of Sleep MedicineCharité University HospitalBerlinGermany

Personalised recommendations