Sleep and Breathing

, Volume 15, Issue 3, pp 431–437

Diagnostic characteristics of clinical prediction models for obstructive sleep apnea in different clinic populations

  • See-Meng Khoo
  • Hze-Khoong Poh
  • Yiong-Huak Chan
  • Wang-Jee Ngerng
  • Dong-Xia Shi
  • T. K. Lim
Original Article

Abstract

Purpose

As predictive factors and their diagnostic values are affected by the characteristics of the population studied, clinical prediction model for obstructive sleep apnea (OSA) may exhibit different diagnostic characteristics in different populations. We aimed to compare the diagnostic characteristics of clinical prediction models developed in two different populations.

Methods

One hundred seventeen consecutive clinic patients (group 1) were evaluated to develop a clinical prediction model for OSA (local model). The diagnostic characteristics of this local model were compared with those of a foreign model by applying both models to another group of 52 patients who were referred to the same clinic (group 2). All patients underwent overnight polysomnography.

Results

The local model had an area under receiver operator characteristics curve of 79%. A cutoff of 0.6 was associated with sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 77.9%, 72.5%, 84.5%, and 63.0%, respectively. The overall diagnostic accuracy, sensitivity, specificity, PPV, and NPV of the local model using data from patients in group 2 were 69.0%, 78.1%, 45.0%, 69.4%, and 56.3%, respectively. The foreign model had an overall diagnostic accuracy of 64.0% when applied to data from patients in group 2. At the optimal cutoff of 17, the foreign model was associated with sensitivity of 38.2%, specificity of 83.3%, NPV of 41.7% and PPV of 81.3%.

Conclusions

Clinical prediction model for OSA derived from a foreign population exhibits markedly different diagnostic characteristics from one that is developed locally, even though the overall accuracy is similar. Our findings challenge the predictive usefulness and the external validity of clinical prediction models.

Keywords

Clinical prediction models Obstructive sleep apnea Diagnostic characteristics Different populations 

Abbreviations

AHI

Apnea–hypopnea index

BMI

Body mass index

CPAP

Continuous positive airway pressure

NPV

Negative predictive value

OSA

Obstructive sleep apnea

PPV

Positive predictive value

PSG

Polysomnography

ROC

Receiver operating characteristic

References

  1. 1.
    Lam B, Lam DC, Ip MS (2007) Obstructive sleep apnoea in Asia. Int J Tuberc Lung Dis 11:2–11PubMedGoogle Scholar
  2. 2.
    Mulgrew AT, Fox N, Ayas NT, Ryan CF (2007) Diagnosis and initial management of obstructive sleep apnea without polysomnography: a randomized validation study. Ann Intern Med 146:157–166PubMedGoogle Scholar
  3. 3.
    Flemons WW, Whitelaw WA, Brant R, Remmers JE (1994) Likelihood ratios for a sleep apnea clinical prediction rule. Am J Respir Crit Care Med 150:1279–1285PubMedGoogle Scholar
  4. 4.
    Gurubhagavatula I, Maislin G, Pack AI (2001) An algorithm to stratify sleep apnea risk in a sleep disorders clinic population. Am J Respir Crit Care Med 164:1904–1909PubMedGoogle Scholar
  5. 5.
    Kushida CA, Efron B, Guilleminault C (1997) A predictive morphometric model for the obstructive sleep apnea syndrome. Ann Intern Med 127:581–587PubMedGoogle Scholar
  6. 6.
    Lam B, Ip MS, Tench E, Ryan CF (2005) Craniofacial profile in Asian and white subjects with obstructive sleep apnoea. Thorax 60:504–510PubMedCrossRefGoogle Scholar
  7. 7.
    Maislin G, Pack AI, Kribbs NB et al (1995) A survey screen for prediction of apnea. Sleep 18:158–166PubMedGoogle Scholar
  8. 8.
    Netzer NC, Stoohs RA, Netzer CM, Clark K, Strohl KP (1999) Using the Berlin questionnaire to identify patients at risk for the sleep apnea syndrome. Ann Intern Med 131:485–491PubMedGoogle Scholar
  9. 9.
    Rowley JA, Aboussouan LS, Badr MS (2000) The use of clinical prediction formulas in the evaluation of obstructive sleep apnea. Sleep 23:929–938PubMedGoogle Scholar
  10. 10.
    Sharma SK, Malik V, Vasudev C et al (2006) Prediction of obstructive sleep apnea in patients presenting to a tertiary care center. Sleep Breath 10:147–154PubMedCrossRefGoogle Scholar
  11. 11.
    Santaolalla Montoya F, Iriondo Bedialauneta JR, Aguirre Larracoechea U, Martinez Ibarguen A, Sanchez Del Rey A, Sanchez Fernandez JM (2007) The predictive value of clinical and epidemiological parameters in the identification of patients with obstructive sleep apnoea (OSA): a clinical prediction algorithm in the evaluation of OSA. Eur Arch Otorhinolaryngol 264:637–643PubMedCrossRefGoogle Scholar
  12. 12.
    Ramachandran SK, Josephs LA (2009) A meta-analysis of clinical screening tests for obstructive sleep apnea. Anesthesiology 110:928–939PubMedCrossRefGoogle Scholar
  13. 13.
    Li KK, Kushida C, Powell NB, Riley RW, Guilleminault C (2000) Obstructive sleep apnea syndrome: a comparison between Far-East Asian and white men. Laryngoscope 110:1689–1693PubMedCrossRefGoogle Scholar
  14. 14.
    Rechtschaffen A, Kales A (1968) A manual of standardized terminology, techniques and scoring systems for sleep stages of human subjects. National Institute of Health, Washington DCGoogle Scholar
  15. 15.
    Hanley JA, McNeil BJ (1983) A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148:839–843PubMedGoogle Scholar
  16. 16.
    Viner S, Szalai JP, Hoffstein V (1991) Are history and physical examination a good screening test for sleep apnea? Ann Intern Med 115:356–359PubMedGoogle Scholar
  17. 17.
    Crocker BD, Olson LG, Saunders NA et al (1990) Estimation of the probability of disturbed breathing during sleep before a sleep study. Am Rev Respir Dis 142:14–18PubMedGoogle Scholar
  18. 18.
    Whitty C, Jones RJ (1992) A comparison of prospective and retrospective diary methods of assessing alcohol use among university undergraduates. J Public Health Med 14:264–270PubMedGoogle Scholar
  19. 19.
    Barakzai MD, Gregory J, Fraser D (2007) The effect of culture on symptom reporting: Hispanics and irritable bowel syndrome. J Am Acad Nurse Pract 19:261–267PubMedCrossRefGoogle Scholar
  20. 20.
    Crawford SL, Avis NE, Gold E et al (2008) Sensitivity and specificity of recalled vasomotor symptoms in a multiethnic cohort. Am J Epidemiol 168:1452–1459PubMedCrossRefGoogle Scholar
  21. 21.
    Irwin ML, Ainsworth BE, Conway JM (2001) Estimation of energy expenditure from physical activity measures: determinants of accuracy. Obes Res 9:517–525PubMedCrossRefGoogle Scholar
  22. 22.
    Hilton ME (1989) A comparison of a prospective diary and two summary recall techniques for recording alcohol consumption. Br J Addict 84:1085–1092PubMedCrossRefGoogle Scholar
  23. 23.
    Leigh BC, Gillmore MR, Morrison DM (1998) Comparison of diary and retrospective measures for recording alcohol consumption and sexual activity. J Clin Epidemiol 51:119–127PubMedCrossRefGoogle Scholar
  24. 24.
    Townshend JM, Duka T (2002) Patterns of alcohol drinking in a population of young social drinkers: a comparison of questionnaire and diary measures. Alcohol Alcohol 37:187–192PubMedGoogle Scholar
  25. 25.
    Udwadia ZF, Doshi AV, Lonkar SG, Singh CI (2004) Prevalence of sleep-disordered breathing and sleep apnea in middle-aged urban Indian men. Am J Respir Crit Care Med 169:168–173PubMedCrossRefGoogle Scholar
  26. 26.
    Khoo SM, Tan WC, Ng TP, Ho CH (2004) Risk factors associated with habitual snoring and sleep-disordered breathing in a multi-ethnic Asian population: a population-based study. Respir Med 98:557–566PubMedCrossRefGoogle Scholar
  27. 27.
    Roche N, Herer B, Roig C, Huchon G (2002) Prospective testing of two models based on clinical and oximetric variables for prediction of obstructive sleep apnea. Chest 121:747–752PubMedCrossRefGoogle Scholar
  28. 28.
    Kapuniai LE, Andrew DJ, Crowell DH, Pearce JW (1988) Identifying sleep apnea from self-reports. Sleep 11:430–436PubMedGoogle Scholar
  29. 29.
    Martinez-Garcia MA, Soler-Cataluna JJ, Ejarque-Martinez L et al (2009) Continuous positive airway pressure treatment reduces mortality in patients with ischemic stroke and obstructive sleep apnea: a 5-year follow-up study. Am J Respir Crit Care Med 180:36–41PubMedCrossRefGoogle Scholar
  30. 30.
    Ancoli-Israel S, Kripke DF, Klauber MR, Mason WJ, Fell R, Kaplan O (1991) Sleep-disordered breathing in community-dwelling elderly. Sleep 14:486–495PubMedGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • See-Meng Khoo
    • 1
  • Hze-Khoong Poh
    • 2
  • Yiong-Huak Chan
    • 3
  • Wang-Jee Ngerng
    • 1
  • Dong-Xia Shi
    • 1
  • T. K. Lim
    • 1
  1. 1.Division of Respiratory and Critical Care Medicine, University Medicine ClusterNational University Health SystemSingaporeSingapore
  2. 2.Department of Oral and Maxillofacial Surgery, Faculty of DentistryNational University of SingaporeSingaporeSingapore
  3. 3.Biostatistics Unit, Yong Loo Lin School of MedicineNational University of SingaporeSingaporeSingapore

Personalised recommendations