Meta-analysis of Calibration, Discrimination, and Stratum-Specific Likelihood Ratios for the CRB-65 Score
The CRB-65 score is recommended as a decision support tool to help identify patients with community-acquired pneumonia (CAP) who can safely be treated as outpatients.
To perform an updated meta-analysis of the accuracy, discrimination, and calibration of the CRB-65 score using a novel approach to calculation of stratum-specific likelihood ratios.
Meta-analysis of accuracy, discrimination, and calibration.
We searched PubMed, Google, previous systematic reviews, and reference lists of included studies. Data was abstracted and quality assessed in parallel by two investigators. The quality assessment used an adaptation of the TRIPOD and PROBAST criteria. Measures of discrimination, calibration, and stratum-specific likelihood ratios are reported.
Twenty-nine studies met our inclusion criteria and provided usable data. Most studies were set in Europe, none in North America, and 12 were judged to be at low risk of bias. The pooled estimate of area under the receiver operating characteristic curve was 0.74 (95% CI 0.71–0.77) for all studies. Calibration was good although there was significant heterogeneity; the pooled estimate of the ratio of observed to expected mortality for all studies was 1.04 (95% CI 0.91–1.19). The corresponding values for studies at low risk of bias where patients could be treated as outpatients or inpatients were 0.76 (0.70–0.81) and 0.88 (0.69–1.13). Summary estimates of stratum-specific likelihood ratios for all studies were 0.19 for the low-risk group, 1.1 for the moderate-risk group, and 4.5 for the high-risk group, and 0.13, 1.3, and 5.6 for studies at low risk of bias where patients could be treated as outpatients or inpatients.
The CRB-65 is useful for identifying low-risk patients for outpatient therapy. Given a 4% overall mortality risk, patients classified as low risk by the CRB-65 had an outpatient mortality risk of no more than 0.5%.
KEY WORDScommunity-acquired pneumonia risk prediction models clinical decision rules CRB-65 adults meta-analysis
Dr. Ebell’s collaboration with Drs. Fahey and Walsh was supported by a 2018/2019 Fulbright Teaching/Research award.
Role of Investigators
Mark Ebell and Tom Fahey were responsible for overall design and conduct of the study. Mark Ebell participated in data abstraction, analysis, and primary authorship of the manuscript. Maggie Kearney, Mary Walsh, and Christian Marchello participated in data abstraction. Mary Walsh performed the random effects meta-analysis of observed/expected ratio and AUC, while Mark Ebell performed meta-analysis of stratum-specific likelihood ratios. Tom Fahey also participated in writing the manuscript and in the analysis, and all authors reviewed and approved the final manuscript.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they do not have a conflict of interest.
- 6.Debray TPA, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ. 2017;356.Google Scholar
- 7.Buising KL, et al. Identifying severe community-acquired pneumonia in the emergency department: a simple clinical prediction tool. Emerg Med Australas. 2007;19(5):418–26.Google Scholar
- 10.Snell KI, et al. Meta-analysis of prediction model performance across multiple studies: Which scale helps ensure between-study normality for the C-statistic and calibration measures? Stat Methods Med Res. 2017;962280217705678.Google Scholar
- 21.British Thoracic S, et al. Severity assessment criteria recommended by the British Thoracic Society (BTS) for community-acquired pneumonia (CAP) and older patients. Should SOAR (systolic blood pressure, oxygenation, age and respiratory rate) criteria be used in older people? A compilation study of two prospective cohorts. Age Ageing. 2006;35(3):286–91.CrossRefGoogle Scholar
- 24.Zuberi FF, Khan JA. Prospective comparison of prediction rules of mortality risk for CAP in a developing country. Int J Tuberc Lung Dis. 2008;12(4):447–52.Google Scholar