Validation of the All Patient Refined Diagnosis Related Group (APR-DRG) Risk of Mortality and Severity of Illness Modifiers as a Measure of Perioperative Risk

  • Patrick J. McCormick
  • Hung-mo Lin
  • Stacie G. Deiner
  • Matthew A. Levin
Systems-Level Quality Improvement
Part of the following topical collections:
  1. Systems-Level Quality Improvement


The All Patient Refined Diagnosis Related Group (APR-DRG) is an inpatient visit classification system that assigns a diagnostic related group, a Risk of Mortality (ROM) subclass and a Severity of Illness (SOI) subclass. While extensively used for cost adjustment, no study has compared the APR-DRG subclass modifiers to the popular Charlson Comorbidity Index as a measure of comorbidity severity in models for perioperative in-hospital mortality. In this study we attempt to validate the use of these subclasses to predict mortality in a cohort of surgical patients. We analyzed all adult (age over 18 years) inpatient non-cardiac surgery at our institution between December 2005 and July 2013. After exclusions, we split the cohort into training and validation sets. We created prediction models of inpatient mortality using the Charlson Comorbidity Index, ROM only, SOI only, and ROM with SOI. Models were compared by receiver-operator characteristic (ROC) curve, area under the ROC curve (AUC), and Brier score. After exclusions, we analyzed 63,681 patient-visits. Overall in-hospital mortality was 1.3%. The median number of ICD-9-CM diagnosis codes was 6 (Q1-Q3 4–10). The median Charlson Comorbidity Index was 0 (Q1-Q3 0–2). When the model was applied to the validation set, the c-statistic for Charlson was 0.865, c-statistic for ROM was 0.975, and for ROM and SOI combined the c-statistic was 0.977. The scaled Brier score for Charlson was 0.044, Brier for ROM only was 0.230, and Brier for ROM and SOI was 0.257. The APR-DRG ROM or SOI subclasses are better predictors than the Charlson Comorbidity Index of in-hospital mortality among surgical patients.


Risk adjustment Case mix Diagnostic related group Comorbidity Perioperative mortality 


Author Contributions

Patrick J. McCormick has seen the study data, reviewed the analysis of the data, and approved the final manuscript. Hung-mo Lin has seen the study data, reviewed the analysis of the data, and approved the final manuscript. Stacie G. Deiner has seen the study data, reviewed the analysis of the data, and approved the final manuscript. Matthew A. Levin has seen the study data, reviewed the analysis of the data, and approved the final manuscript.


Departmental sources only.

Compliance with Ethical Standards

Conflict of Interest

Patrick J. McCormick reported no conflicts of interest.

Hung-mo Lin reported no conflicts of interest.

Stacie G. Deiner reported no conflicts of interest.

Matthew A. Levin reported no conflicts of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was waived by the Mount Sinai Institutional Review Board ( due to the retrospective nature of the study.


  1. 1.
    Charlson, M. E., Pompei, P., Ales, K. L. et al., A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 40:373–383, 1987.CrossRefPubMedGoogle Scholar
  2. 2.
    St-Louis, E., Iqbal, S., Feldman, L. S. et al., Using the age-adjusted Charlson comorbidity index to predict outcomes in emergency general surgery. J Trauma Acute Care Surg 78:318–323, 2015. Scholar
  3. 3.
    Kirkland, L. L., Kashiwagi, D. T., Burton, M. C. et al., The Charlson Comorbidity Index Score as a predictor of 30-day mortality after hip fracture surgery. Am J Med Qual 26:461–467, 2011. Scholar
  4. 4.
    Fish, D. R., Mancuso, C. A., Garcia-Aguilar, J. E. et al., Readmission After Ileostomy Creation: Retrospective Review of a Common and Significant Event. Ann Surg 265:379–387, 2017. Scholar
  5. 5.
    Guo, R., Yu, W., Meng, Y. et al., Correlation of ASA Grade and the Charlson Comorbidity Index With Complications in Patients After Transurethral Resection of Prostate. Urology 98:120–125, 2016. Scholar
  6. 6.
    Deyo, R. A., Cherkin, D. C., and Ciol, M. A., Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 45:613–619, 1992.CrossRefPubMedGoogle Scholar
  7. 7.
    Quan, H., Sundararajan, V., Halfon, P. et al., Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 43:1130–1139, 2005.CrossRefPubMedGoogle Scholar
  8. 8.
    Meurer S. Mortality Risk Adjustment Methodology for University Health System’s Clinical Data Base. AHRQ/IHI Mortal. Meas. Meet. 2008;:1–13. (accessed 7 Mar 2018).
  9. 9.
    Averill, R. F., Goldfield, N., Muldoon, J. et al., A closer look at all-patient refined DRGs. J AHIMA 73:46–50, 2002.PubMedGoogle Scholar
  10. 10.
    3M HIS. An Overview of 3M All Patient Refined Diagnostic Related Groups (3M APR DRG). 2012. (accessed 7 Mar 2018).
  11. 11.
    Goldfield, N., and Averill, R. F., On ‘Risk-adjusting acute myocardial infarction mortality: Are APR-DRGs the right tool?’. Health Serv Res 34:1491–1498, 2000.PubMedPubMedCentralGoogle Scholar
  12. 12.
    Baram, D., Daroowalla, F., Garcia, R. et al., Use of the All Patient Refined-Diagnosis Related Group (APR-DRG) Risk of Mortality Score as a Severity Adjustor in the Medical ICU. Clin Med Circ Respirat Pulm Med 2:19–25, 2008.PubMedPubMedCentralGoogle Scholar
  13. 13.
    Sessler, D. I., Sigl, J. C., Manberg, P. J. et al., Broadly applicable risk stratification system for predicting duration of hospitalization and mortality. Anesthesiology 113:1026–1037, 2010. Scholar
  14. 14.
    Yurkovich, M., Avina-Zubieta, J. A., Thomas, J. et al., A systematic review identifies valid comorbidity indices derived from administrative health data. J Clin Epidemiol 68:3–14, 2015. Scholar
  15. 15.
    Schneeweiss, S., Wang, P. S., Avorn, J. et al., Improved comorbidity adjustment for predicting mortality in Medicare populations. Health Serv Res 38:1103–1120, 2003.CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Romano, P. S., Roos, L. L., and Jollis, J. G., Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol 46:1075–1090, 1993. Scholar
  17. 17.
    Sundararajan, V., Henderson, T., Perry, C. et al., New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. J Clin Epidemiol 57:1288–1294, 2004. Scholar
  18. 18.
    van Walraven, C., Austin, P. C., Jennings, A. et al., A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care 47:626–633, 2009. Scholar
  19. 19.
    Sharabiani, M. T., Aylin, P., and Bottle, A., Systematic Review of Comorbidity Indices for Administrative Data. Med Care 50:1, 2012. Scholar
  20. 20.
    Charlson, M. E., Szatrowski, T. P., Peterson, J. et al., Validation of a combined comorbidity index. J Clin Epidemiol 47:1245–1251, 1994.CrossRefPubMedGoogle Scholar
  21. 21.
    Amrock, L. G., Neuman, M. D., Lin, H.-M. et al., Can routine preoperative data predict adverse outcomes in the elderly? Development and validation of a simple risk model incorporating a chart-derived frailty score. J Am Coll Surg 219:684–694, 2014. Scholar
  22. 22.
    Dalton, J. E., Kurz, A., Turan, A. et al., Development and validation of a risk quantification index for 30-day postoperative mortality and morbidity in noncardiac surgical patients. Anesthesiology 114:1336–1344, 2011. Scholar
  23. 23.
    Sigakis, M. J. G., Bittner, E. A., and Wanderer, J. P., Validation of a risk stratification index and risk quantification index for predicting patient outcomes: in-hospital mortality, 30-day mortality, 1-year mortality, and length-of-stay. Anesthesiology 119:525–540, 2013. Scholar
  24. 24.
    Reyes, C., Greenbaum, A., Porto, C. et al., Implementation of a Clinical Documentation Improvement Curriculum Improves Quality Metrics and Hospital Charges in an Academic Surgery Department. J Am Coll Surg 224:301–309, 2017. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Patrick J. McCormick
    • 1
  • Hung-mo Lin
    • 2
    • 3
  • Stacie G. Deiner
    • 3
    • 4
    • 5
  • Matthew A. Levin
    • 3
    • 6
  1. 1.Department of Anesthesiology & Critical CareMemorial Sloan Kettering Cancer CenterNew YorkUSA
  2. 2.Department of Population Health Science and PolicyIcahn School of Medicine at Mount SinaiNew YorkUSA
  3. 3.Department of Anesthesiology, Perioperative and Pain MedicineIcahn School of Medicine at Mount SinaiNew YorkUSA
  4. 4.Department of NeurosurgeryIcahn School of Medicine at Mount SinaiNew YorkUSA
  5. 5.Department of Geriatrics and Palliative CareIcahn School of Medicine at Mount SinaiNew YorkUSA
  6. 6.Department of Genetics and Genomic SciencesIcahn School of Medicine at Mount SinaiNew YorkUSA

Personalised recommendations