Prospective Validation of the Iowa Rectal Surgery Risk Calculator
- 94 Downloads
The Iowa Rectal Surgery Risk Calculator estimates risk for proctectomy procedures. The Iowa Calculator performed well on NSQIP 2010–2011 training and 2005–2009 validation datasets, but was not prospectively validated and did not include low anterior resections. This study sought to demonstrate validity on new independent data, to update the calculator to include low anterior resection, and to compare performance to other risk assessment tools.
Non-emergent ACS-NSQIP proctectomy and low anterior resection data from 2010 to 2015 (n = 65,683) were included. The Iowa Calculator generated risk estimates for 30-day morbidity using 2012–2015 data. An Updated Calculator used 2010–2011 training data to include low anterior resection, with validation on 2012–2015 data. NSQIP data provided NSQIP Morbidity Model predictions and a custom web-script collected ACS-NSQIP Online Surgical Risk Calculator predictions for all patients.
Proctectomy morbidity (not including low anterior resection) decreased from 40.4% in 2010–2011 to 37.0% in 2012–2015. Low anterior resection had lower morbidity (22.4% in 2012–15). The Iowa Calculator demonstrated good discrimination and calibration using 2012–2015 data (C-statistic 0.676, deviance + 9.2%). After including low anterior resection, the Updated Iowa Calculator performed well during training (c-statistic 0.696, deviance 0%) and validation (C-statistic 0.706, deviance + 7.9%). The Updated Iowa Calculator had significantly better discrimination and calibration than morbidity predictions from the ACS Online Calculator (C-statistic 0.693, P < 0.001, deviance − 28.1%) and NSQIP General/Vascular Surgery Model (C-statistic 0.703, P < 0.05, deviance − 40.8%).
When applied to new independent data, the Iowa Calculator supplies accurate risk estimates. The Updated Iowa Calculator includes low anterior resection, and both are prospectively validated. Risk estimation by the Iowa Calculators was superior to ACS-provided risk tools.
KeywordsRectal surgery Morbidity Complications Risk calculator
This study was supported by NIH 5T32#CA148062-03 (SKS and JEH). The authors thank Mary E. Belding-Schmitt, RN, BSN for database support.
Sherman: study design, data analysis, data interpretation, writing, critical revision; Hrabe: study design, data interpretation, writing, critical revision; Huang: data interpretation and critical revision; Cromwell: study design, data interpretation, critical revision; Byrn: study design, data interpretation, critical revision.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they have no conflicts of interest.
Bolded Names Indicate Co-First Authorship.
- 4.Hrabe, J. E., Sherman, S. K., Charlton, M. E., Cromwell, J. W. & Byrn, J. C. Effect of BMI on outcomes in proctectomy. Dis. Colon rectum 2014 May; 57(5):608–615.Google Scholar
- 8.User Guide for the 2015 ACS-NSQIP Participant Use Data File (PUF). American College of Surgeons. https://www.facs.org/~/media/files/quality%20programs/nsqip/nsqip_puf_user_guide_2015.ashx (Oct. 2016).
- 9.ACS-NSQIP Surgical Risk Calculator. American College of Surgeons. https://riskcalculator.facs.org/RiskCalculator/, Accessed February–March, 2017.
- 14.Wilks, D. S. Statistical Methods in the Atmospheric Sciences. Academic Press, San Diego, CA, Mar. 1995.Google Scholar
- 16.Collins, G. S. et al. External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC Med. Res. Methodol. 2014 Mar; 14:40. 17.Google Scholar
- 21.Rosenberger, L. H., Politano, A. D. & Sawyer, R. G. The surgical care improvement project and prevention of post-operative infection, including surgical site infection. Surg. Infect. (Larchmt) 2011 Jun; 12(3):163–168.Google Scholar
- 29.Kramer, A. A. & Zimmerman, J. E. Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited. Crit. Care Med. 2007 Sep; 35(9):2052–2056. 30.Google Scholar
- 33.Liu, Y., Cohen, M. E., Ko, C. Y., Bilimoria, K. Y. & Hall, B. L. Considerations in Releasing Equations for the American College of Surgeons NSQIP Surgical Risk Calculator: In Reply to Wanderer and Ehrenfeld. J. Am. Coll. Surg. 2016 Oct; 223(4):674–675.Google Scholar