Skip to main content


Log in

Developing prediction models for short-term mortality after surgery for colorectal cancer using a Danish national quality assurance database

  • Research
  • Published:
International Journal of Colorectal Disease Aims and scope Submit manuscript



The majority of colorectal cancer surgeries are performed electively, and treatment is often decided at the multidisciplinary team conference. Although the average 30-day mortality rate is low, there is substantial population heterogeneity from young, healthy patients to frail, elderly patients. The individual risk of surgery can vary widely, and tailoring treatment for colorectal cancer may lead to better outcomes. This requires prediction of risk that is accurate and available prior to surgery.


Data from the Danish Colorectal Cancer Group database was transformed into the Observational Medical Outcomes Partnership Common Data Model. Models were developed to predict the risk of mortality within 30, 90, and 180 days after colorectal cancer surgery using only covariates decided at the multidisciplinary team conference. Several machine-learning models were trained, but due to superior performance, a Least Absolute Shrinkage and Selection Operator logistic regression was used for the final model. Performance was assessed with discrimination (area under the receiver operating characteristic and precision recall curve) and calibration measures (calibration in large, intercept, slope, and Brier score).


The cohort contained 65,612 patients operated for colorectal cancer in the period from 2001 to 2019 in Denmark. The Least Absolute Shrinkage and Selection Operator model showed an area under the receiver operating characteristic for 30-, 90-, and 180-day mortality after colorectal cancer surgery of 0.871 (95% CI: 0.86–0.882), 0.874 (95% CI: 0.864–0.882), and 0.876 (95% CI: 0.867–0.883) and calibration in large of 1.01, 0.98, and 1.01, respectively.


The postoperative short-term mortality prediction model showed excellent discrimination and calibration using only preoperatively known predictors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others


  1. Center MM, Jemal A, Smith RA, Ward E (2010) Worldwide variations in colorectal cancer. Dis Colon Rectum 53:1099.

    Article  Google Scholar 

  2. Bojesen RD, Degett TH, Dalton SO, Gögenur I (2021) High World Heath Organization performance status is associated with short and long-term outcomes after colorectal cancer surgery. Dis Colon Rectum 58–66.

  3. Bojesen RD, Jørgensen LB, Grube C, Skou ST, Johansen C, Dalton SO et al (2022) Fit for surgery — feasibility of short-course multimodal individualized prehabilitation in high-risk frail colon cancer patients prior to surgery. Pilot Feasibility Stud 1–13.

  4. Palmer G, Martling A, Cedermark B, Holm T (2011) Preoperative tumour staging with multidisciplinary team assessment improves the outcome in locally advanced primary rectal cancer. Colorectal Dis 1361–9.

  5. Rosander E, Holm T, Sjövall A, Hjern F, Weibull CE, Nordenvall C (2021) Preoperative multidisciplinary team assessment is associated with improved survival in patients with locally advanced colon cancer; a nationwide cohort study in 3157 patients. Eur J Surg Oncol 47:2398–2404.

    Article  CAS  PubMed  Google Scholar 

  6. Ingeholm P, Gögenur I, Iversen LH (2016) Danish colorectal cancer group database. Clin Epidemiol 8:465–468.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Klein MF, Gögenur I, Ingeholm P, Njor SH, Iversen LH, Emmertsen KJ (2020) Validation of the Danish Colorectal Cancer Group ( database - on behalf of the Danish Colorectal Cancer Group. Color Dis Off J Assoc Coloproctology Gt Britain Irel 22:2057–67.

  8. Vogelsang RP, Bojesen RD, Hoelmich ER, Orhan A, Buzquurz F, Cai L et al (2021) Prediction of 90-day mortality after surgery for colorectal cancer using standardized nationwide quality-assurance data. BJS Open 5.

  9. Observational health data sciences and informatics (2019) The Book of OHDSI 1–470.

  10. Danish Colorectal Cancer Group (DCCG) (2022) No title. Var List 2022. (Accessed 16 June 2022).

  11. Mandrekar JN (2010) Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol 5:1315–1316.

    Article  PubMed  Google Scholar 

  12. Rufibach K (2010) Use of Brier score to assess binary predictions. J Clin Epidemiol 63:938–939.

    Article  PubMed  Google Scholar 

  13. Collins GS, Reitsma JB, Altman DG, Moons KGM (2015) Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 162:55–63.

    Article  PubMed  Google Scholar 

  14. Iversen LH, Ingeholm P, Gögenur I, Laurberg S (2014) Major reduction in 30-day mortality after elective colorectal cancer surgery: a nationwide population-based study in Denmark 2001–2011. Ann Surg Oncol 21:2267–2273.

    Article  PubMed  Google Scholar 

  15. Ingeholm P (2017) Landsdækkende database for kræft i tyk- og endetarm ( Klinisk rapport. Danish Color Cancer Gr

  16. Panis Y, Maggiori L, Caranhac G, Bretagnol F, Vicaut E (2011) Mortality after colorectal cancer surgery: a French survey of more than 84,000 patients. Ann Surg 254:738–743.

    Article  PubMed  Google Scholar 

  17. Morris EJA, Taylor EF, Thomas JD, Quirke P, Finan PJ, Coleman MP et al (2011) Thirty-day postoperative mortality after colorectal cancer surgery in England. Gut 60:806–813.

    Article  PubMed  Google Scholar 

  18. Ketelaers SHJ, Orsini RG, Burger JWA, Nieuwenhuijzen GAP, Rutten HJT (2019) Significant improvement in postoperative and 1-year mortality after colorectal cancer surgery in recent years. Eur J Surg Oncol 45:2052–2058.

    Article  CAS  PubMed  Google Scholar 

  19. Gietelink L, Wouters MWJM, Bemelman WA, Dekker JW, Tollenaar RAEM, Tanis PJ (2016) Reduced 30-day mortality after laparoscopic colorectal cancer surgery. Ann Surg 264:135–140.

    Article  PubMed  Google Scholar 

  20. Cavallaro P, Bordeianou L (2019) Implementation of an ERAS pathway in colorectal surgery. Clin Colon Rectal Surg 32:102–108.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Bilimoria KY, Liu Y, Paruch JL, Zhou L, Kmiecik TE, Ko CY et al (2013) Surgical risk calculator: a decision aide and informed consent tool for patients and surgeons. J Am Coll Surg 217(5):833-842.e3.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Fazio VW, Tekkis PP, Remzi F, Lavery IC (2004) Assessment of operative risk in colorectal cancer surgery: the cleveland clinic foundation colorectal cancer model. Dis Colon Rectum 47:2015–2024.

    Article  PubMed  Google Scholar 

  23. van der Sluis FJ, Espin E, Vallribera F, de Bock GH, Hoekstra HJ, van Leeuwen BL et al (2014) Predicting postoperative mortality after colorectal surgery: a novel clinical model. Color Dis 16:631–639.

    Article  Google Scholar 

  24. Van Den Bosch T, Warps ALK, De Nerée Tot Babberich MPM, Stamm C, Geerts BF, Vermeulen L et al (2021) Predictors of 30-day mortality among Dutch patients undergoing colorectal cancer surgery, 2011–2016. JAMA Netw Open 4:2011–6.

  25. Assel M, Sjoberg DD, Vickers AJ (2017) The Brier score does not evaluate the clinical utility of diagnostic tests or prediction models. Diagnostic Progn Res 1:19.

    Article  Google Scholar 

  26. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N et al (2010) Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 21:128–138.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Scott I, Carter S, Coiera E (2021) Clinician checklist for assessing suitability of machine learning applications in healthcare. BMJ Heal Care Informatics 28.

  28. Kim JH (2019) Introduction multicollinearity and misleading statistical results KJA. Korean J Anesth 558–69.

  29. Cepeda MS, Reps J, Ryan P (2018) Finding factors that predict treatment-resistant depression: results of a cohort study. Depress Anxiety 35:668–673.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references


The authors thank The Danish Clinical Quality Program and Danish Colorectal Cancer Group for access to their outstanding data. We also sincerely thank the European Health Data Evidence Network (EHDEN), edenceHealth, and Computerome for contributions and support during the project. Lastly, we thank Peter Rijnbeek and Iannis Drakos for sparring and advice in the process.

Author information

Authors and Affiliations



All authors contributed to the study design. Karoline Bendix Bräuner, Julie Sparholt Walbech, and Adamantia Tsouchnika did the statistical analyses. Karoline Bendix Bräuner, Andreas Weinberger Rosen, Viviane Annabell Lin, Mikail Gögenur, and Johan Stub Rønø Clausen defined the research questions and interpreted the results. Karoline Bendix Bräuner and Julie Sparholt Walbech prepared tables and figures. Karoline Bendix Bräuner wrote the main manuscript text. All authors have reviewed the manuscript and given approval for submission to International Journal of Colorectal Disease.

Corresponding author

Correspondence to Karoline B. Bräuner.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This paper in an earlier format has been presented as a poster at the Observational Health Data Science and Informatics Symposium 2021 (online convention) running from Sunday September 12th 2021 to Wednesday September 15th 2021 and as a lightning talk on the Danish Surgical Society annual meeting in Copenhagen, Denmark, running from Thursday November 18th 2021 to Friday November 19th 2021.


• This study utilizes a national quality assurance register as cohort for the development of the prediction model.

• This study presents a model with good discrimination and calibration (for 30-day mortality AUROC of 0.871 and Brier score of 0.04) using only predictors known at the multidisciplinary team conference making it available in this setting.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 27 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bräuner, K.B., Rosen, A.W., Tsouchnika, A. et al. Developing prediction models for short-term mortality after surgery for colorectal cancer using a Danish national quality assurance database. Int J Colorectal Dis 37, 1835–1843 (2022).

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: