Predicting Overall Survival in Patients with Metastatic Rectal Cancer: a Machine Learning Approach

  • Beiqun Zhao
  • Rodney A. Gabriel
  • Florin Vaida
  • Nicole E. Lopez
  • Samuel Eisenstein
  • Bryan M. Clary
Original Article



A significant proportion of patients with rectal cancer will present with synchronous metastasis at the time of diagnosis. Overall survival (OS) for these patients are highly variable and previous attempts to build predictive models often have low predictive power, with concordance indexes (c-index) less than 0.70.


Using the National Cancer Database (2010–2014), we identified patients with synchronous metastatic rectal cancer. The data was split into a training dataset (diagnosis years 2010–2012), which was used to build the machine learning model, and a testing dataset (diagnosis years 2013–2014), which was used to externally validate the model. A nomogram predicting 3-year OS was created using Cox proportional hazard regression with lasso penalization. Predictors were selected based on clinical significance and availability in NCDB. Performance of the machine learning model was assessed by c-index.


A total of 4098 and 3107 patients were used to construct and validate the nomogram, respectively. Internally validated c-indexes at 1, 2, and 3 years were 0.816 (95% CI 0.813–0.818), 0.789 (95% CI 0.786–0.790), and 0.778 (95% CI 0.775–0.780), respectively. External validated c-indexes at 1, 2, and 3 years were 0.811, 0.779, and 0.778, respectively.


There is wide variability in the OS for patients with metastatic rectal cancer, making accurate predictions difficult. However, using machine learning techniques, more accurate models can be built. This will aid patients and clinicians in setting expectations and making clinical decisions in this group of challenging patients.


Rectal cancer Machine learning Nomograms Lasso NCDB 


Grant Support

Dr. Zhao is supported by the National Library of Medicine Training Grant (NIH Grant: T15LM011271).

Author Contribution Statement

All authors met the standards set by the International Committee of Medical Journal Editors to be listed as an author for this manuscript.

Compliance with Ethical Standards

Conflicts of Interest

The authors declare that they have no conflict of interest.


The funding source had no role in the design and/or general conduct of this study; had no access to the data or role in data collection, management, analysis, or interpretation; had no role in the preparation, review, or approval of the manuscript; and had no role in the decision to submit the manuscript for publication.


  1. 1.
    American Cancer Society (2018) Cancer Facts & Figures.Google Scholar
  2. 2.
    Manfredi S, Lepage C, Hatem C, Coatmeur O, Faivre J, Bouvier AM (2006) Epidemiology and management of liver metastases from colorectal cancer. Ann Surg 244:254–259. CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Hackl C, Neumann P, Gerken M, Loss M, Klinkhammer-Shalke M, Schlitt HJ (2014) Treatment of colorectal liver metastases in Germany: a ten-year population-based analysis of 5772 cases of primary colorectal adenocarcinoma. BMC Cancer 14:810–820. CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Riihimaki M, Hemminki A, Sundquist J, Hemminki K (2016) Patterns of metastasis in colon and rectal cancer. Sci Rep 6:1–9. CrossRefGoogle Scholar
  5. 5.
    Pawlik TM, Scoggins CR, Zorzi D, Abdalla EK, Andres A, Eng C, Curley SA, Loyer EM, Muratore A, Mentha G, Capussotti L, Vauthey JN, Choti MA, Adams RB, Bolton JS, Hemming AW, Cofer JB, Smythe WR, Clary BM, Vauthey JN (2005) Effect of surgical margin status on survival and site of recurrence after hepatic resection for colorectal metastases. Ann Surg 241:715–724. CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Portier G, Elias D, Bouche O, Rougier P, Bosset JF, Saric J, Belghiti J, Piedbois P, Guimbaud R, Nordlinger B, Bugat R, Lazorthes F, Bedenne L (2006) Multicenter randomized trial of adjuvant fluorouracil and folinic acid compared with surgery alone after resection of colorectal liver metastases: FFCD ACHBTH AURC 9002 trial. J Clin Oncol 24:4976–4982. CrossRefPubMedGoogle Scholar
  7. 7.
    Aloia T, Sebagh M, Plasse M, Karam V, Lévi F, Giacchetti S, Azoulay D, Bismuth H, Castaing D, Adam R (2006) Liver Histology and Surgical Outcomes After Preoperative Chemotherapy With Fluorouracil Plus Oxaliplatin in Colorectal Cancer Liver Metastases. J Clin Oncol 24:4983–4990. CrossRefPubMedGoogle Scholar
  8. 8.
    Kopetz S, Chang GJ, Overman MJ, Eng C, Sargent DJ, Larson DW, Grothey A, Vauthey JN, Nagorney DM, McWilliams RR (2009) Improved survival in metastatic colorectal cancer is associated with adoption of hepatic resection and improved chemotherapy. J Clin Oncol 27:3677–3683. CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Harrell FE, Califf RM, Pryor DB, Lee KL, Rosati RA (1982) Evaluating the Yield of Medical Tests. JAMA 247:2543–2546. CrossRefPubMedGoogle Scholar
  10. 10.
    Beppu T, Sakamoto Y, Hasegawa K, Honda G, Tanaka K, Kotera Y, Nitta H, Yoshidome H, Hatano E, Ueno M, Takamura H, Baba H, Kosuge T, Kokudo N, Takahashi K, Endo I, Wakabayashi G, Miyazaki M, Uemoto S, Ohta T, Kikuchi K, Yamaue H, Yamamoto M, Takada T (2012) A nomogram predicting disease-free survival in patients with colorectal liver metastases treated with hepatic resection: Multicenter data collection as a project study for hepatic surgery of the Japanese Society of Hepato-Biliary-Pancreatic Surgery. J Hepatobiliary Pancreat Sci 19:72–84. CrossRefPubMedGoogle Scholar
  11. 11.
    Kanemitsu Y, Kato T (2008) Prognostic models for predicting death after hepatectomy in individuals with hepatic metastases from colorectal cancer. World J Surg 32:1097–1107. CrossRefPubMedGoogle Scholar
  12. 12.
    Kanemitsu Y, Kato T, Hirai T, Yasui K (2004) Preoperative probability model for predicting overall survival after resection of pulmonary metastases from colorectal cancer. Br J Surg 91:112–120. CrossRefPubMedGoogle Scholar
  13. 13.
    Kattan MW, Gönen M, Jarnagin WR, DeMatteo R, D’Angelica M, Weiser M, Blumgart LH, Fong Y (2008) A nomogram for predicting disease-specific survival after hepatic resection for metastatic colorectal cancer. Ann Surg 247:282–287. CrossRefPubMedGoogle Scholar
  14. 14.
    Nagashima I, Takada T, Matsuda K, Adachi M, Nagawa H, Muto T, Okinaga K (2004) A new scoring system to classify patients with colorectal liver metastases: Proposal of criteria to select candidates for hepatic resection. J Hepatobiliary Pancreat Surg 11:79–83. CrossRefPubMedGoogle Scholar
  15. 15.
    Nordlinger B, Guiguet M, Vaillant J, Balladur P, Boudjema K, Bachellier P, Jaeck D (1996) Surgical Resection of Colorectal Carcinoma Metastases to the Liver: A Prognostic Scoring System to Improve Case Selection, Based on 1568 Patients. Cancer 77:1254–62CrossRefPubMedGoogle Scholar
  16. 16.
    Sasaki K, Morioka D, Conci S, Margonis GA, Sawada Y, Ruzzenente A, Kumamoto T, Iacono C, Andreatos N, Guglielmi A, Endo I, Pawlik TM (2018) The Tumor Burden Score: A New “metro-ticket” Prognostic Tool for Colorectal Liver Metastases Based on Tumor Size and Number of Tumors. Ann Surg 267:132–141. CrossRefPubMedGoogle Scholar
  17. 17.
    Elias D, Faron M, Goéré D, Dumont F, Honoré C, Boige V, Malka D, Ducreux M (2014) A simple tumor load-based nomogram for surgery in patients with colorectal liver and peritoneal metastases. Ann Surg Oncol 21:2052–2058. CrossRefPubMedGoogle Scholar
  18. 18.
    Kawai K, Ishihara S, Yamaguchi H, Sunami E, Kitayama J, Miyata H, Sugihara K, Watanabe T (2015) Nomograms for predicting the prognosis of stage IV colorectal cancer after curative resection: A multicenter retrospective study. Eur J Surg Oncol 41:457–465. CrossRefPubMedGoogle Scholar
  19. 19.
    Zacharakis M, Xynos ID, Lazaris A, Smaro T, Kosmas C, Dokou A, Felekouras E, Antoniou E, Polyzos A, Sarantonis J, Syrios J, Zografos G, Papalambros A, Tsavaris N (2010) Predictors of survival in stage IV metastatic colorectal cancer. Anticancer Res 30:653–660. CrossRefPubMedGoogle Scholar
  20. 20.
    Qiu M, Hu J, Yang D, Cosgrove DP, Xu R (2015) Pattern of distant metastases in colorectal cancer: a SEER based study. Oncotarget 6:38658–38666PubMedPubMedCentralGoogle Scholar
  21. 21.
    Deyo RA, Cherkin DC, Ciol MA (1992) Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 45:613–619CrossRefPubMedGoogle Scholar
  22. 22.
    Tibshirani R (1996) Regression Shrinkage and Selection via the Lasso. J R Stat Soc 58:267–288Google Scholar
  23. 23.
    Dahabreh I, Chan J, Earley A, Moorthy D, Avendano E, Trikalinos T, Balk E, Wong J (2017) Modeling and Simulation in the Context of Health Technology Assessment: Review of Existing Guidance, Future Research Needs, and Validity AssessmentGoogle Scholar
  24. 24.
    Xiao N (2016) hdnom: Building Nomograms for Penalized Cox Models with High-Dimensional Survival Data. 1–21Google Scholar
  25. 25.
    Kawai K, Sunami E, Yamaguchi H, Ishihara S, Kazama S, Nozawa H, Hata K, Kiyomatsu T, Tanaka J, Tanaka T, Nishikawa T, Kitayama J, Watanabe T (2015) Nomograms for colorectal cancer: A systematic review. World J Gastroenterol 21:11877–11886. CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    Delaunoit T, Alberts SR, Sergent DJ, Green E, Goldberg RM, Krook J, Fuchs C, Ramanathan RK, Williamson SK, Morton RF, Findlay BP (2005) Chemotherapy permits resection of metastatic colorectal cancer: Experience from intergroup N9741. Ann Oncol 16:425–429. CrossRefPubMedGoogle Scholar
  27. 27.
    Abdalla EK, Adam R, Bilchik AJ, Jaeck D, Vauthey J-N, Mahvi D (2006) Improving Resectability of Hepatic Colorectal Metastases: Expert Consensus Statement. Ann Surg Oncol 13:1271–1280. CrossRefPubMedGoogle Scholar
  28. 28.
    Rocha FG, Helton WS (2012) Resectability of colorectal liver metastases: An evolving definition. Hpb 14:283–284. CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Gulack B, Nussbaum D, Keenan J, Ganapathi A, Sun Z, Worni M, Migaly J, Mantyh C (2016) Surgical Resection of the Primary Tumor in Stage IV Colorectal without Metastasectomy is Associated with Improved Overall Survival Compared to Chemotherapy/Radiation Therapy Alone. Dis Colon Rectum 59:299–305. CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Nash GM, Saltz LB, Kemeny NE, Minsky B, Sharma S, Schwartz GK, Ilson DH, O’Reilly E, Kelsen DP, Nathanson DR, Weiser M, Guillem JG, Douglas Wong W, Cohen AM, Paty PB (2002) Radical resection of rectal cancer primary tumor provides effective local therapy in patients with stage IV disease. Ann Surg Oncol 9:954–960. CrossRefPubMedGoogle Scholar
  31. 31.
    Alawadi Z, Phatak UR, Hu CY, Bailey CE, You YN, Kao LS, Massarweh NN, Feig BW, Rodriguez-Bigas MA, Skibber JM, Chang GJ (2017) Comparative effectiveness of primary tumor resection in patients with stage IV colon cancer. Cancer 123:1124–1133. CrossRefPubMedGoogle Scholar
  32. 32.
    Temple LKF, Hsieh L, Wong WD, Saltz L, Schrag D (2004) Use of surgery among elderly patients with stage IV colorectal cancer. J Clin Oncol 22:3475–3484. CrossRefPubMedGoogle Scholar
  33. 33.
    Massacesi C, Norman A, Price T, Hill M, Ross P, Cunningham D (2000) A clinical nomogram for predicting long-term survival in advanced colorectal cancer. Eur J Cancer 36:2044–2052. CrossRefPubMedGoogle Scholar
  34. 34.
    Baek SJ, Hur H, Min BS, Baik SH, Lee KY, Kim NK (2016) The Characteristics of Bone Metastasis in Patients with Colorectal Cancer: A Long-Term Report from a Single Institution. World J Surg 40:982–986. CrossRefPubMedGoogle Scholar
  35. 35.
    Damiens K, Ayoub JM, Lemieux B, Aubin F, Saliba W, Campeau M, Tehfe MA (2012) Clinical features and course of brain metastases in colorectal cancer: Experience from a single institution. Curr Oncol 19.
  36. 36.
    Michl M, Thurmaier J, Schubert-Fritschle G, Wiedemann M, Laubender RP, Nüssler NC, Ruppert R, Kleeff J, Schepp W, Reuter C, Löhe F, Karthaus M, Neumann J, Kirchner T, Engel J, Heinemann V (2015) Brain Metastasis in Colorectal Cancer Patients: Survival and Analysis of Prognostic Factors. Clin Colorectal Cancer 14:281–290. CrossRefPubMedGoogle Scholar
  37. 37.
    Pfannschmidt J, Dienemann H, Hoffmann H (2007) Surgical Resection of Pulmonary Metastases From Colorectal Cancer: A Systematic Review of Published Series. Ann Thorac Surg 84:324–338. CrossRefPubMedGoogle Scholar
  38. 38.
    Engstrand J, Nilsson H, Strömberg C, Jonas E, Freedman J (2018) Colorectal cancer liver metastases - a population-based study on incidence, management and survival. BMC Cancer 18:1–11. CrossRefGoogle Scholar
  39. 39.
    Manyam B V., Mallick IH, Abdel-Wahab MM, Reddy CA, Remzi FH, Kalady MF, Lavery I, Koyfman SA (2015) The Impact of Preoperative Radiation Therapy on Locoregional Recurrence in Patients with Stage IV Rectal Cancer Treated with Definitive Surgical Resection and Contemporary Chemotherapy. J Gastrointest Surg 19:1676–1683. CrossRefPubMedGoogle Scholar
  40. 40.
    Repka MC, Aghdam N, Karlin AW, Unger KR (2017) Social determinants of stage IV anal cancer and the impact of pelvic radiotherapy in the metastatic setting. Cancer Med 6:2497–2506. CrossRefPubMedPubMedCentralGoogle Scholar
  41. 41.
    Renz P, Wegner RE, Hasan S, Brookover R, Finley G, Monga D, Raj M, McCormick J, Kirichenko A (2019) Survival Outcomes After Surgical Management of the Primary Tumor With and Without Radiotherapy for Metastatic Rectal Adenocarcinoma: A National Cancer Database (NCDB) Analysis. Clin Colorectal Cancer 1–7.
  42. 42.
    Bejnordi BE, Veta M, Van Diest PJ, Van Ginneken B, Karssemeijer N, Litjens G, Van Der Laak JAWM, Hermsen M, Manson QF, Balkenhol M, Geessink O, Stathonikos N, Van Dijk MCRF, Bult P, Beca F, Beck AH, Wang D, Khosla A, Gargeya R, Irshad H, Zhong A, Dou Q, Li Q, Chen H, Lin HJ, Heng PA, Haß C, Bruni E, Wong Q, Halici U, Öner MÜ, Cetin-Atalay R, Berseth M, Khvatkov V, Vylegzhanin A, Kraus O, Shaban M, Rajpoot N, Awan R, Sirinukunwattana K, Qaiser T, Tsang YW, Tellez D, Annuscheit J, Hufnagl P, Valkonen M, Kartasalo K, Latonen L, Ruusuvuori P, Liimatainen K, Albarqouni S, Mungal B, George A, Demirci S, Navab N, Watanabe S, Seno S, Takenaka Y, Matsuda H, Phoulady HA, Kovalev V, Kalinovsky A, Liauchuk V, Bueno G, Fernandez-Carrobles MM, Serrano I, Deniz O, Racoceanu D, Venâncio R (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318:2199–2210. CrossRefGoogle Scholar
  43. 43.
    Upstill-Goddard R, Eccles D, Fliege J, Collins A (2013) Machine learning approaches for the discovery of gene-gene interactions in disease data. Brief Bioinform 14:251–260. CrossRefPubMedGoogle Scholar
  44. 44.
    Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88. CrossRefPubMedGoogle Scholar
  45. 45.
    Kehl KL, Landrum MB, Arora NK, Ganz PA, Van Ryn M, Mack JW, Keating NL (2015) Association of actual and preferred decision roles with patient-reported quality of care: Shared decision making in cancer care. JAMA Oncol 1:50–58. CrossRefPubMedPubMedCentralGoogle Scholar
  46. 46.
    Balachandran VP, Gonen M, Smith JJ, DeMatteo RP (2015) Nomograms in Oncology - More than Meets the Eye. Lancet Oncol 16:173–180. CrossRefGoogle Scholar
  47. 47.
    Caulfield S, Menezes G, Marignol L, Poole C (2018) Nomograms are key decision-making tools in prostate cancer radiation therapy. Urol Oncol Semin Orig Investig 36:283–292. CrossRefGoogle Scholar

Copyright information

© The Society for Surgery of the Alimentary Tract 2019

Authors and Affiliations

  1. 1.Department of SurgeryUniversity of California San DiegoSan DiegoUSA
  2. 2.San DiegoUSA
  3. 3.Department of AnesthesiologyUniversity of California San DiegoSan DiegoUSA
  4. 4.Department of Family Medicine and Public HealthUniversity of California San DiegoSan DiegoUSA

Personalised recommendations