Skip to main content

Advertisement

SpringerLink
Log in

External validation and comparison of MR-based radiomics models for predicting pathological complete response in locally advanced rectal cancer: a two-centre, multi-vendor study

  • Imaging Informatics and Artificial Intelligence
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

The aim of this study was two-fold: (1) to develop and externally validate a multiparameter MR-based machine learning model to predict the pathological complete response (pCR) in locally advanced rectal cancer (LARC) patients after neoadjuvant chemoradiotherapy (nCRT), and (2) to compare different classifiers’ discriminative performance for pCR prediction.

Methods

This retrospective study includes 151 LARC patients divided into internal (centre A, n = 100) and external validation set (centre B, n = 51). The clinical and MR radiomics features were derived to construct clinical, radiomics, and clinical-radiomics model. Random forest (RF), support vector machine (SVM), logistic regression (LR), K-nearest neighbor (KNN), naive Bayes (NB), and extreme gradient boosting (XGBoost) were used as classifiers. The predictive performance was assessed using the receiver operating characteristic (ROC) curve.

Results

Eleven radiomics and four clinical features were chosen as pCR-related signatures. In the radiomics model, the RF algorithm achieved 74.0% accuracy (an AUC of 0.863) and 84.4% (an AUC of 0.829) in the internal and external validation sets. In the clinical-radiomics model, RF algorithm exhibited high and stable predictive performance in the internal and external validation datasets with an AUC of 0.906 (87.3% sensitivity, 73.7% specificity, 76.0% accuracy) and 0.872 (77.3% sensitivity, 88.2% specificity, 86.3% accuracy), respectively. RF showed a better predictive performance than the other classifiers in the external validation datasets of three models.

Conclusions

The multiparametric clinical-radiomics model combined with RF algorithm is optimal for predicting pCR in the internal and external sets, and might help improve clinical stratifying management of LARC patients.

Key Points

• A two-centre study showed that radiomics analysis of pre- and post-nCRT multiparameter MR images could predict pCR in patients with LARC.

• The combined model was superior to the clinical and radiomics model in predicting pCR in locally advanced rectal cancer.

• The RF classifier performed best in the current study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

AUC:

Area under the ROC curve

CEA:

Carcinoembryonic antigen

DL:

Deep learning

DWI:

Diffusion-weighted imaging

KNN:

K-nearest neighbor

LARC:

Locally advanced rectal cancer

LASSO:

Least absolute shrinkage and selection operator

LoG:

Laplacian of Gaussian

LR:

Logistic regression

ML:

Machine learning

NB:

Naive Bayes

nCRT:

Neoadjuvant chemoradiotherapy

pCR:

Pathologic complete response

pN:

Pathologic N stage

pT:

Pathologic N stage

RF:

Random forest

SVM:

Support vector machine

T2WI:

T2-weighted imaging

TME:

Total mesorectal excision

TRG:

Tumour regression grade

VOI:

Volume of interest

XGBoost:

Extreme gradient boosting

References

  1. Cardoso R, Guo F, Heisser T et al (2021) Colorectal cancer incidence, mortality, and stage distribution in European countries in the colorectal cancer screening era: an international population-based study. Lancet Oncol 22:1002–1013

    Article  PubMed  Google Scholar 

  2. Araghi M, Soerjomataram I, Jenkins M et al (2019) Global trends in colorectal cancer mortality: projections to the year 2035. Int J Cancer 144:2992–3000

    Article  CAS  PubMed  Google Scholar 

  3. Siegel RL, Miller KD, Fuchs HE, Jemal A (2022) Cancer statistics, 2022. CA Cancer J Clin 72:7–33

  4. Oronsky B, Reid T, Larson C, Knox SJ (2020) Locally advanced rectal cancer: the past, present, and future. Semin Oncol 47:85–92

    Article  PubMed  Google Scholar 

  5. Polanco PM, Mokdad AA, Zhu H, Choti MA, Huerta S (2018) Association of adjuvant chemotherapy with overall survival in patients with rectal cancer and pathologic complete response following neoadjuvant chemotherapy and resection. JAMA Oncol 4:938–943

    Article  PubMed  PubMed Central  Google Scholar 

  6. Maas M, Nelemans PJ, Valentini V et al (2010) Long-term outcome in patients with a pathological complete response after chemoradiation for rectal cancer: a pooled analysis of individual patient data. Lancet Oncol 11:835–844

    Article  PubMed  Google Scholar 

  7. Martin S, Heneghan H, Winter D (2012) Systematic review and meta-analysis of outcomes following pathological complete response to neoadjuvant chemoradiotherapy for rectal cancer. J Br Surg 99:918–928

    Article  CAS  Google Scholar 

  8. Smith JJ, Strombom P, Chow OS et al (2019) Assessment of a watch-and-wait strategy for rectal cancer in patients with a complete response after neoadjuvant therapy. JAMA Oncol 5:e185896

    Article  PubMed  PubMed Central  Google Scholar 

  9. Dossa F, Chesney TR, Acuna SA, Baxter NN (2017) A watch-and-wait approach for locally advanced rectal cancer after a clinical complete response following neoadjuvant chemoradiation: a systematic review and meta-analysis. Lancet Gastroenterol Hepatol 2:501–513

    Article  PubMed  Google Scholar 

  10. van der Valk MJ, Hilling DE, Bastiaannet E et al (2018) Long-term outcomes of clinical complete responders after neoadjuvant treatment for rectal cancer in the International Watch & Wait Database (IWWD): an international multicentre registry study. Lancet 391:2537–2545

    Article  PubMed  Google Scholar 

  11. Roh MS, Colangelo LH, O’Connell MJ et al (2009) Preoperative multimodality therapy improves disease-free survival in patients with carcinoma of the rectum: NSABP R-03. J Clin Oncol 27:5124

    Article  PubMed  PubMed Central  Google Scholar 

  12. Al-Sukhni E, Attwood K, Mattson DM, Gabriel E, Nurkin SJ (2016) Predictors of pathologic complete response following neoadjuvant chemoradiotherapy for rectal cancer. Ann Surg Oncol 23:1177–1186

    Article  PubMed  Google Scholar 

  13. Meng X, Xia W, Xie P et al (2019) Preoperative radiomic signature based on multiparametric magnetic resonance imaging for noninvasive evaluation of biological characteristics in rectal cancer. Eur Radiol 29:3200–3209

    Article  PubMed  Google Scholar 

  14. Huang CM, Huang MY, Huang CW et al (2020) Machine learning for predicting pathological complete response in patients with locally advanced rectal cancer after neoadjuvant chemoradiotherapy. Sci Rep 10:12555

    Article  PubMed  PubMed Central  Google Scholar 

  15. Zhao X, Xie P, Wang M et al (2020) Deep learning-based fully automated detection and segmentation of lymph nodes on multiparametric-mri for rectal cancer: a multicentre study. EBioMedicine 56:102780

    Article  PubMed  PubMed Central  Google Scholar 

  16. Liang M, Cai Z, Zhang H et al (2019) Machine learning-based analysis of rectal cancer MRI radiomics for prediction of metachronous liver metastasis. Acad Radiol 26:1495–1504

    Article  PubMed  Google Scholar 

  17. Jin C, Yu H, Ke J et al (2021) Predicting treatment response from longitudinal images using multi-task deep learning. Nat Commun 12:1851

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Cui Y, Yang X, Shi Z et al (2019) Radiomics analysis of multiparametric MRI for prediction of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Eur Radiol 29:1211–1220

    Article  PubMed  Google Scholar 

  19. Nie K, Shi L, Chen Q et al (2016) Rectal cancer: assessment of neoadjuvant chemoradiation outcome based on radiomics of multiparametric MRI. Clin Cancer Res 22:5256–5264

    Article  PubMed  Google Scholar 

  20. Yi X, Pei Q, Zhang Y et al (2019) MRI-based radiomics predicts tumor response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Front Oncol 9:552

    Article  PubMed  PubMed Central  Google Scholar 

  21. Petresc B, Lebovici A, Caraiani C, Feier D, Graur F, Buruian M (2020) Pre-treatment T2-WI based radiomics features for prediction of locally advanced rectal cancer non-response to neoadjuvant chemoradiotherapy: a preliminary study. Cancers (Basel) 12:1894

  22. Liu Z, Zhang XY, Shi YJ et al (2017) Radiomics analysis for evaluation of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Clin Cancer Res 23:7253–7262

    Article  CAS  PubMed  Google Scholar 

  23. Ho SY, Phua K, Wong L, Bin Goh WW (2020) Extensions of the external validation for checking learned model interpretability and generalizability. Patterns (N Y) 1:100129

    Article  PubMed  Google Scholar 

  24. Orlhac F, Lecler A, Savatovski J et al (2021) How can we combat multicenter variability in MR radiomics? Validation of a correction procedure. Eur Radiol 31:2272–2280

    Article  PubMed  Google Scholar 

  25. Da-Ano R, Visvikis D, Hatt M (2020) Harmonization strategies for multicenter radiomics investigations. Phys Med Biol 65:24tr02

    Article  CAS  PubMed  Google Scholar 

  26. Kociołek M, Strzelecki M, Obuchowicz R (2020) Does image normalization and intensity resolution impact texture classification? Comput Med Imaging Graph 81:101716

    Article  PubMed  Google Scholar 

  27. Kotsiantis S, Zaharakis I, Pintelas P (2006) Machine learning: a review of classification and combining techniques. Artif Intell Rev 26:159–190

    Article  Google Scholar 

  28. Shaish H, Aukerman A, Vanguri R et al (2020) Radiomics of MRI for pretreatment prediction of pathologic complete response, tumor regression grade, and neoadjuvant rectal score in patients with locally advanced rectal cancer undergoing neoadjuvant chemoradiation: an international multicenter study. Eur Radiol 30:6263–6273

    Article  CAS  PubMed  Google Scholar 

  29. van Griethuysen JJM, Lambregts DMJ, Trebeschi S et al (2020) Radiomics performs comparable to morphologic assessment by expert radiologists for prediction of response to neoadjuvant chemoradiotherapy on baseline staging MRI in rectal cancer. Abdom Radiol (NY) 45:632–643

  30. Deist TM, Dankers F, Valdes G et al (2018) Machine learning algorithms for outcome prediction in (chemo)radiotherapy: an empirical comparison of classifiers. Med Phys 45:3449–3459

    Article  PubMed  PubMed Central  Google Scholar 

  31. Nyúl LG, Udupa JK, Zhang X (2000) New variants of a method of MRI scale standardization. IEEE Trans Med Imaging 19:143–150

    Article  PubMed  Google Scholar 

  32. Zhang X, Zhong L, Zhang B et al (2019) The effects of volume of interest delineation on MRI-based radiomics analysis: evaluation with two disease groups. Cancer Imaging 19:89

    Article  PubMed  PubMed Central  Google Scholar 

  33. Johnson WE, Li C, Rabinovic A (2007) Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics 8:118–127

    Article  PubMed  Google Scholar 

  34. Shi L, Zhang Y, Nie K et al (2019) Machine learning for prediction of chemoradiation therapy response in rectal cancer using pre-treatment and mid-radiation multi-parametric MRI. Magn Reson Imaging 61:33–40

    Article  PubMed  PubMed Central  Google Scholar 

  35. Antunes JT, Ofshteyn A, Bera K et al (2020) Radiomic features of primary rectal cancers on baseline T2-weighted MRI are associated with pathologic complete response to neoadjuvant chemoradiation: a multisite study. J Magn Reson Imaging 52:1531–1541

    Article  PubMed  PubMed Central  Google Scholar 

  36. Bulens P, Couwenberg A, Intven M et al (2020) Predicting the tumor response to chemoradiotherapy for rectal cancer: model development and external validation using MRI radiomics. Radiother Oncol 142:246–252

    Article  CAS  PubMed  Google Scholar 

  37. Buch K, Kuno H, Qureshi MM, Li B, Sakai O (2018) Quantitative variations in texture analysis features dependent on MRI scanning parameters: a phantom model. J Appl Clin Med Phys 19:253–264

    Article  PubMed  PubMed Central  Google Scholar 

  38. Ford J, Dogan N, Young L, Yang F (2018) Quantitative radiomics: impact of pulse sequence parameter selection on MRI-based textural features of the brain. Contrast Media Mol Imaging:2018:1729071

  39. Guillemaud R, Brady M (1997) Estimating the bias field of MR images. IEEE Trans Med Imaging 16:238–251

    Article  CAS  PubMed  Google Scholar 

  40. Khatami A, Khosravi A, Nguyen T, Lim CP, Nahavandi S (2017) Medical image analysis using wavelet transform and deep belief networks. Exp Syst Appl 86:190–198

    Article  Google Scholar 

  41. Li L, Mao F, Qian W, Clarke LP (1997) Wavelet transform for directional feature extraction in medical imaging. Proceedings of international conference on image processing. IEEE, pp 500-503

  42. Kuo L-J, Liu M-C, Jian JJ-M et al (2007) Is final TNM staging a predictor for survival in locally advanced rectal cancer after preoperative chemoradiation therapy? Ann Surg Oncol 14:2766–2772

    Article  PubMed  Google Scholar 

  43. Wallin U, Rothenberger D, Lowry A, Luepker R, Mellgren A (2013) CEA–a predictor for pathologic complete response after neoadjuvant therapy for rectal cancer. Dis Colon Rectum 56:859–868

    Article  PubMed  Google Scholar 

  44. Huang C-M, Huang C-W, Ma C-J et al (2020) Predictive value of FOLFOX-based regimen, long interval, hemoglobin levels and clinical negative nodal status, and postchemoradiotherapy CEA levels for pathological complete response in patients with locally advanced rectal cancer after neoadjuvant chemoradiotherapy. J Oncol 2020:9437684

  45. Gambacorta MA, Masciocchi C, Chiloiro G et al (2021) Timing to achieve the highest rate of pCR after preoperative radiochemotherapy in rectal cancer: a pooled analysis of 3085 patients from 7 randomized trials. Radiother Oncol 154:154–160

    Article  CAS  PubMed  Google Scholar 

  46. Allegra CJ, Yothers G, O’Connell MJ et al (2015) Neoadjuvant 5-FU or capecitabine plus radiation with or without oxaliplatin in rectal cancer patients: a phase III randomized clinical trial. J Natl Cancer Inst 107:djv248

    Article  PubMed  PubMed Central  Google Scholar 

  47. Huang C-M, Huang M-Y, Huang C-W et al (2020) Machine learning for predicting pathological complete response in patients with locally advanced rectal cancer after neoadjuvant chemoradiotherapy. Sci Rep 10:1–10

    Google Scholar 

  48. Ali J, Khan R, Ahmad N, Maqsood I (2012) Random forests and decision trees. Int J Comput Sci Issues (IJCSI) 9:272

    Google Scholar 

  49. Guo L, Ma Y, Cukic B, Singh H (2004) Robust prediction of fault-proneness by random forests. 15th international symposium on software reliability engineering. IEEE, pp 417-428

  50. Razzak MI, Naz S, Zaib A (2018) Deep learning for medical image processing: overview, challenges and the future. In: Dey N, Ashour AS, Borra S, (eds) Classification in BioApps: automation of decision making. Springer International Publishing, Cham, pp 323–350

Download references

Funding

This study has received funding from the Youth Talent Project of The Second Affiliated Hospital of Guangzhou University of Chinese Medicine (Grant Number ZY2022YL05).

Author information

Author notes

  1. Qiurong Wei and Zeli Chen share equal contribution.

Authors and Affiliations

  1. Department of Radiology, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, 510120, China

    Qiurong Wei, Weicui Chen, Liting Mao & Xian Liu

  2. Guangdong Provincial Key Laboratory of Medical Image Processing, School of Biomedical Engineering, Southern Medical University, Guangzhou, 510515, China

    Zeli Chen, Liming Zhong & Wei Yang

  3. Department of Radiology, First Affiliated Hospital of Guangxi Medical University, Nanning, 530021, China

    Yehuan Tang

  4. Department of Pathology, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, 510120, China

    Shaowei Hu

  5. Department of Medical Imaging, Nanfang Hospital, Southern Medical University, Guangzhou, 510515, China

    Yuankui Wu

  6. Clinical Science, Philips Healthcare, Guangzhou, China

    Kan Deng

Authors
  1. Qiurong Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zeli Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yehuan Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weicui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liming Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Liting Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shaowei Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yuankui Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Kan Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Wei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Xian Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Wei Yang or Xian Liu.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Xian Liu Ph.D., M.D., and Wei Yang Ph.D., M.D.

Conflict of interest

One of the authors (Kan Deng) is an employee of Philips Healthineers. The remaining authors declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was not required for this study because of the retrospective nature of the study.

Ethical approval

Institutional review board approval was obtained.

Methodology

• retrospective

• diagnostic or prognostic study

• performed at two institutions

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, Q., Chen, Z., Tang, Y. et al. External validation and comparison of MR-based radiomics models for predicting pathological complete response in locally advanced rectal cancer: a two-centre, multi-vendor study. Eur Radiol 33, 1906–1917 (2023). https://doi.org/10.1007/s00330-022-09204-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-022-09204-5

Keywords

Search

Navigation