Abstract
Evaluating treatment response is essential in patients who develop colorectal liver metastases to decide the necessity for second-line treatment or the admissibility for surgery. Currently, RECIST1.1 is the most widely used criteria in this context. However, it involves time-consuming, precise manual delineation and size measurement of main liver metastases from Computed Tomography (CT) images. Moreover, an early prediction of the treatment response given a specific chemotherapy regimen and the initial CT scan would be of tremendous use to clinicians. To overcome these challenges, this paper proposes a deep learning-based treatment response assessment pipeline and its extension for prediction purposes. Based on a newly designed 3D Siamese classification network, our method assigns a response group to patients given CT scans from two consecutive follow-ups during the treatment period. Further, we extended the network to predict the treatment response given only the image acquired at first time point. The pipelines are trained on the PRODIGE20 dataset collected from a phase-II multi-center clinical trial in colorectal cancer with liver metastases and exploit an in-house dataset to integrate metastases delineations derived from a U-Net inspired network as additional information. Our approach achieves overall accuracies of 94.94% and 86.86% for treatment response assessment and early prediction respectively, suggesting that both treatment response assessment and prediction issues can be effectively solved with deep learning.
Keywords
- Treatment response
- Siamese network
- Colorectal cancer
- Liver metastases
- Longitudinal analysis
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Kow, A.W.C.: Hepatic metastasis from colorectal cancer. J. Gastrointest. Oncol. 10(6), 1274 (2019)
van Gestel, Y., et al.: Patterns of metachronous metastases after curative treatment of colorectal cancer. Cancer Epidemiol. 38(4), 448–454 (2014)
Eisenhauer, E.A., Therasse, P., Bogaerts, J., Schwartz, L.H., et al.: New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur. J. Cancer 45(2), 228–247 (2009)
Kuhl, C.K., et al.: Validity of RECIST version 1.1 for response assessment in metastatic cancer: a prospective, multireader study. Radiology 290(2), 349–356 (2019)
Dohan, A., et al.: Early evaluation using a radiomic signature of unresectable hepatic metastases to predict outcome in patients with colorectal cancer treated with FOLFIRI and bevacizumab. Gut 69(3), 531–539 (2020)
Topol, E.J.: High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 25(1), 44–56 (2019)
Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017)
Gulshan, V., et al.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316(22), 2402–2410 (2016)
Lee, H., et al.: An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets. Nat. Biomed. Eng. 3(3), 173–182 (2019)
Maaref, A., et al.: Predicting the response to FOLFOX-based chemotherapy regimen from untreated liver metastases on baseline CT: a deep neural network approach. J. Digit. Imaging 33(4), 937–945 (2020)
Graffy, P.M., Liu, J., Pickhardt, P.J., Burns, J.E., Yao, J., Summers, R.M.: Deep learning-based muscle segmentation and quantification at abdominal CT: application to a longitudinal adult screening cohort for sarcopenia assessment. Br. J. Radiol. 92(1100), 20190327 (2019)
Jin, C., et al.: Predicting treatment response from longitudinal images using multi-task deep learning. Nat. Commun. 12(1), 1–11 (2021)
Cao, Y., et al.: Longitudinal assessment of COVID-19 using a deep learning-based quantitative CT pipeline: illustration of two cases. Radiol.: Cardiothorac. Imaging 2(2) (2020)
Zhu, H.-B., et al.: Deep learning-assisted magnetic resonance imaging prediction of tumor response to chemotherapy in patients with colorectal liver metastases. Int. J. Cancer 148, 1717–1730 (2021)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)
Conze, P.-H., Brochard, S., Burdin, V., Sheehan, F.T., Pons, C.: Healthy versus pathological learning transferability in shoulder muscle MRI segmentation using deep convolutional encoder-decoders. Comput. Med. Imaging Graph. 83, 101733 (2020)
Conze, P.-H., et al.: Abdominal multi-organ segmentation with cascaded convolutional and adversarial deep networks. Artif. Intell. Med. 117, 102109 (2021)
Yan, Y., et al.: Longitudinal detection of diabetic retinopathy early severity grade changes using deep learning. In: Fu, H., Garvin, M.K., MacGillivray, T., Xu, Y., Zheng, Y. (eds.) OMIA 2021. LNCS, vol. 12970, pp. 11–20. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87000-3_2
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)
Roy, A.G., Navab, N., Wachinger, C.: Concurrent spatial and channel ‘squeeze & excitation’ in fully convolutional networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 421–429. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_48
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Aparicio, T., et al.: Bevacizumab+chemotherapy versus chemotherapy alone in elderly patients with untreated metastatic colorectal cancer: a randomized phase II trial-PRODIGE 20 study results. Ann. Oncol. 29(1), 133–138 (2018)
Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(2), 281–305 (2012)
Rücker, G., Schumacher, M.: Summary ROC curve based on a weighted Youden index for selecting an optimal cutpoint in meta-analysis of diagnostic accuracy. Stat. Med. 29(30), 3069–3078 (2010)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: IEEE International Conference on Computer Vision, pp. 618–626 (2017)
Acknowledgments
This work was partially funded by Ligue contre le Cancer. The PRODIGE20 dataset was provided with the support from FFCD (Fédération Francophone de Cancérologie Digestive). The authors would like to thank all the PRODIGE20 investigators as well as A. Dohan from AP-HP for fruitful discussions.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Compliance with Ethical Standards
CT data acquisition was performed in line with the principles of the Declaration of Helsinki. Ethical approval was provided by the Ethics Committee CPP EST I DIJON n\(^\circ \)100109 in Jan, 26 2010 and registered in clinicaltrials.gov with number NCT01900717. Authors declare that they do not have any conflicts of interest.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Islam, M.M. et al. (2022). Deep Treatment Response Assessment and Prediction of Colorectal Cancer Liver Metastases. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13433. Springer, Cham. https://doi.org/10.1007/978-3-031-16437-8_46
Download citation
DOI: https://doi.org/10.1007/978-3-031-16437-8_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16436-1
Online ISBN: 978-3-031-16437-8
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://miccai.org/
