Abstract
Objective
To evaluate diagnostic performance of a radiomics model for classifying hepatic cyst, hemangioma, and metastasis in patients with colorectal cancer (CRC) from portal-phase abdominopelvic CT images.
Methods
This retrospective study included 502 CRC patients who underwent contrast-enhanced CT and contrast-enhanced liver MRI between January 2005 and December 2010. Portal-phase CT images of training (n = 386) and validation (n = 116) cohorts were used to develop a radiomics model for differentiating three classes of liver lesions. Among multiple handcrafted features, the feature selection was performed using ReliefF method, and random forest classifiers were used to train the selected features. Diagnostic performance of the developed model was compared with that of four radiologists. A subgroup analysis was conducted based on lesion size.
Results
The radiomics model demonstrated significantly lower overall and hemangioma- and metastasis-specific polytomous discrimination index (PDI) (overall, 0.8037; hemangioma-specific, 0.6653; metastasis-specific, 0.8027) than the radiologists (overall, 0.9622–0.9680; hemangioma-specific, 0.9452–0.9630; metastasis-specific, 0.9511–0.9869). For subgroup analysis, the PDI of the radiomics model was different according to the lesion size (< 10 mm, 0.6486; ≥ 10 mm, 0.8264) while that of the radiologists was relatively maintained. For classifying metastasis from benign lesions, the radiomics model showed excellent diagnostic performance, with an accuracy of 84.36% and an AUC of 0.9426.
Conclusion
Albeit inferior to the radiologists, the radiomics model achieved substantial diagnostic performance when differentiating hepatic lesions from portal-phase CT images of CRC patients. This model was limited particularly to classifying hemangiomas and subcentimeter lesions.
Key Points
• Albeit inferior to the radiologists, the radiomics model could differentiate cyst, hemangioma, and metastasis with substantial diagnostic performance using portal-phase CT images of colorectal cancer patients.
• The radiomics model demonstrated limitations especially in classifying hemangiomas and subcentimeter liver lesions.
Similar content being viewed by others
Abbreviations
- CCP:
-
Correct classification percentage
- CRC:
-
Colorectal cancer
- CRLM:
-
Colorectal liver metastasis
- PDI:
-
Polytomous discrimination index
- RF:
-
Random forest
References
Manfredi S, Lepage C, Hatem C, Coatmeur O, Faivre J, Bouvier A-M (2006) Epidemiology and management of liver metastases from colorectal cancer. Ann Surg 244:254–259
Zarour LR, Anand S, Billingsley KG et al (2017) Colorectal cancer liver metastasis: evolving paradigms and future directions. Cell Mol Gastroenterol Hepatol 3:163–173
Bengtsson G, Carlsson G, Hafstrom L, Jonsson PE (1981) Natural history of patients with untreated liver metastases from colorectal cancer. Am J Surg 141:586–589
Abdalla EK, Vauthey J-N, Ellis LM et al (2004) Recurrence and outcomes following hepatic resection, radiofrequency ablation, and combined resection/ablation for colorectal liver metastases. Ann Surg 239:818–827
Jones RP, Kokudo N, Folprecht G et al (2016) Colorectal liver metastases: a critical review of state of the art. Liver Cancer 6:66–71
Limkin EJ, Sun R, Dercle L et al (2017) Promises and challenges for the implementation of computational medical imaging (radiomics) in oncology. Ann Oncol 28:1191–1206
Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: images are more than pictures, they are data. Radiology 278:563–577
Alahmer H, Ahmed A (2016) Computer-aided classification of liver lesions from CT images based on multiple ROI. Procedia Comput Sci 90:80–86
Gletsos M, Mougiakakou SG, Matsopoulos GK, Nikita KS, Nikita AS, Kelekis D (2003) A computer-aided diagnostic system to characterize CT focal liver lesions: design and optimization of a neural network classifier. IEEE Trans Inf Technol Biomed 7:153–162
Chang CC, Chen HH, Chang YC et al (2017) Computer-aided diagnosis of liver tumors on computed tomography images. Comput Methods Prog Biomed 145:45–51
Song S, Li Z, Niu L et al (2019) Hypervascular hepatic focal lesions on dynamic contrast-enhanced CT: preliminary data from arterial phase scans texture analysis for classification. Clin Radiol 74:653.e11–653.e18
Park SH, Han K (2018) Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology 286:800–809
Park JE, Park SY, Kim HJ, Kim HS (2019) Reproducibility and generalizability in radiomics modeling: possible strategies in radiologic and statistical perspectives. Korean J Radiol 20:1124–1137
Tirumani SH, Kim KW, Nishino M et al (2014) Update on the role of imaging in management of metastatic colorectal cancer. Radiographics 34:1908–1928
Floriani I, Torri V, Rulli E et al (2010) Performance of imaging modalities in diagnosis of liver metastases from colorectal cancer: a systematic review and meta-analysis. J Magn Reson Imaging 31:19–31
Rojas Llimpe FL, Di Fabio F, Ercolani G et al (2014) Imaging in resectable colorectal liver metastasis patients with or without preoperative chemotherapy: results of the PROMETEO-01 study. Br J Cancer 111:667–673
Sivesgaard K, Larsen LP, Sorensen M et al (2018) Diagnostic accuracy of CE-CT, MRI and FDG PET/CT for detecting colorectal cancer liver metastases in patients considered eligible for hepatic resection and/or local ablation. Eur Radiol 28:4735–4747
Kim HJ, Lee SS, Byun JH et al (2015) Incremental value of liver MR imaging in patients with potentially curable colorectal hepatic metastasis detected at CT: a prospective comparison of diffusion-weighted imaging, gadoxetic acid-enhanced MR imaging, and a combination of both MR techniques. Radiology 274:712–722
Niekel MC, Bipat S, Stoker J (2010) Diagnostic imaging of colorectal liver metastases with CT, MR imaging, FDG PET, and/or FDG PET/CT: a meta-analysis of prospective studies including patients who have not previously undergone treatment. Radiology 257:674–684
Zech CJ, Korpraphong P, Huppertz A et al (2014) Randomized multicentre trial of gadoxetic acid-enhanced MRI versus conventional MRI or CT in the staging of colorectal cancer liver metastases. Br J Surg 101:613–621
Jhaveri KS, Fischer SE, Hosseini-Nik H et al (2017) Prospective comparison of gadoxetic acid-enhanced liver MRI and contrast-enhanced CT with histopathological correlation for preoperative detection of colorectal liver metastases following chemotherapy and potential impact on surgical plan. HPB (Oxford) 19:992–1000
McAuliffe MJ, Lalonde FM, McGarry D, Gandler W, Csaky K, Trus BL (2001) Medical image processing, analysis and visualization in clinical research. Proceedings 14th IEEE Symposium on Computer-Based Medical Systems CBMS 2001. https://doi.org/10.1109/CBMS.2001.941749
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174
Lee HS, Hong H, Jung DC, Park S, Kim J (2017) Differentiation of fat-poor angiomyolipoma from clear cell renal cell carcinoma in contrast-enhanced MDCT images using quantitative feature classification. Med Phys 44:3604–3614
Lee H, Hong H, Kim J, Jung DC (2018) Deep feature classification of angiomyolipoma without visible fat and renal cell carcinoma in abdominal contrast-enhanced CT images with texture image patches and hand-crafted feature concatenation. Med Phys 45:1550–1561
Robnik-Sikonja M, Kononenko I (1997) An adaptation of Relief for attribute estimation in regression. Proceedings of the Fourteenth International Conference on Machine Learning (ICML’97). Morgan Kaufmann Publishers Inc., San Francisco, CA
Ho TK (1995) Random decision forests Proceedings of 3rd international conference on document analysis and recognition. https://doi.org/10.1109/ICDAR.1995.598994
Van Calster B, Van Belle V, Vergouwe Y, Timmerman D, Van Huffel S, Steyerberg EW (2012) Extending the c-statistic to nominal polytomous outcomes: the Polytomous Discrimination Index. Stat Med 31:2610–2626
Dreižienė L, Dučinskas K, Paulionienė L (2015) Correct classification rates in multi-category discriminant analysis of spatial Gaussian data. Open J Stat 5:21–26
Huang YL, Chen JH, Shen WC (2006) Diagnosis of hepatic tumors with texture analysis in nonenhanced computed tomography images. Acad Radiol 13:713–720
Mougiakakou SG, Valavanis IK, Nikita A, Nikita KS (2007) Differential diagnosis of CT focal liver lesions using texture features, feature selection and ensemble driven classifiers. Artif Intell Med 41:25–37
Acharya UR, Koh JEW, Hagiwara Y et al (2018) Automated diagnosis of focal liver lesions using bidirectional empirical mode decomposition features. Comput Biol Med 94:11–18
Yasaka K, Akai H, Abe O, Kiryu S (2018) Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology 286:887–896
Ye J, Sun Y, Wang S, Gu L, Qian L, Xu J (2009) Multi-phase CT image based hepatic lesion diagnosis by SVM 2009. 2nd International Conference on Biomedical Engineering and Informatics. https://doi.org/10.1109/BMEI.2009.5304774
Klotz T, Montoriol PF, Da Ines D, Petitcolin V, Joubert-Zakeyh J, Garcier JM (2013) Hepatic haemangioma: common and uncommon imaging features. Diagn Interv Imaging 94:849–859
Caseiro-Alves F, Brito J, Araujo AE et al (2007) Liver haemangioma: common and uncommon findings and how to improve the differential diagnosis. Eur Radiol 17:1544–1554
Khalil HI, Patterson SA, Panicek DM (2005) Hepatic lesions deemed too small to characterize at CT: prevalence and importance in women with breast cancer. Radiology 235:872–878
Jones EC, Chezmar JL, Nelson RC, Bernardino ME (1992) The frequency and significance of small (less than or equal to 15 mm) hepatic lesions detected by CT. AJR Am J Roentgenol 158:535–539
Schwartz LH, Gandras EJ, Colangelo SM, Ercolani MC, Panicek DM (1999) Prevalence and importance of small hepatic lesions found at CT in patients with cancer. Radiology 210:71–74
Lim GH, Koh DC, Cheong WK, Wong KS, Tsang CB (2009) Natural history of small, “indeterminate” hepatic lesions in patients with colorectal cancer. Dis Colon Rectum 52:1487–1491
Jang HJ, Lim HK, Lee WJ, Lee SJ, Yun JY, Choi D (2002) Small hypoattenuating lesions in the liver on single-phase helical CT in preoperative patients with gastric and colorectal cancer: prevalence, significance, and differentiating features. J Comput Assist Tomogr 26:718–724
Dankerl P, Cavallaro A, Tsymbal A et al (2013) A retrieval-based computer-aided diagnosis system for the characterization of liver lesions in CT scans. Acad Radiol 20:1526–1534
Hamm CA, Wang CJ, Savic LJ et al (2019) Deep learning for liver tumor diagnosis part I: development of a convolutional neural network classifier for multi-phasic MRI. Eur Radiol 29:3338–3347
Oakden-Rayner L, Palmer L (2020) Docs are ROCs: a simple off-the-shelf approach for estimating average human performance in diagnostic studies. arXiv preprint. Available via https://arxiv.org/abs/2009.11060v2. Accessed 15 Jan 2021
Funding
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (Ministry of Science and ICT) (2017R1D1A1B03029631).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Guarantor
The scientific guarantor of this publication is Dr. Joon Seok Lim.
Conflict of interest
The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.
Statistics and biometry
Dr. Kyunghwa Han, one of the authors, kindly provided statistical advice for this manuscript.
Informed consent
Written informed consent was waived by the Institutional Review Board.
Ethical approval
Institutional Review Board approval was obtained.
Methodology
• retrospective
• diagnostic or prognostic study
• performed at one institution
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
ESM 1
(DOCX 3.13 mb)
Rights and permissions
About this article
Cite this article
Bae, H., Lee, H., Kim, S. et al. Radiomics analysis of contrast-enhanced CT for classification of hepatic focal lesions in colorectal cancer patients: its limitations compared to radiologists. Eur Radiol 31, 8786–8796 (2021). https://doi.org/10.1007/s00330-021-07877-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00330-021-07877-y