Skip to main content

Radiomics analysis of contrast-enhanced CT for classification of hepatic focal lesions in colorectal cancer patients: its limitations compared to radiologists

European Radiology volume 31, pages 8786–8796 (2021)Cite this article

Abstract

Objective

To evaluate diagnostic performance of a radiomics model for classifying hepatic cyst, hemangioma, and metastasis in patients with colorectal cancer (CRC) from portal-phase abdominopelvic CT images.

Methods

This retrospective study included 502 CRC patients who underwent contrast-enhanced CT and contrast-enhanced liver MRI between January 2005 and December 2010. Portal-phase CT images of training (n = 386) and validation (n = 116) cohorts were used to develop a radiomics model for differentiating three classes of liver lesions. Among multiple handcrafted features, the feature selection was performed using ReliefF method, and random forest classifiers were used to train the selected features. Diagnostic performance of the developed model was compared with that of four radiologists. A subgroup analysis was conducted based on lesion size.

Results

The radiomics model demonstrated significantly lower overall and hemangioma- and metastasis-specific polytomous discrimination index (PDI) (overall, 0.8037; hemangioma-specific, 0.6653; metastasis-specific, 0.8027) than the radiologists (overall, 0.9622–0.9680; hemangioma-specific, 0.9452–0.9630; metastasis-specific, 0.9511–0.9869). For subgroup analysis, the PDI of the radiomics model was different according to the lesion size (< 10 mm, 0.6486; ≥ 10 mm, 0.8264) while that of the radiologists was relatively maintained. For classifying metastasis from benign lesions, the radiomics model showed excellent diagnostic performance, with an accuracy of 84.36% and an AUC of 0.9426.

Conclusion

Albeit inferior to the radiologists, the radiomics model achieved substantial diagnostic performance when differentiating hepatic lesions from portal-phase CT images of CRC patients. This model was limited particularly to classifying hemangiomas and subcentimeter lesions.

Key Points

• Albeit inferior to the radiologists, the radiomics model could differentiate cyst, hemangioma, and metastasis with substantial diagnostic performance using portal-phase CT images of colorectal cancer patients.

• The radiomics model demonstrated limitations especially in classifying hemangiomas and subcentimeter liver lesions.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

Abbreviations

CCP:

Correct classification percentage

CRC:

Colorectal cancer

CRLM:

Colorectal liver metastasis

PDI:

Polytomous discrimination index

RF:

Random forest

References

  1. Manfredi S, Lepage C, Hatem C, Coatmeur O, Faivre J, Bouvier A-M (2006) Epidemiology and management of liver metastases from colorectal cancer. Ann Surg 244:254–259

    Article  Google Scholar 

  2. Zarour LR, Anand S, Billingsley KG et al (2017) Colorectal cancer liver metastasis: evolving paradigms and future directions. Cell Mol Gastroenterol Hepatol 3:163–173

    Article  Google Scholar 

  3. Bengtsson G, Carlsson G, Hafstrom L, Jonsson PE (1981) Natural history of patients with untreated liver metastases from colorectal cancer. Am J Surg 141:586–589

    CAS  Article  Google Scholar 

  4. Abdalla EK, Vauthey J-N, Ellis LM et al (2004) Recurrence and outcomes following hepatic resection, radiofrequency ablation, and combined resection/ablation for colorectal liver metastases. Ann Surg 239:818–827

    Article  Google Scholar 

  5. Jones RP, Kokudo N, Folprecht G et al (2016) Colorectal liver metastases: a critical review of state of the art. Liver Cancer 6:66–71

    Article  Google Scholar 

  6. Limkin EJ, Sun R, Dercle L et al (2017) Promises and challenges for the implementation of computational medical imaging (radiomics) in oncology. Ann Oncol 28:1191–1206

    CAS  Article  Google Scholar 

  7. Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: images are more than pictures, they are data. Radiology 278:563–577

    Article  Google Scholar 

  8. Alahmer H, Ahmed A (2016) Computer-aided classification of liver lesions from CT images based on multiple ROI. Procedia Comput Sci 90:80–86

    Article  Google Scholar 

  9. Gletsos M, Mougiakakou SG, Matsopoulos GK, Nikita KS, Nikita AS, Kelekis D (2003) A computer-aided diagnostic system to characterize CT focal liver lesions: design and optimization of a neural network classifier. IEEE Trans Inf Technol Biomed 7:153–162

    Article  Google Scholar 

  10. Chang CC, Chen HH, Chang YC et al (2017) Computer-aided diagnosis of liver tumors on computed tomography images. Comput Methods Prog Biomed 145:45–51

    Article  Google Scholar 

  11. Song S, Li Z, Niu L et al (2019) Hypervascular hepatic focal lesions on dynamic contrast-enhanced CT: preliminary data from arterial phase scans texture analysis for classification. Clin Radiol 74:653.e11–653.e18

    CAS  Article  Google Scholar 

  12. Park SH, Han K (2018) Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology 286:800–809

    Article  Google Scholar 

  13. Park JE, Park SY, Kim HJ, Kim HS (2019) Reproducibility and generalizability in radiomics modeling: possible strategies in radiologic and statistical perspectives. Korean J Radiol 20:1124–1137

    Article  Google Scholar 

  14. Tirumani SH, Kim KW, Nishino M et al (2014) Update on the role of imaging in management of metastatic colorectal cancer. Radiographics 34:1908–1928

    Article  Google Scholar 

  15. Floriani I, Torri V, Rulli E et al (2010) Performance of imaging modalities in diagnosis of liver metastases from colorectal cancer: a systematic review and meta-analysis. J Magn Reson Imaging 31:19–31

    Article  Google Scholar 

  16. Rojas Llimpe FL, Di Fabio F, Ercolani G et al (2014) Imaging in resectable colorectal liver metastasis patients with or without preoperative chemotherapy: results of the PROMETEO-01 study. Br J Cancer 111:667–673

    CAS  Article  Google Scholar 

  17. Sivesgaard K, Larsen LP, Sorensen M et al (2018) Diagnostic accuracy of CE-CT, MRI and FDG PET/CT for detecting colorectal cancer liver metastases in patients considered eligible for hepatic resection and/or local ablation. Eur Radiol 28:4735–4747

    Article  Google Scholar 

  18. Kim HJ, Lee SS, Byun JH et al (2015) Incremental value of liver MR imaging in patients with potentially curable colorectal hepatic metastasis detected at CT: a prospective comparison of diffusion-weighted imaging, gadoxetic acid-enhanced MR imaging, and a combination of both MR techniques. Radiology 274:712–722

    Article  Google Scholar 

  19. Niekel MC, Bipat S, Stoker J (2010) Diagnostic imaging of colorectal liver metastases with CT, MR imaging, FDG PET, and/or FDG PET/CT: a meta-analysis of prospective studies including patients who have not previously undergone treatment. Radiology 257:674–684

    Article  Google Scholar 

  20. Zech CJ, Korpraphong P, Huppertz A et al (2014) Randomized multicentre trial of gadoxetic acid-enhanced MRI versus conventional MRI or CT in the staging of colorectal cancer liver metastases. Br J Surg 101:613–621

    CAS  Article  Google Scholar 

  21. Jhaveri KS, Fischer SE, Hosseini-Nik H et al (2017) Prospective comparison of gadoxetic acid-enhanced liver MRI and contrast-enhanced CT with histopathological correlation for preoperative detection of colorectal liver metastases following chemotherapy and potential impact on surgical plan. HPB (Oxford) 19:992–1000

    Article  Google Scholar 

  22. McAuliffe MJ, Lalonde FM, McGarry D, Gandler W, Csaky K, Trus BL (2001) Medical image processing, analysis and visualization in clinical research. Proceedings 14th IEEE Symposium on Computer-Based Medical Systems CBMS 2001. https://doi.org/10.1109/CBMS.2001.941749

  23. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    CAS  Article  Google Scholar 

  24. Lee HS, Hong H, Jung DC, Park S, Kim J (2017) Differentiation of fat-poor angiomyolipoma from clear cell renal cell carcinoma in contrast-enhanced MDCT images using quantitative feature classification. Med Phys 44:3604–3614

    CAS  Article  Google Scholar 

  25. Lee H, Hong H, Kim J, Jung DC (2018) Deep feature classification of angiomyolipoma without visible fat and renal cell carcinoma in abdominal contrast-enhanced CT images with texture image patches and hand-crafted feature concatenation. Med Phys 45:1550–1561

    Article  Google Scholar 

  26. Robnik-Sikonja M, Kononenko I (1997) An adaptation of Relief for attribute estimation in regression. Proceedings of the Fourteenth International Conference on Machine Learning (ICML’97). Morgan Kaufmann Publishers Inc., San Francisco, CA

    Google Scholar 

  27. Ho TK (1995) Random decision forests Proceedings of 3rd international conference on document analysis and recognition. https://doi.org/10.1109/ICDAR.1995.598994

  28. Van Calster B, Van Belle V, Vergouwe Y, Timmerman D, Van Huffel S, Steyerberg EW (2012) Extending the c-statistic to nominal polytomous outcomes: the Polytomous Discrimination Index. Stat Med 31:2610–2626

    Article  Google Scholar 

  29. Dreižienė L, Dučinskas K, Paulionienė L (2015) Correct classification rates in multi-category discriminant analysis of spatial Gaussian data. Open J Stat 5:21–26

    Article  Google Scholar 

  30. Huang YL, Chen JH, Shen WC (2006) Diagnosis of hepatic tumors with texture analysis in nonenhanced computed tomography images. Acad Radiol 13:713–720

    Article  Google Scholar 

  31. Mougiakakou SG, Valavanis IK, Nikita A, Nikita KS (2007) Differential diagnosis of CT focal liver lesions using texture features, feature selection and ensemble driven classifiers. Artif Intell Med 41:25–37

    Article  Google Scholar 

  32. Acharya UR, Koh JEW, Hagiwara Y et al (2018) Automated diagnosis of focal liver lesions using bidirectional empirical mode decomposition features. Comput Biol Med 94:11–18

    Article  Google Scholar 

  33. Yasaka K, Akai H, Abe O, Kiryu S (2018) Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology 286:887–896

    Article  Google Scholar 

  34. Ye J, Sun Y, Wang S, Gu L, Qian L, Xu J (2009) Multi-phase CT image based hepatic lesion diagnosis by SVM 2009. 2nd International Conference on Biomedical Engineering and Informatics. https://doi.org/10.1109/BMEI.2009.5304774

  35. Klotz T, Montoriol PF, Da Ines D, Petitcolin V, Joubert-Zakeyh J, Garcier JM (2013) Hepatic haemangioma: common and uncommon imaging features. Diagn Interv Imaging 94:849–859

    CAS  Article  Google Scholar 

  36. Caseiro-Alves F, Brito J, Araujo AE et al (2007) Liver haemangioma: common and uncommon findings and how to improve the differential diagnosis. Eur Radiol 17:1544–1554

    Article  Google Scholar 

  37. Khalil HI, Patterson SA, Panicek DM (2005) Hepatic lesions deemed too small to characterize at CT: prevalence and importance in women with breast cancer. Radiology 235:872–878

    Article  Google Scholar 

  38. Jones EC, Chezmar JL, Nelson RC, Bernardino ME (1992) The frequency and significance of small (less than or equal to 15 mm) hepatic lesions detected by CT. AJR Am J Roentgenol 158:535–539

    CAS  Article  Google Scholar 

  39. Schwartz LH, Gandras EJ, Colangelo SM, Ercolani MC, Panicek DM (1999) Prevalence and importance of small hepatic lesions found at CT in patients with cancer. Radiology 210:71–74

    CAS  Article  Google Scholar 

  40. Lim GH, Koh DC, Cheong WK, Wong KS, Tsang CB (2009) Natural history of small, “indeterminate” hepatic lesions in patients with colorectal cancer. Dis Colon Rectum 52:1487–1491

    Article  Google Scholar 

  41. Jang HJ, Lim HK, Lee WJ, Lee SJ, Yun JY, Choi D (2002) Small hypoattenuating lesions in the liver on single-phase helical CT in preoperative patients with gastric and colorectal cancer: prevalence, significance, and differentiating features. J Comput Assist Tomogr 26:718–724

    Article  Google Scholar 

  42. Dankerl P, Cavallaro A, Tsymbal A et al (2013) A retrieval-based computer-aided diagnosis system for the characterization of liver lesions in CT scans. Acad Radiol 20:1526–1534

    Article  Google Scholar 

  43. Hamm CA, Wang CJ, Savic LJ et al (2019) Deep learning for liver tumor diagnosis part I: development of a convolutional neural network classifier for multi-phasic MRI. Eur Radiol 29:3338–3347

    Article  Google Scholar 

  44. Oakden-Rayner L, Palmer L (2020) Docs are ROCs: a simple off-the-shelf approach for estimating average human performance in diagnostic studies. arXiv preprint. Available via https://arxiv.org/abs/2009.11060v2. Accessed 15 Jan 2021

Download references

Funding

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (Ministry of Science and ICT) (2017R1D1A1B03029631).

Author information

Authors and Affiliations

  1. Department of Radiology, Research Institute of Radiological Science, Severance Hospital, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea

    Heejin Bae, Sungwon Kim, Kyunghwa Han, Hyungjin Rhee, Dong-kyu Kim, Hyuk Kwon & Joon Seok Lim

  2. School of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea

    Hansang Lee

  3. Department of Software Convergence, College of Interdisciplinary Studies for Emerging Industries, Seoul Women’s University, Seoul, Republic of Korea

    Helen Hong

Authors
  1. Heejin Bae
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hansang Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sungwon Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kyunghwa Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hyungjin Rhee
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dong-kyu Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hyuk Kwon
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Helen Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Joon Seok Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joon Seok Lim.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Dr. Joon Seok Lim.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

Dr. Kyunghwa Han, one of the authors, kindly provided statistical advice for this manuscript.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

• retrospective

• diagnostic or prognostic study

• performed at one institution

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

ESM 1

(DOCX 3.13 mb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bae, H., Lee, H., Kim, S. et al. Radiomics analysis of contrast-enhanced CT for classification of hepatic focal lesions in colorectal cancer patients: its limitations compared to radiologists. Eur Radiol 31, 8786–8796 (2021). https://doi.org/10.1007/s00330-021-07877-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-021-07877-y

Keywords

  • Radiomics
  • Multidetector computed tomography
  • Colorectal cancer
  • Liver neoplasms
  • Classification