European Radiology

, Volume 30, Issue 1, pp 558–570 | Cite as

Radiomics machine-learning signature for diagnosis of hepatocellular carcinoma in cirrhotic patients with indeterminate liver nodules

  • Fatima-Zohra MokraneEmail author
  • Lin Lu
  • Adrien Vavasseur
  • Philippe Otal
  • Jean-Marie Peron
  • Lyndon Luk
  • Hao Yang
  • Samy Ammari
  • Yvonne Saenger
  • Herve Rousseau
  • Binsheng Zhao
  • Lawrence H. Schwartz
  • Laurent Dercle
Imaging Informatics and Artificial Intelligence



To enhance clinician’s decision-making by diagnosing hepatocellular carcinoma (HCC) in cirrhotic patients with indeterminate liver nodules using quantitative imaging features extracted from triphasic CT scans.

Material and methods

We retrospectively analyzed 178 cirrhotic patients from 27 institutions, with biopsy-proven liver nodules classified as indeterminate using the European Association for the Study of the Liver (EASL) guidelines. Patients were randomly assigned to a discovery cohort (142 patients (pts.)) and a validation cohort (36 pts.). Each liver nodule was segmented on each phase of triphasic CT scans, and 13,920 quantitative imaging features (12 sets of 1160 features each reflecting the phenotype at one single phase or its change between two phases) were extracted. Using machine-learning techniques, the signature was trained and calibrated (discovery cohort), and validated (validation cohort) to classify liver nodules as HCC vs. non-HCC. Effects of segmentation and contrast enhancement quality were also evaluated.


Patients were predominantly male (88%) and CHILD A (65%). Biopsy was positive for HCC in 77% of patients. LI-RADS scores were not different between HCC and non-HCC patients. The signature included a single radiomics feature quantifying changes between arterial and portal venous phases: DeltaV-A_DWT1_LL_Variance-2D and reached area under the receiver operating characteristic curve (AUC) of 0.70 (95%CI 0.61–0.80) and 0.66 (95%CI 0.64–0.84) in discovery and validation cohorts, respectively. The signature was influenced neither by segmentation nor by contrast enhancement.


A signature using a single feature was validated in a multicenter retrospective cohort to diagnose HCC in cirrhotic patients with indeterminate liver nodules. Artificial intelligence could enhance clinicians’ decision by identifying a subgroup of patients with high HCC risk.

Key Points

• In cirrhotic patients with visually indeterminate liver nodules, expert visual assessment using current guidelines cannot accurately differentiate HCC from differential diagnoses. Current clinical protocols do not entail biopsy due to procedural risks. Radiomics can be used to non-invasively diagnose HCC in cirrhotic patients with indeterminate liver nodules, which could be leveraged to optimize patient management.

• Radiomics features contributing the most to a better characterization of visually indeterminate liver nodules include changes in nodule phenotype between arterial and portal venous phases: the “washout” pattern appraised visually using EASL and EASL guidelines.

• A clinical decision algorithm using radiomics could be applied to reduce the rate of cirrhotic patients requiring liver biopsy (EASL guidelines) or wait-and-see strategy (AASLD guidelines) and therefore improve their management and outcome.


Cirrhosis Radiomics Hepatocellular carcinoma 



Arterial CT phase


The American Association for the Study of Liver Diseases


Area under the receiver operating characteristic curve


Concordance correlation coefficient


Computed tomography


Dual phase defined by the change between imaging features extracted from two different CT phases (A-NC, V-A, V-NC)


Dual phase defined by the change between arterial and non-contrast CT phases, using standardized subtraction (delta 1)


Dual phase defined by the change between portal venous and arterial CT phases, using standardized subtraction (delta 1)


Dual phase defined by the change between portal venous and non-contrast CT phases, using standardized subtraction (delta 1)


Dual phase defined by the change between arterial and non-contrast CT phases, using direct subtraction (delta 2)


Dual phase defined by the change between portal venous and arterial CT phases, using direct subtraction (delta 2)


Dual phase defined by the change between portal venous and non-contrast CT phases, using direct subtraction (delta 2)


Dual phase defined by the change between arterial and non-contrast CT phases, using relative subtraction (delta 3)


Dual phase defined by the change between portal venous and arterial CT phases, using relative subtraction (delta 3)


Dual phase defined by the change between portal venous and non-contrast CT phases, using relative subtraction (delta 3)


Discrete wavelet frame in the low-pass channel radiomics feature


Dual-tree wavelet transform in the low-pass channel radiomics feature


The European Association for the Study of the Liver


Gray-level co-occurrence matrix radiomics feature


Hepatocellular carcinoma


K-Nearest neighbor machine-learning algorithm


Liver Imaging Reporting and Data System

LR-1 to 5 and M

LI-RADS categories, according to LI-RADS criteria version 2017


Magnetic resonance imaging


Non-alcoholic steatohepatitis


Non-contrast CT phase


Random forest machine-learning algorithm


Synthetic Minority Over-sampling Technique


Support vector machine machine-learning algorithm


Portal venous CT phase



This manuscript was revised by Qian Min, statistician. Sincere appreciation is expressed to Tavis Allison and Ronan Trépos for their assistance in the preparation of this manuscript.


This study was supported through research grants from Alain Rahmouni French Society of Radiology-CERF 2018 (FZM), Fondation Philanthropia (LD), and Fondation Nuovo-Soldati (LD).

Compliance with ethical standards


The scientific guarantor of this publication is Fatima-Zohra Mokrane.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

Qian Min, PhD, and Ronan Trépos, PhD, kindly provided statistical advice for this manuscript.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained (IRB form: AAAR9377).


• retrospective

• diagnostic

• multicenter study

Supplementary material

330_2019_6347_MOESM1_ESM.docx (23.2 mb)
ESM 1 (DOCX 23794 kb)


  1. 1.
    European Association for the Study of the Liver (2018) EASL clinical practice guidelines: management of hepatocellular carcinoma. J Hepatol CrossRefGoogle Scholar
  2. 2.
    El-Serag HB, Rudolph KL (2007) Hepatocellular carcinoma: epidemiology and molecular carcinogenesis. Gastroenterology 132:2557–2576CrossRefGoogle Scholar
  3. 3.
    Tsukuma H, Hiyama T, Tanaka S et al (1993) Risk factors for hepatocellular carcinoma among patients with chronic liver disease. N Engl J Med 328:1797–1801CrossRefGoogle Scholar
  4. 4.
    Sangiovanni A, Prati GM, Fasani P et al (2006) The natural history of compensated cirrhosis due to hepatitis C virus: a 17-year cohort study of 214 patients. Hepatology 43:1303–1310CrossRefGoogle Scholar
  5. 5.
    Llovet JM, Brú C, Bruix J (1999) Prognosis of hepatocellular carcinoma: the BCLC staging classification. Semin Liver Dis 19:329–338CrossRefGoogle Scholar
  6. 6.
    Di Martino M, De Filippis G, De Santis A et al (2013) Hepatocellular carcinoma in cirrhotic patients: prospective comparison of US, CT and MR imaging. Eur Radiol 23:887–896CrossRefGoogle Scholar
  7. 7.
    Ronot M, Fouque O, Esvan M, Lebigot J, Aubé C, Vilgrain V (2017) Comparison of the accuracy of AASLD and LI-RADS criteria for the non-invasive diagnosis of HCC smaller than 3cm. J Hepatol. CrossRefGoogle Scholar
  8. 8.
    Sersté T, Barrau V, Ozenne V et al (2012) Accuracy and disagreement of computed tomography and magnetic resonance imaging for the diagnosis of small hepatocellular carcinoma and dysplastic nodules: role of biopsy. Hepatology 55:800–806CrossRefGoogle Scholar
  9. 9.
    Forner A, Vilana R, Ayuso C et al (2008) Diagnosis of hepatic nodules 20 mm or smaller in cirrhosis: prospective validation of the noninvasive diagnostic criteria for hepatocellular carcinoma. Hepatology 47:97–104CrossRefGoogle Scholar
  10. 10.
    Heimbach JK, Kulik LM, Finn RS et al (2018) AASLD guidelines for the treatment of hepatocellular carcinoma. Hepatology 67:358–380CrossRefGoogle Scholar
  11. 11.
    Freeman Richard B, Mithoefer A, Ruthazer R et al (2006) Optimizing staging for hepatocellular carcinoma before liver transplantation: a retrospective analysis of the UNOS/OPTN database. Liver Transpl 12:1504–1511CrossRefGoogle Scholar
  12. 12.
    Yang H, Schwartz LH, Zhao B (2016) A response assessment platform for development and validation of imaging biomarkers in oncology. Tomography 2:406–410CrossRefGoogle Scholar
  13. 13.
    Lin LI (1989) A concordance correlation coefficient to evaluate reproducibility. Biometrics 45:255–268CrossRefGoogle Scholar
  14. 14.
    Berenguer R, Pastor-Juan MR, Canales-Vázquez J et al (2018) Radiomics of CT features may be nonreproducible and redundant: influence of CT acquisition parameters. Radiology 288:407–415 %@ 0033-8419CrossRefGoogle Scholar
  15. 15.
    Dercle L, Lu L, Lichtenstein P et al (2017) Impact of variability in portal venous phase acquisition timing in tumor density measurement and treatment response assessment: metastatic colorectal cancer as a paradigm. JCO Clin Cancer Inform (1):1–8Google Scholar
  16. 16.
    Dercle L, Billey C, Cognet T et al (2018) Spleno-hepatic index to predict portal hypertension by equilibrium radionuclide ventriculography. Nucl Med Commun 39:1138–1142CrossRefGoogle Scholar
  17. 17.
    Aerts HJ, Velazquez ER, Leijenaar RT et al (2014) Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 5:4006CrossRefGoogle Scholar
  18. 18.
    Hatt M, Tixier F, Pierce L, Kinahan PE, Le Rest CC, Visvikis D (2017) Characterization of PET/CT images using texture analysis: the past, the present… any future? Eur J Nucl Med Mol Imaging 44:151–165CrossRefGoogle Scholar
  19. 19.
    Banerjee S, Wang DS, Kim HJ et al (2015) A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma. Hepatology 62:792–800CrossRefGoogle Scholar
  20. 20.
    Bakr SH, Echegaray S, Shah RP et al (2017) Noninvasive radiomics signature based on quantitative analysis of computed tomography images as a surrogate for microvascular invasion in hepatocellular carcinoma: a pilot study. J Med Imaging (Bellingham) 4:041303CrossRefGoogle Scholar
  21. 21.
    Renzulli M, Brocchi S, Cucchetti A et al (2015) Can current preoperative imaging be used to detect microvascular invasion of hepatocellular carcinoma? Radiology 279:432–442CrossRefGoogle Scholar
  22. 22.
    Yasaka K, Akai H, Abe O, Kiryu S (2018) Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology 286:887–896CrossRefGoogle Scholar
  23. 23.
    Cerny M, Bergeron C, Billiard JS et al (2018) LI-RADS for MR imaging diagnosis of hepatocellular carcinoma: performance of major and ancillary features. Radiology 171678. CrossRefGoogle Scholar
  24. 24.
    Darnell A, Forner A, Rimola J et al (2015) Liver imaging reporting and data system with MR imaging: evaluation in nodules 20 mm or smaller detected in cirrhosis at screening US. Radiology 275:698–707CrossRefGoogle Scholar
  25. 25.
    Tanabe M, Kanki A, Wolfson T et al (2016) Imaging outcomes of liver imaging reporting and data system version 2014 category 2, 3, and 4 observations detected at CT and MR imaging. Radiology 281:129–139CrossRefGoogle Scholar
  26. 26.
    Aubé C, Oberti F, Lonjon J et al (2017) EASL and AASLD recommendations for the diagnosis of HCC to the test of daily practice. Liver Int 37:1515–1525CrossRefGoogle Scholar
  27. 27.
    Kim DW, Jang HY, Kim KW, Shin Y, Park SH (2019) Design characteristics of studies reporting the performance of artificial intelligence algorithms for diagnostic analysis of medical images: results from recently published papers. Korean J Radiol 20:405–410 %@ 1229-6929CrossRefGoogle Scholar
  28. 28.
    Dercle L, Ammari S, Bateson M et al (2017) Limits of radiomic-based entropy as a surrogate of tumor heterogeneity: ROI-area, acquisition protocol and tissue site exert substantial influence. Sci Rep 7:7952CrossRefGoogle Scholar
  29. 29.
    Dercle L, Hartl D, Rozenblum-Beddok L et al (2018) Diagnostic and prognostic value of 18F-FDG PET, CT, and MRI in perineural spread of head and neck malignancies. Eur Radiol 28:1761–1770CrossRefGoogle Scholar
  30. 30.
    Limkin E, Sun R, Dercle L et al (2017) Promises and challenges for the implementation of computational medical imaging (radiomics) in oncology. Ann Oncol 28:1191–1206CrossRefGoogle Scholar
  31. 31.
    Sun R, Limkin EJ, Vakalopoulou M et al (2018) A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: an imaging biomarker, retrospective multicohort study. Lancet Oncol 19:1180–1191CrossRefGoogle Scholar

Copyright information

© European Society of Radiology 2019

Authors and Affiliations

  • Fatima-Zohra Mokrane
    • 1
    • 2
    Email author
  • Lin Lu
    • 2
  • Adrien Vavasseur
    • 1
  • Philippe Otal
    • 1
  • Jean-Marie Peron
    • 3
  • Lyndon Luk
    • 2
  • Hao Yang
    • 2
  • Samy Ammari
    • 4
  • Yvonne Saenger
    • 5
  • Herve Rousseau
    • 1
  • Binsheng Zhao
    • 2
  • Lawrence H. Schwartz
    • 2
  • Laurent Dercle
    • 2
    • 6
  1. 1.Radiology DepartmentRangueil University HospitalToulouseFrance
  2. 2.Department of Radiology, New York Presbyterian HospitalColumbia University Vagelos College of Physicians and SurgeonsNew York CityUSA
  3. 3.Hepatology DepartmentPurpan University HospitalToulouseFrance
  4. 4.Service de Radiologie, Gustave-RoussyUniversité Paris-SaclayVillejuifFrance
  5. 5.Department of Medicine, Division of Hematology/OncologyColumbia University Medical Center/New York PresbyterianNew YorkUSA
  6. 6.INSERM U1015, Gustave Roussy InstituteUniversité Paris-SaclayVillejuifFrance

Personalised recommendations