Skip to main content

Convolutional neural network for classifying primary liver cancer based on triple-phase CT and tumor marker information: a pilot study



To develop convolutional neural network (CNN) models for differentiating intrahepatic cholangiocarcinoma (ICC) from hepatocellular carcinoma (HCC) and predicting histopathological grade of HCC.

Materials and methods

Preoperative computed tomography and tumor marker information of 617 primary liver cancer patients were retrospectively collected to develop CNN models categorizing tumors into three categories: moderately differentiated HCC (mHCC), poorly differentiated HCC (pHCC), and ICC, where the histopathological diagnoses were considered as ground truths. The models processed manually cropped tumor with and without tumor marker information (two-input and one-input models, respectively). Overall accuracy was assessed using a held-out dataset (10%). Area under the curve, sensitivity, and specificity for differentiating ICC from HCCs (mHCC + pHCC), and pHCC from mHCC were also evaluated. We assessed two radiologists’ performance without tumor marker information as references (overall accuracy, sensitivity, and specificity). The two-input model was compared with the one-input model and radiologists using permutation tests.


The overall accuracy was 0.61, 0.60, 0.55, 0.53 for the two-input model, one-input model, radiologist 1, and radiologist 2, respectively. For differentiating pHCC from mHCC, the two-input model showed significantly higher specificity than radiologist 1 (0.68 [95% confidence interval: 0.50–0.83] vs 0.45 [95% confidence interval: 0.27–0.63]; p = 0.04).


Our CNN model with tumor marker information showed feasibility and potential for three-class classification within primary liver cancer.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6




CA 19-9:

Carbohydrate antigen 19-9


Carcinoembryonic antigen


Convolutional neural network


Hepatocellular carcinoma


Hounsfield units


Intrahepatic cholangiocarcinoma


Liver imaging reporting and data system


Moderately differentiated hepatocellular carcinoma


Poorly differentiated hepatocellular carcinoma


Prothrombin induced by vitamin K absence-II


Receiver-operating characteristic


Region of interest


  1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2020. Cancer J Clin. 2020;70:7–30.

    Article  Google Scholar 

  2. Weber SM, Ribero D, O’Reilly EM, et al. Intrahepatic Cholangiocarcinoma: expert consensus statement. Am J Med Sci. 2015;17:669–80.

    Article  Google Scholar 

  3. Oishi K, Itamoto T, Amano H, et al. Clinicopathologic features of poorly differentiated hepatocellular carcinoma. J Surg Oncol. 2007;95:311–6.

    Article  PubMed  Google Scholar 

  4. Chernyak V, Fowler KJ, Kamaya A, et al. Liver imaging reporting and data system (LI-RADS) version 2018: imaging of hepatocellular carcinoma in at-risk patients. Radiology. 2018;289:816–30.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Seo N, Kim DY, Choi J-Y. Cross-sectional imaging of intrahepatic cholangiocarcinoma: development, growth, spread, and prognosis. AJR Am J Roentgenol. 2017;209:W64–75.

    Article  PubMed  Google Scholar 

  6. Nakachi K, Tamai H, Mori Y, et al. Prediction of poorly differentiated hepatocellular carcinoma using contrast computed tomography. Cancer Imaging. 2014;14:1–6.

    Article  Google Scholar 

  7. Nishie A, Yoshimitsu K, Okamoto D, et al. CT prediction of histological grade of hypervascular hepatocellular carcinoma: utility of the portal phase. Jpn J Radiol. 2012;31:89–98.

    Article  PubMed  Google Scholar 

  8. Zakhary NI, Khodeer SM, Shafik HE, Malak CAA. Impact of PIVKA-II in diagnosis of hepatocellular carcinoma. J Adv Res. 2013;4:539–46.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Dodson RM, Weiss MJ, Cosgrove D, et al. Intrahepatic cholangiocarcinoma: management options and emerging therapies. J Am Coll Surg. 2013;217(736–750):e4.

    Article  Google Scholar 

  10. Daniele B, Bencivenga A, Megna AS, Tinessa V. α-fetoprotein and ultrasonography screening for hepatocellular carcinoma. Gastroenterology. 2004;127:S108–12.

    Article  PubMed  Google Scholar 

  11. Park H, Park JY. Clinical significance of AFP and PIVKA-II responses for monitoring treatment outcomes and predicting prognosis in patients with hepatocellular carcinoma. Biomed Res Int. 2013;2013:310427.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  12. Yamashita R, Nishio M, Do RKG, Togashi K. Convolutional neural networks: an overview and application in radiology. Insights into Imaging. 2018;521:1–19.

    Article  Google Scholar 

  13. Chartrand G, Cheng PM, Vorontsov E, et al. Deep learning: a primer for radiologists. Radiographics. 2017;37:2113–31.

    Article  PubMed  Google Scholar 

  14. Hamm CA, Wang CJ, Savic LJ, et al. Deep learning for liver tumor diagnosis part I: development of a convolutional neural network classifier for multi-phasic MRI. Eur Radiol. 2019;29:1–10.

    Article  Google Scholar 

  15. Yasaka K, Akai H, Abe O, Kiryu S. Deep Learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology. 2018;286:887–96.

    Article  PubMed  Google Scholar 

  16. Sasaki K, Matsuda M, Ohkura Y, et al. In Hepatocellular carcinomas, any proportion of poorly differentiated components is associated with poor prognosis after hepatectomy. World J Surg. 2014;38(5):1147–53.

    Article  PubMed  Google Scholar 

  17. Choi JY, Lee JM, Sirlin CB. CT and MR imaging diagnosis and staging of hepatocellular carcinoma: part I Development, growth, and spread: key pathologic and imaging aspects. Radiology. 2014;272(3):635–54.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Tsunematsu S, Chuma M, Kamiyama T, et al. Intratumoral artery on contrast-enhanced computed tomography imaging: differentiating intrahepatic cholangiocarcinoma from poorly differentiated hepatocellular carcinoma. Abdom Imaging. 2015;40:1492–9.

    Article  PubMed  Google Scholar 

  19. Lee JH, Lee JM, Kim SJ, et al. Enhancement patterns of hepatocellular carcinomas on multiphasic multidetector row CT: comparison with pathological differentiation. Br J Radiol. 2012;85:e573–83.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  20. Zhao Y-J, Chen W-X, Wu D-S, et al. Differentiation of mass-forming intrahepatic cholangiocarcinoma from poorly differentiated hepatocellular carcinoma: based on the multivariate analysis of contrast-enhanced computed tomography findings. Abdom Radiol (NY). 2016;41:978–89.

    Article  Google Scholar 

  21. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.

    CAS  Article  Google Scholar 

  22. Yamashita R, Mittendorf A, Zhu Z, et al. Deep convolutional neural network applied to the liver imaging reporting and data system (LI-RADS) version 2014 category classification: a pilot study. Abdom Radiol (NY). 2020;45:24–35.

    Article  Google Scholar 

  23. Ludwig DR, Fraum TJ, Cannella R, et al. Hepatocellular carcinoma (HCC) versus non-HCC: accuracy and reliability of liver imaging reporting and data system v2018. Abdom Radiol (NY). 2019;44:2116–32.

    Article  Google Scholar 

  24. Yang D-W, Jia X-B, Xiao Y-J, Wang X-P, Wang Z-C, Yang Z-H. Noninvasive evaluation of the pathologic grade of hepatocellular carcinoma using MCF-3DCNN: a pilot study. Biomed Res Int. 2019.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Bartolozzi C, Cioni D, Donati D, Lencioni R. Focal liver lesions: MR imaging-pathologic correlation. Eur Radiol. 2001;11:1374–88.

    CAS  Article  Google Scholar 

  26. Pamela S, Jeong M-L, Ijin J, et al. Evaluation of the impact of iterative reconstruction algorithms on computed tomography texture features of the liver parenchyma using the filtration-histogram method. Korean J Radiol. 2019;20:558–68.

    Article  Google Scholar 

  27. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. Int J Comput Vis. 2020;128:336–59.

    Article  Google Scholar 

Download references


We thank Andrea Baird, MD, from Edanz Group ( for editing a draft of this manuscript.


We have no funding to declare.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Hirotsugu Nakai.

Ethics declarations

Ethical statement

This study was approved by our institutional review board and was conducted according to the principles of the Declaration of Helsinki.

Statement authorship

This article has not been published or presented elsewhere in part or in entirety, and is not under consideration by another journal. Each author has participated sufficiently in the study to take public responsibility for it. The authors have all approved the manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 106 KB)

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nakai, H., Fujimoto, K., Yamashita, R. et al. Convolutional neural network for classifying primary liver cancer based on triple-phase CT and tumor marker information: a pilot study. Jpn J Radiol 39, 690–702 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • Deep learning
  • Hepatocellular cancer
  • Cholangiocellular carcinoma
  • Tumor grading
  • Computed tomography
  • X-ray