Skip to main content

Predicting Symptoms from Multiphasic MRI via Multi-instance Attention Learning for Hepatocellular Carcinoma Grading

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12905)


Liver cancer is the third leading cause of cancer death in the world, where the hepatocellular carcinoma (HCC) is the most common case in primary liver cancer. In general diagnosis, accurate prediction of HCC grades is of great help to the subsequent treatment to improve the survival rate. Rather than to straightly predict HCC grades from images, it will be more interpretable in clinic to first predict the symptoms and then obtain the HCC grades from the Liver Imaging Reporting and Data System (LI-RADS). Accordingly, we propose a two-stage method for automatically predicting HCC grades according to multiphasic magnetic resonance imaging (MRI). The first stage uses multi-instance learning (MIL) to classify the LI-RADS symptoms while the second stage resorts LI-RADS to grade from the predicted symptoms. Since our method provides more diagnostic basis besides the grading results, it is more interpretable and closer to the clinical process. Experimental results on a dataset with 439 patients indicate that our two-stage method is more accurate than the straight HCC grading approach.


  • Hepatocellular carcinoma
  • Multiphasic MRI
  • Multi-instance learning

This work is mainly completed under the collaboration of Z. Qiu and Y. Pan.

Z. Qiu and Y. Pan—Contribute equally.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. Sung, H., et al.: Global cancer statistics 2020: GLOBOCAN 2020 estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 71, 209–249 (2021)

    CrossRef  Google Scholar 

  2. El-Serag, H.B., Rudolph, K.L.: Hepatocellular carcinoma: epidemiology and molecular carcinogenesis. Gastroenterology 132(7), 2557–2576 (2007)

    CrossRef  Google Scholar 

  3. Liu, Z., et al.: The trends in incidence of primary liver cancer caused by specific etiologies: results from the global burden of disease study 2016 and implications for liver cancer prevention. J. Hepatol. 70(4), 674–683 (2019)

    CrossRef  Google Scholar 

  4. Mulé, S., et al.: Multiphase liver MRI for identifying the macrotrabecular-massive subtype of hepatocellular carcinoma. Radiology 295(3), 562–571 (2020)

    CrossRef  Google Scholar 

  5. Block, K.T., Uecker, M., Frahm, J.: Undersampled radial MRI with multiple coils. Iterative image reconstruction using a total variation constraint. Magn. Reson. Med. 57(6), 1086–1098 (2007)

    CrossRef  Google Scholar 

  6. American College of Radiology: Liver imaging reporting and data system version 2018.

  7. Chernyak, V., et al.: Liver imaging reporting and data system (LI-RADS) version 2018: imaging of hepatocellular carcinoma in at-risk patients. Radiology 289(3), 816–830 (2018)

    CrossRef  Google Scholar 

  8. Wu, Y., et al.: Deep learning LI-RADS grading system based on contrast enhanced multiphase MRI for differentiation between LR-3 and LR-4/LR-5 liver tumors. Ann. Trans. Med. 8(11), 701 (2020)

    CrossRef  Google Scholar 

  9. Li, X., Chen, H., Qi, X., Dou, Q., Fu, C.W., Heng, P.A.: H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging 37(12), 2663–2674 (2018)

    CrossRef  Google Scholar 

  10. Kawka, M., Dawidziuk, A., Jiao, L.R., Gall, T.M.H.: Artificial intelligence in the detection, characterisation and prediction of hepatocellular carcinoma: a narrative review. Transl. Gastroenterol. Hepatol. (2020)

    Google Scholar 

  11. Hamm, C.A., et al.: Deep learning for liver tumor diagnosis part i: development of a convolutional neural network classifier for multi-phasic MRI. Eur. Radiol. 29(7), 3338–3347 (2019)

    CrossRef  Google Scholar 

  12. Yamashita, R., et al.: Deep convolutional neural network applied to the liver imaging reporting and data system (LI-RADS) version 2014 category classification: a pilot study. Abdom. Radiol. 45(1), 24–35 (2020)

    CrossRef  Google Scholar 

  13. Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: Gan-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321, 321–331 (2018)

    CrossRef  Google Scholar 

  14. Liang, D., et al.: Residual convolutional neural networks with global and local pathways for classification of focal liver lesions. In: Geng, X., Kang, B.-H. (eds.) PRICAI 2018. LNCS (LNAI), vol. 11012, pp. 617–628. Springer, Cham (2018).

    CrossRef  Google Scholar 

  15. Yasaka, K., Akai, H., Abe, O., Kiryu, S.: Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology 286(3), 887–896 (2018)

    CrossRef  Google Scholar 

  16. Trivizakis, E., et al.: Extending 2-D convolutional neural networks to 3-D for advancing deep learning cancer classification with application to MRI liver tumor differentiation. IEEE J. Biomed. Health Inform. 23(3), 923–930 (2018)

    CrossRef  Google Scholar 

  17. Zhen, S., et al.: Deep learning for accurate diagnosis of liver tumor based on magnetic resonance imaging and clinical data. Front. Oncol. 10, 680 (2020)

    CrossRef  Google Scholar 

  18. Wu, J., Yu, Y., Huang, C., Yu, K.: Deep multiple instance learning for image classification and auto-annotation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3460–3469 (2015)

    Google Scholar 

  19. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  20. Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning, pp. 2127–2136. PMLR (2018)

    Google Scholar 

  21. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

    Google Scholar 

  22. Kermany, D.S., et al.: Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5), 1122–1131 (2018)

    CrossRef  Google Scholar 

  23. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition 2009, pp. 248–255. IEEE (2009)

    Google Scholar 

  24. Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016)

Download references


This work was supported in part by the National Natural Science Foundation of China under Grants 61771397, in part by the CAAI-Huawei MindSpore Open Fund under Grants CAAIXSJLJJ-2020-005B, and in part by the China Postdoctoral Science Foundation under Grants BX2021333.

Author information

Authors and Affiliations


Corresponding authors

Correspondence to Dijia Wu , Yong Xia or Dinggang Shen .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Qiu, Z., Pan, Y., Wei, J., Wu, D., Xia, Y., Shen, D. (2021). Predicting Symptoms from Multiphasic MRI via Multi-instance Attention Learning for Hepatocellular Carcinoma Grading. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12905. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87239-7

  • Online ISBN: 978-3-030-87240-3

  • eBook Packages: Computer ScienceComputer Science (R0)