Skip to main content

Advertisement

SpringerLink
Log in

Machine learning models in the prediction of 1-year mortality in patients with advanced hepatocellular cancer on immunotherapy: a proof-of-concept study

  • Original Article
  • Published:
Hepatology International Aims and scope Submit manuscript

Abstract

Introduction

Immunotherapy is a new promising treatment for patients with advanced hepatocellular carcinoma (HCC), but is costly and potentially associated with considerable side effects. This study aimed to evaluate the role of machine learning (ML) models in predicting the 1-year cancer-related mortality in advanced HCC patients treated with immunotherapy.

Method

395 HCC patients who had received immunotherapy (including nivolumab, pembrolizumab or ipilimumab) between 2014 and 2019 in Hong Kong were included. The whole data sets were randomly divided into training (n = 316) and internal validation (n = 79) set. The data set, including 47 clinical variables, was used to construct six different ML models in predicting the risk of 1-year mortality. The performances of ML models were measured by the area under receiver operating characteristic curve (AUC) and their performances were compared with C-Reactive protein and Alpha Fetoprotein in ImmunoTherapY score (CRAFITY) and albumin–bilirubin (ALBI) score. The ML models were further validated with an external cohort between 2020 and 2021.

Results

The 1-year cancer-related mortality was 51.1%. Of the six ML models, the random forest (RF) has the highest AUC of 0.92 (95% CI 0.87–0.98), which was better than logistic regression (0.82, p = 0.01) as well as the CRAFITY (0.68, p < 0.01) and ALBI score (0.84, p = 0.04). RF had the lowest false positive (2.0%) and false negative rate (5.2%), and performed better than CRAFITY score in the external validation cohort (0.91 vs 0.66, p < 0.01). High baseline AFP, bilirubin and alkaline phosphatase were three common risk factors identified by all ML models.

Conclusion

ML models could predict 1-year cancer-related mortality in HCC patients treated with immunotherapy, which may help to select patients who would benefit from this treatment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

Supporting data for this study are available from the corresponding author and the first author upon reasonable request.

Abbreviations

AUC:

Area under the receiver operator characteristics curve

CDARS:

Clinical Data Analysis and Reporting System

CI:

Confidence interval

LASSO:

Least absolute shrinkage and selection operator

LR:

Logistic regression

ML:

Machine learning

NLR:

Negative likelihood ratio

NPV:

Negative predictive value

PLR:

Positive likelihood ratio

PPV:

Positive predictive value

NN:

Sparse neural network

GBM:

Stochastic gradient boosting

SVM:

Support vector machine

XGBoost:

Extreme gradient boosting

CPS:

Child–Pugh score

HCC:

Hepatocellular carcinoma

MAE:

Mean absolute error

ICD-9:

The International Classification of Diseases, Ninth Revision

TACE:

Trans-arterial chemoembolization

ICI:

Immune checkpoint inhibitor

TKI:

Tyrosine kinase inhibitor

HBV:

Chronic hepatitis B infection

DM:

Diabetes mellitus

HT:

Hypertension

IHD:

Ischemic heart disease

AF:

Atrial fibrillation

CHF:

Congestive heart failure

CRF:

Chronic renal failure

PPI:

Proton pump inhibitors

AFP:

Alpha fetoprotein

ALP:

Alkaline phosphatase

ALT:

Alanine aminotransferase

AST:

Aspartate aminotransferase

GGT:

Gamma-glutamyl transferase

PT:

Prothrombin time

INR:

International normalized ratio (INR)

Na:

Sodium

WBC:

White blood cell

References

  1. Fitzmaurice C, Abate D, Abbasi N, et al. Global, Regional, and National Cancer Incidence, Mortality, Years of life lost, years lived with disability, and disability-adjusted life-years for 29 cancer groups, 1990 to 2017: a systematic analysis for the Global Burden of Disease Study. JAMA Oncol 2019;5(12):1749–1768

    Article  Google Scholar 

  2. Lee MJ, Chang SW, Kim JH, et al. Real-world systemic sequential therapy with sorafenib and regorafenib for advanced hepatocellular carcinoma: a multicenter retrospective study in Korea. Invest New Drugs 2021;39(1):260–268

    Article  CAS  Google Scholar 

  3. Sangro B, Sarobe P, Hervas-Stubbs S, Melero I. Advances in immunotherapy for hepatocellular carcinoma. Nat Rev Gastroenterol Hepatol 2021. https://doi.org/10.1038/s41575-41021-00438-41570

    Article  PubMed  PubMed Central  Google Scholar 

  4. Xie Y, Xiang Y, Sheng J, et al. Immunotherapy for hepatocellular carcinoma: current advances and future expectations. J Immunol Res 2018;2018:1–15

    Article  Google Scholar 

  5. Nakano S, Eso Y, Okada H, Takai A, Takahashi K, Seno H. Recent advances in immunotherapy for hepatocellular carcinoma. Cancers (Basel) 2020;12(4):1–15

    Article  Google Scholar 

  6. Ziogas IA, Evangeliou AP, Giannis D, et al. The role of immunotherapy in hepatocellular carcinoma: a systematic review and pooled analysis of 2,402 patients. Oncologist. 2021;26(6):e1036–e1049. https://doi.org/10.1002/onco.13638

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Pencina MJ, Goldstein BA, D’Agostino RB. Prediction models—development, evaluation, and clinical application. N Engl J Med 2020;382(17):1583–1586

    Article  Google Scholar 

  8. Sendak MP, Ratliff W, Sarro D, et al. Real-world integration of a sepsis deep learning technology into routine clinical care: implementation study. JMIR Med Inf 2020;8(7): e15182

    Article  Google Scholar 

  9. Milea D, Najjar RP, Zhubo J, et al. Artificial intelligence to detect papilledema from ocular fundus photographs. N Engl J Med 2020;382(18):1687–1695

    Article  Google Scholar 

  10. Richter AN, Khoshgoftaar TM. A review of statistical and machine learning methods for modeling cancer risk using structured clinical data. Artif Intell Med 2018;90:1–14

    Article  Google Scholar 

  11. Lui TKL, Wong KKY, Mak LLY, Ko MKL, Tsao SKK, Leung WK. Endoscopic prediction of deeply submucosal invasive carcinoma with use of artificial intelligence. Endosc Int Open 2019;7(4):E514–E520

    Article  Google Scholar 

  12. Leung WK, Cheung KS, Li B, Law SYK, Lui TKL. Applications of machine learning models in the prediction of gastric cancer risk in patients after Helicobacter pylori eradication. Aliment Pharmacol Ther 2021;53(8):864–872

    PubMed  Google Scholar 

  13. Vanderbeck S, Bockhorst J, Komorowski R, Kleiner DE, Gawrieh S. Automatic classification of white regions in liver biopsies by supervised machine learning. Hum Pathol 2014;45(4):785–792

    Article  Google Scholar 

  14. Byra M, Styczynski G, Szmigielski C, et al. Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images. Int J Comput Assist Radiol Surg 2018;13(12):1895–1903

    Article  Google Scholar 

  15. Ma H, Xu CF, Shen Z, Yu CH, Li YM. Application of machine learning techniques for clinical predictive modeling: a cross-sectional study on nonalcoholic fatty liver disease in China. Biomed Res Int 2018;2018:4304376

    PubMed  PubMed Central  Google Scholar 

  16. Konerman MA, Zhang Y, Zhu J, Higgins PD, Lok AS, Waljee AK. Improvement of predictive models of risk of disease progression in chronic hepatitis C by incorporating longitudinal data. Hepatology 2015;61(6):1832–1841

    Article  Google Scholar 

  17. Wei R, Wang J, Wang X, et al. Clinical prediction of HBV and HCV related hepatic fibrosis using machine learning. EBioMedicine. 2018;35:124–132

    Article  Google Scholar 

  18. Cao Y, Hu ZD, Liu XF, Deng AM, Hu CJ. An MLP classifier for prediction of HBV-induced liver cirrhosis using routinely available clinical parameters. Dis Mark 2013;35(6):653–660

    Article  Google Scholar 

  19. Eaton JE, Vesterhus M, McCauley BM, et al. Primary sclerosing cholangitis risk estimate tool (PREsTo) predicts outcomes of the disease: a derivation and validation study using machine learning. Hepatology 2020;71(1):214–224

    Article  CAS  Google Scholar 

  20. Andres A, Montano-Loza A, Greiner R, et al. A novel learning algorithm to predict individual survival after liver transplantation for primary sclerosing cholangitis. PLoS One 2018;13(3): e0193523

    Article  Google Scholar 

  21. Kim JW, Ye Q, Forgues M, et al. Cancer-associated molecular signature in the tissue samples of patients with cirrhosis. Hepatology 2004;39(2):518–527

    Article  CAS  Google Scholar 

  22. Zou H, Yang X, Li QL, Zhou QX, Xiong L, Wen Y. A comparative study of albumin-bilirubin score with child-pugh score, model for end-stage liver disease score and indocyanine green R15 in predicting posthepatectomy liver failure for hepatocellular carcinoma patients. Dig Dis 2018;36(3):236–243

    Article  Google Scholar 

  23. Scheiner B, Pomej K, Kirstein MM, et al. Prognosis of patients with hepatocellular carcinoma treated with immunotherapy—development and validation of the CRAFITY score. J Hepatol 2022;76(2):353–363

    Article  CAS  Google Scholar 

  24. Mao S, Yu X, Shan Y, Fan R, Wu S, Lu C. Albumin-bilirubin (ALBI) and Monocyte to lymphocyte ratio (MLR)-based nomogram model to predict tumor recurrence of AFP-negative hepatocellular carcinoma. J Hepatocell Carcinoma 2021;8:1355–1365

    Article  CAS  Google Scholar 

  25. Chiu SS, Lau YL, Chan KH, Wong WHS, Peiris JSM. Influenza-related hospitalizations among children in Hong Kong. N Engl J Med 2002;347(26):2097–2103

    Article  Google Scholar 

  26. Chan EW, Lau WC, Leung WK, et al. Prevention of dabigatran-related gastrointestinal bleeding with gastroprotective agents: a population-based study. Gastroenterology 2015;149(3):586–595

    Article  CAS  Google Scholar 

  27. Cheung KS, Seto WK, Fung J, Lai CL, Yuen MF. Epidemiology and natural history of primary biliary cholangitis in the Chinese: a territory-based study in Hong Kong between 2000 and 2015. Clin Transl Gastroenterol 2017;8(8): e116

    Article  Google Scholar 

  28. Leung WK, Wong IOL, Cheung KS, et al. Effects of Helicobacter pylori treatment on incidence of gastric cancer in older individuals. Gastroenterology 2018;155(1):67–75

    Article  Google Scholar 

  29. Cheung KS, Chan EW, Wong AYS, Chen LJ, Wong ICK, Leung WK. Long-term proton pump inhibitors and risk of gastric cancer development after treatment for Helicobacter pylori: a population-based study. Gut 2018;67(1):28–35

    Article  CAS  Google Scholar 

  30. Cheung KS, Chan EW, Chen L, Seto WK, Wong ICK, Leung WK. Diabetes increases risk of gastric cancer after Helicobacter pylori eradication: a territory-wide study with propensity score analysis. Diabetes Care. 2019;42(9):1769–1775

    Article  CAS  Google Scholar 

  31. United States, Department of Health and Human Services, Food and Drug Administration. FDA grants accelerated approval to nivolumab for HCC previously treated with sorafenib. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-grants-accelerated-approval-nivolumab-hcc-previously-treated-sorafenib. Accessed 6 May 2021

  32. United States, Department of Health and Human Services, Food and Drug Administration. FDA grants accelerated approval to pembrolizumab for hepatocellular carcinoma. https://www.fda.gov/drugs/fda-grants-accelerated-approval-pembrolizumab-hepatocellular-carcinoma. Accessed 6 May 2021

  33. Lee JH, Shi Z, Gao Z. On LASSO for Predictive Regression. arXiv Website. https://arxiv.org/abs/1810.03140.pdf. Accessed 27 July 2020.

  34. Kelvin PM. Machine Learning: A Probabilistic Perspective. Cambridge: MIT Press; 2012

    Google Scholar 

  35. Chen T, Guestrin C. XGBoost: a scalable tree boosting system. arXiv Website. https://arxiv.org/pdf/1603.02754.pdf. Accessed 24 Mar 2019.

  36. Schonlau M, Zou RY. The random forest algorithm for statistical learning. Stata J 2020;20(1):3–29

    Article  Google Scholar 

  37. Friedman JH. Stochastic gradient boosting. Comput Stat Data Anal 2002;38:367–378

    Article  Google Scholar 

  38. Suraj S, Akshayvarun S, Venkatesh B. Training Sparse Neural Networks. arXiv Website. https://arxiv.org/abs/1611.06694.pdf. Accessed 27 July 2020.

  39. Bruix J, Llovet JM. Prognostic prediction and treatment strategy in hepatocellular carcinoma. Hepatology 2002;35(3):519–524

    Article  Google Scholar 

  40. Ikai I, Yamamoto Y, Yamamoto N, et al. Results of hepatic resection for hepatocellular carcinoma invading major portal and/or hepatic veins. Surg Oncol Clin N Am 2003;12(1):65–75 (ix)

    Article  Google Scholar 

  41. Pawlik TM, Poon RT, Abdalla EK, et al. Hepatectomy for hepatocellular carcinoma with major portal or hepatic vein invasion: results of a multicenter study. Surgery 2005;137(4):403–410

    Article  Google Scholar 

  42. Chang H, Jung W, Kim A, et al. Expression and prognostic significance of programmed death protein 1 and programmed death ligand-1, and cytotoxic T lymphocyte-associated molecule-4 in hepatocellular carcinoma. APMIS 2017;125(8):690–698

    Article  CAS  Google Scholar 

  43. Li J, Huang L, Liu C, et al. Risk factors and clinical outcomes of extrahepatic recurrence in patients with post-hepatectomy recurrent hepatocellular carcinoma. ANZ J Surg. 2021;91(6):1174–1179. https://doi.org/10.1111/ans.16737

    Article  PubMed  Google Scholar 

  44. Chen KL, Gao J. Factors influencing the short-term and long-term survival of hepatocellular carcinoma patients with portal vein tumor thrombosis who underwent chemoembolization. World J Gastroenterol 2021;27(13):1330–1340

    Article  CAS  Google Scholar 

  45. Xiong D, Zhang K, Xu M. Prognostic factors of nivolumab in advanced hepatocellular carcinoma: a systematic review and meta-analysis. Panminerva Med. 2021. https://doi.org/10.23736/S0031-0808.21.04282-8

    Article  PubMed  Google Scholar 

  46. Llovet JM, Bru C, Bruix J. Prognosis of hepatocellular carcinoma: the BCLC staging classification. Semin Liver Dis 1999;19(3):329–338

    Article  CAS  Google Scholar 

  47. Yau T, Tang VY, Yao TJ, Fan ST, Lo CM, Poon RT. Development of Hong Kong Liver Cancer staging system with treatment stratification for patients with hepatocellular carcinoma. Gastroenterology 2014;146(7):1691–1700

    Article  Google Scholar 

  48. Forner A, Reig ME, de Lope CR, Bruix J. Current strategy for staging and treatment: the BCLC update and future prospects. Semin Liver Dis 2010;30(1):61–74

    Article  CAS  Google Scholar 

  49. Heinrich S, Sprinzl M, Schmidtmann I, et al. Validation of prognostic accuracy of MESH, HKLC, and BCLC classifications in a large German cohort of hepatocellular carcinoma patients. United European Gastroenterol J. 2020;8(4):444–452

    Article  Google Scholar 

Download references

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Department of Medicine, University of Hong Kong, 4/F, Professorial Block, Queen Mary Hospital, 102 Pokfulam Road, Hong Kong, China

    Thomas Ka Luen Lui, Ka Shing Cheung & Wai Keung Leung

Authors
  1. Thomas Ka Luen Lui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ka Shing Cheung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wai Keung Leung
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

WKL was involved in the study design and conception; interpretation of the data; drafting of manuscript and study supervision. KSC were involved in the data acquisition and critical review of the manuscript. TKLT was involved in the study design, conception, data analysis and drafting of the manuscript.

Corresponding author

Correspondence to Wai Keung Leung.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Human and animal research

This article does not contain any studies with animals performed by any of the authors.

Consent to participate

This study was approved by the ethics committees of our hospital and fully complied with the Declaration of Helsinki and the Guideline for Good Clinical Practice.

Consent for publication

Informed consent for publication was obtained from all author.

Plant reproducibility

This article does not contain any information regarding this issue.

Clinical trials registration

This article does not have clinical trial registration.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 8543 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lui, T.K.L., Cheung, K.S. & Leung, W.K. Machine learning models in the prediction of 1-year mortality in patients with advanced hepatocellular cancer on immunotherapy: a proof-of-concept study. Hepatol Int 16, 879–891 (2022). https://doi.org/10.1007/s12072-022-10370-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12072-022-10370-3

Keywords

Search

Navigation