Early Warning Models to Estimate the 30-Day Mortality Risk After Stent Placement for Patients with Malignant Biliary Obstruction



To develop, validate, and compare early warning models of the 30-day mortality risk for patients with malignant biliary obstruction (MBO) undergoing percutaneous transhepatic biliary stent placement (PTBS).

Materials and Methods

Between January 2013 and October 2018, this multicenter retrospective study included 299 patients with MBOs who underwent PTBS. The training set consisted of 166 patients from four cohorts, and another two independent cohorts were allocated as external validation sets A and B with 75 patients and 58 patients, respectively. A logistic model and an artificial neural network (ANN) model were developed to predict the risk of 30-day mortality after PTBS. The predictive performance of these two models was validated internally and externally.


The ANN model had higher values of area under the curve than the logistic model in the training set (0.819 vs 0.797), especially in the validation sets A (0.802 vs 0.714) and B (0.732 vs 0.568). Both models had high accuracy in the three sets (75.9–83.1%). Along with a high specificity, the ANN model improved the sensitivity. The net reclassification improvement and integrated discrimination improvement also demonstrated that the ANN model led to improvements in predictive ability compared with the logistic model.


Early warning models were proposed to predict the risk of 30-day mortality after PTBS in patients with MBO. The ANN model has higher accuracy and better generalizability than the logistic model.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3



Artificial neural network


Malignant biliary obstruction


Percutaneous transhepatic biliary stent placement


Area under the curve


Net reclassification improvement


Integrated discrimination improvement


Eastern Cooperative Oncology Group


Neutrophil-to-lymphocyte ratio


Cancer antigen


Carcinoembryonic antigen


Odds ratio


Confidence interval


  1. 1.

    Pu LZCT, Singh R, Loong CK, de Moura EGH. Malignant biliary obstruction: evidence for best practice. Gastroenterol Res Pract. 2016;2016(3296801):1–7.

    Article  Google Scholar 

  2. 2.

    Kapoor BS, Mauri G, Lorenz JM. Management of biliary strictures: state-of-the-art review. Radiology. 2018;289(3):590–603.

    Article  Google Scholar 

  3. 3.

    Riaz A, Pinkard JP, Salem R, Lewandowski RJ. Percutaneous management of malignant biliary disease. J Surg Oncol. 2019;120(1):45–56.

    Article  Google Scholar 

  4. 4.

    Saad WEA, Wallace MJ, Wojak JC, Kundu S, Cardella JF. Quality improvement guidelines for percutaneous transhepatic cholangiography, biliary drainage, and percutaneous cholecystostomy. J Vasc Interv Radiol. 2010;21(6):789–95.

    Article  Google Scholar 

  5. 5.

    Moole H, Dharmapuri S, Duvvuri A, et al. Endoscopic versus percutaneous biliary drainage in palliation of advanced malignant hilar obstruction: a meta-analysis and systematic review. Can J Gastroenterol Hepatol. 2016;2016(4726078):1–8.

    Article  Google Scholar 

  6. 6.

    Enochsson L, Swahn F, Arnelo U, Nilsson M, Löhr M, Persson G. Nationwide, population-based data from 11,074 ERCP procedures from the Swedish Registry for Gallstone Surgery and ERCP. Gastrointest Endosc. 2010;72(6):1175–84.

    Article  Google Scholar 

  7. 7.

    Bodger K, Bowering K, Sarkar S, Thompson E, Pearson MG. All-cause mortality after first ERCP in England: clinically guided analysis of hospital episode statistics with linkage to registry of death. Gastrointest Endosc. 2011;74(4):825–33.

    Article  Google Scholar 

  8. 8.

    Leng J, Zhang N, Dong J. Percutaneous transhepatic and endoscopic biliary drainage for malignant biliary tract obstruction: a meta-analysis. World J Surg Oncol. 2014;12(1):272.

    Article  Google Scholar 

  9. 9.

    Zhao X, Dong J, Jiang K, Huang X, Zhang W. Comparison of percutaneous transhepatic biliary drainage and endoscopic biliary drainage in the management of malignant biliary tract obstruction: a meta-analysis. Dig Endosc. 2015;27(1):137–45.

    Article  Google Scholar 

  10. 10.

    Duan F, Cui L, Bai Y, Li X, Yan J, Liu X. Comparison of efficacy and complications of endoscopic and percutaneous biliary drainage in malignant obstructive jaundice: a systematic review and meta-analysis. Cancer Imaging. 2017;17(27):1–7.

    Google Scholar 

  11. 11.

    Uberoi R, Das N, Moss J, Robertson I. British society of interventional radiology: biliary drainage and stenting registry (BDSR). Cardiovasc Intervent Radiol. 2012;35(1):127–38.

    CAS  Article  Google Scholar 

  12. 12.

    Rai R, Dick R, Doctor N, Dafnios N, Morris R, Davidson BR. Predicting early mortality following percutaneous stent insertion for malignant biliary obstruction: a multivariate risk factor analysis. Eur J Gastroenterol Hepatol. 2000;12(10):1095–100.

    CAS  Article  Google Scholar 

  13. 13.

    Tapping CR, Byass OR, Cast JEI. Percutaneous transhepatic biliary drainage (PTBD) with or without stenting—complications, re-stent rate and a new risk stratification score. Eur Radiol. 2011;21(9):1948–55.

    CAS  Article  Google Scholar 

  14. 14.

    Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. 2019;380(14):1347–58.

    Article  Google Scholar 

  15. 15.

    Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–9.

    CAS  Article  Google Scholar 

  16. 16.

    Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148(3):839–43.

    CAS  Article  Google Scholar 

  17. 17.

    Obuchowski NA, McClish DK. Sample size determination for diagnostic accuracy studies involving binormal ROC curve indices. Stat Med. 1997;16(13):1529–42.

    CAS  Article  Google Scholar 

  18. 18.

    Kerr KF, Brown MD, Zhu K, Janes H. Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use. J Clin Oncol. 2016;34(21):2534–40.

    Article  Google Scholar 

  19. 19.

    Rerknimitr R, Angsuwatcharakon P, Ratanachu-ek T, et al. Asia-Pacific consensus recommendations for endoscopic and interventional management of hilar cholangiocarcinoma. J Gastroenterol Hepatol. 2013;28(4):593–607.

    Article  Google Scholar 

  20. 20.

    Toll DB, Janssen KJM, Vergouwe Y, Moons KGM. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol. 2008;61(11):1085–94.

    CAS  Article  Google Scholar 

  21. 21.

    Steyerberg E. Clinical prediction models. New York City, NY: Springer; 2009.

    Google Scholar 

  22. 22.

    Amin MB, Edge S, Greene F, et al. AJCC cancer staging manual. Berlin: Springer; 2017.

    Google Scholar 

  23. 23.

    Zhu HD, Guo JH, Huang M, et al. Irradiation stents vs. conventional metal stents for unresectable malignant biliary obstruction: a multicenter trial. J Hepatol. 2018;68(5):970–7.

    Article  Google Scholar 

  24. 24.

    Matsumoto M, Nakabayashi Y, Fujiwara Y, et al. Duration of preoperative biliary drainage as a prognostic factor after pancreaticoduodenectomy for pancreatic head cancer. Anticancer Res. 2017;37(6):3215–9.

    PubMed  Google Scholar 

  25. 25.

    Hyun H, Choi SY, Kim KA, Ko SB. Safety and efficacy of percutaneous biliary covered stent placement in patients with malignant biliary hilar obstruction; correlation with liver function. Cardiovasc Intervent Radiol. 2016;39(9):1298–305.

    Article  Google Scholar 

  26. 26.

    Li M, Li K, Qi X, et al. Percutaneous transhepatic biliary stent implantation for obstructive jaundice of perihilar cholangiocarcinoma: a prospective study on predictors of stent patency and survival in 92 patients. J Vasc Interv Radiol. 2016;27(7):1047–55.

    Article  Google Scholar 

  27. 27.

    Fujii T, Yamada S, Suenaga M, et al. Preoperative internal biliary drainage increases the risk of bile juice infection and pancreatic fistula after pancreatoduodenectomy: a prospective observational study. Pancreas. 2015;44(3):465–70.

    PubMed  Google Scholar 

  28. 28.

    Jethwa P, Breuning E, Bhati C, Buckles J, Mirza D, Bremhall S. The microbiological impact of pre-operative biliary drainage on patients undergoing hepato-biliary-pancreatic (HPB) surgery. Aliment Pharmacol Ther. 2007;25(10):1175–80.

    CAS  Article  Google Scholar 

  29. 29.

    Sugiura T, Okamura Y, Ito T, et al. Prognostic scoring system for patients who present with a gastric outlet obstruction caused by advanced pancreatic adenocarcinoma. World J Surg. 2017;41(10):2619–24.

    Article  Google Scholar 

  30. 30.

    Ji JH, Song HN, Kim RB, et al. Natural history of metastatic biliary tract cancer (BTC) patients with good performance status (PS) who were treated with only best supportive care (BSC). Jpn J Clin Oncol. 2015;45(3):256–60.

    Article  Google Scholar 

  31. 31.

    Tu JV. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol. 1996;49(11):1225–311.

    CAS  Article  Google Scholar 

  32. 32.

    Guerriere MR, Detsky AS. Neural networks: what are they? Ann Intern Med. 1991;115(11):906–7.

    CAS  Article  Google Scholar 

  33. 33.

    Zhang Z, Beck MW, Winkler DA, et al. Opening the black box of neural networks: methods for interpreting neural network models in clinical applications. Ann Transl Med. 2018;6(11):216.

    Article  Google Scholar 

Download references


The authors thank Dr. Qi Zhang, Dr. Yong Wang, and Dr. Jun-Ying Wang from Zhongda Hospital, Southeast University, Nanjing, China, for their work in the management of patients. The authors thank Dr. Bo Peng from Yunnan Tumor Hospital, the Third Affiliated Hospital of Kunming Medical University, Kunming, China, Dr. Jing-Jing Song from Lishui Central Hospital, Wenzhou Medical University, Lishui, China, Dr. Dong Lu from Anhui Provincial Hospital, the First Affiliated Hospital of University of Science and Technology of China (USTC), Hefei, China, Dr. Wu-Jie Wang from the Second Hospital of Shandong University, Jinan, China, and Dr. Ning Wei from Affiliated Hospital of Xuzhou Medical University, Xuzhou, China, for their efforts in the follow-up of patients.


This study was supported by the National Key Scientific Instrument and Equipment Development Projects of China (81827805), Innovation Platform of Jiangsu Provincial Medical Center (YXZXA2016005), and National Natural Science Foundation of China (81520108015, 81671796). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Author information




All authors contributed to review and critical revision of the manuscript and approved the final version of the manuscript. GJT, HFZ, JL, HDZ, and JHG contributed to study concept and design. JHG, MH, JSJ, WFL, YLL, HX, LC, and GYZ contributed to acquisition of data. HFZ contributed to drafting of the manuscript. GJT, HDZ, and JHG contributed to analysis and interpretation of data. JL contributed to statistical analysis. GJT, HDZ, and JHG supervised and oversaw the study.

Corresponding author

Correspondence to Gao-Jun Teng.

Ethics declarations

Conflict of interest


Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed Consent

The requirement to obtain informed consent was waived due to the retrospective nature of this study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 357 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhou, HF., Lu, J., Zhu, HD. et al. Early Warning Models to Estimate the 30-Day Mortality Risk After Stent Placement for Patients with Malignant Biliary Obstruction. Cardiovasc Intervent Radiol 42, 1751–1759 (2019). https://doi.org/10.1007/s00270-019-02331-5

Download citation


  • Malignant biliary obstruction
  • Biliary stent
  • 30-Day mortality
  • Prediction
  • Artificial neural network