Models for acute on chronic liver failure development and mortality in a veterans affairs cohort


Background and purpose

The diagnosis of acute on chronic liver failure (ACLF) carries a high short-term mortality, making early identification of at-risk patients crucial. To date, there are no models that predict which patients with compensated cirrhosis will develop ACLF, and limited models exist to predict ACLF mortality. We sought to create novel risk prediction models using a large North American cohort.


We performed a retrospective study of 75,922 patients with compensated cirrhosis from the Veterans Outcomes and Costs Associated with Liver Disease (VOCAL) dataset. Using 70% derivation/30% validation sets, we identified ACLF patients using the Asian Pacific Association of Liver (APASL) definition. Multivariable logistic regression was used to derive prediction models (called VOCAL-Penn) for developing ACLF at 3, 6, and 12 months. We then created prediction models for ACLF mortality at 28 and 90 days.


The VOCAL-Penn models for ACLF development had very good discrimination [concordance (C) statistics of 0.93, 0.92, and 0.89 at 3, 6, and 12 months, respectively] and calibration. The mortality models also had good discrimination at 28 and 90 days (C statistics 0.89 and 0.88, respectively), outperforming the Model for End-stage Liver Disease (MELD), MELD-sodium, and the APASL ACLF Research Consortium ACLF scores.


We have developed novel tools for predicting development of ACLF in compensated cirrhosis patients, as well as for ACLF mortality. These tools may be used to proactively guide patient follow-up, prognostication, escalation of care, and transplant evaluation.

Graphic abstract

Receiver operating characteristic (ROC) curves for predicting development of APASL ACLF at 3 months (a), 6 months (b), and 1 year (c)

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3


  1. 1.

    Tapper EB, Parikh ND. Mortality due to cirrhosis and liver cancer in the United States, 1999–2016: observational study. BMJ. 2018;362:k2817.

    Article  PubMed  PubMed Central  Google Scholar 

  2. 2.

    Sarin SK, Choudhury A. Acute-on-chronic liver failure: terminology, mechanisms and management. Nat Rev Gastroenterol Hepatol. 2016;13(3):131–49.

    CAS  Article  PubMed  Google Scholar 

  3. 3.

    Moreau R, Jalan R, Gines P, et al. Acute-on-chronic liver failure is a distinct syndrome that develops in patients with acute decompensation of cirrhosis. Gastroenterology. 2013;144(7):1426–37.–9).

    Article  PubMed  Google Scholar 

  4. 4.

    O’Grady J. Con: acute-on-chronic liver failure. Liver Transpl. 2017;23(10):1325–7.

    Article  PubMed  Google Scholar 

  5. 5.

    Nadim M, Kaplan D, Taddei T, Goldberg DS. Reply to letters to the editor: Diverging definitions and dividing lines in acute‐on‐chronic liver failure. Hepatology. Accessed 23 Dec 2019.

  6. 6.

    Sarin SK, Choudhury A, Sharma MK, et al. Acute-on-chronic liver failure: consensus recommendations of the Asian Pacific association for the study of the liver (APASL): an update. Hepatol Int. 2019;13(4):353–90.

    Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    Singh H, Pai CG. Defining acute-on-chronic liver failure: east, west or middle ground? World J Hepatol. 2015;7(25):2571–7.

    Article  PubMed  PubMed Central  Google Scholar 

  8. 8.

    O’Leary J, Reddy K, Garcia-Tsao G, et al. NACSELD acute-on-chronic liver failure (NACSELD-ACLF) score predicts 30-day survival in hospitalized patients with cirrhosis. Hepatology. 2018;67(6):2367–74.

    CAS  Article  PubMed  Google Scholar 

  9. 9.

    Hernaez R, Kramer JR, Liu Y, et al. Prevalence and short-term mortality of acute-on-chronic liver failure: a national cohort study from the USA. J Hepatol. 2019;70(4):639–47.

    Article  PubMed  Google Scholar 

  10. 10.

    Choudhury A, Jindal A, Maiwall R, et al. Liver failure determines the outcome in patients of acute-on-chronic liver failure (ACLF): comparison of APASL ACLF research consortium (AARC) and CLIF-SOFA models. Hepatol Int. 2017;11(5):461–71.

    CAS  Article  PubMed  Google Scholar 

  11. 11.

    Kaplan DE, Dai F, Aytaman A, et al. Development and performance of an algorithm to estimate the Child–Turcotte–Pugh score from a National Electronic Healthcare Database. Clin Gastroenterol Hepatol. 2015;13(13):2333.e1–41.e61.e6.e6.

    Article  Google Scholar 

  12. 12.

    Goldberg DS, French B, Forde KA, et al. Association of distance from a transplant center with access to waitlist placement, receipt of liver transplantation, and survival among US veterans. JAMA. 2014;311(12):1234–43.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  13. 13.

    Mahmud N, Kaplan DE, Taddei TH, Goldberg DS. Incidence and mortality of acute-on-chronic liver failure using two definitions in patients with compensated cirrhosis. Hepatology. 2019;69(5):2150–63.

    Article  PubMed  PubMed Central  Google Scholar 

  14. 14.

    Mahmud N, Sundaram V, Kaplan DE, Taddei TH, Goldberg DS. Grade 1 acute on chronic liver failure is a predictor for subsequent grade 3 failure. Hepatology. 2019.

    Article  PubMed  PubMed Central  Google Scholar 

  15. 15.

    Goldberg D, Lewis J, Halpern S, Weiner M, Lo RV. Validation of three coding algorithms to identify patients with end-stage liver disease in an administrative database. Pharmacoepidemiol Drug Saf. 2012;21(7):765–9.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  16. 16.

    Lapointe-Shaw L, Georgie F, Carlone D, et al. Identifying cirrhosis, decompensated cirrhosis and hepatocellular carcinoma in health administrative data: a validation study. PLoS ONE. 2018;13(8):e0201120.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  17. 17.

    Bradley KA, DeBenedetti AF, Volk RJ, Williams EC, Frank D, Kivlahan DR. AUDIT-C as a brief screen for alcohol misuse in primary care. Alcohol Clin Exp Res. 2007;31(7):1208–17.

    Article  PubMed  Google Scholar 

  18. 18.

    Beste LA, Leipertz SL, Green PK, Dominitz JA, Ross D, Ioannou GN. Trends in burden of cirrhosis and hepatocellular carcinoma by underlying liver disease in US veterans, 2001–2013. Gastroenterology. 2015;149(6):1471.e5–82.e5. e17–18).

    Article  Google Scholar 

  19. 19.

    Smith GCS, Seaman SR, Wood AM, Royston P, White IR. Correcting for optimistic prediction in small data sets. Am J Epidemiol. 2014;180(3):318–24.

    Article  PubMed  PubMed Central  Google Scholar 

  20. 20.

    Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. 2019;110:12–22.

    Article  PubMed  Google Scholar 

  21. 21.

    Harrell FE Jr. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. Cham: Springer; 2015.

    Book  Google Scholar 

  22. 22.

    Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38.

    Article  PubMed  PubMed Central  Google Scholar 

  23. 23.

    Paul P, Pennell ML, Lemeshow S. Standardizing the power of the Hosmer–Lemeshow goodness of fit test in large data sets. Stat Med. 2013;32(1):67–80.

    Article  PubMed  Google Scholar 

  24. 24.

    Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: The Hosmer–Lemeshow test revisited. Crit Care Med. 2007;35(9):2052–6.

    Article  PubMed  Google Scholar 

  25. 25.

    Kim HY, Jang JW. Sarcopenia in the prognosis of cirrhosis: going beyond the MELD score. World J Gastroenterol. 2015;21(25):7637–47.

    Article  PubMed  PubMed Central  Google Scholar 

  26. 26.

    DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.

    CAS  Article  Google Scholar 

  27. 27.

    Mahmud N, Hubbard RA, Kaplan DE, Taddei TH, Goldberg DS. Risk prediction scores for acute on chronic liver failure development and mortality. Liver Int. 2019.

    Article  PubMed  Google Scholar 

  28. 28.

    Garg H, Kumar A, Garg V, Sharma P, Sharma BC, Sarin SK. Clinical profile and predictors of mortality in patients of acute-on-chronic liver failure. Dig Liver Dis. 2012;44(2):166–71.

    Article  PubMed  Google Scholar 

  29. 29.

    Agrawal S, Duseja A, Gupta T, Dhiman RK, Chawla Y. Simple organ failure count versus CANONIC grading system for predicting mortality in acute-on-chronic liver failure. J Gastroenterol Hepatol. 2015;30(3):575–81.

    Article  PubMed  Google Scholar 

  30. 30.

    Gustot T, Fernandez J, Garcia E, et al. Clinical Course of acute-on-chronic liver failure syndrome and effects on prognosis. Hepatology. 2015;62(1):243–52.

    Article  PubMed  Google Scholar 

Download references


This work was supported by resources and facilities available through the Philadelphia Veterans Affairs Healthcare System and central data repositories maintained by the Veterans Affairs Information Resource Center. The views expressed herein do not reflect position or policy of the Department of Veterans Affairs or the United States government.


Nadim Mahmud is supported by a National Institutes of Health T32 Grant (2-T32-DK007740-21A1).

Author information



Corresponding author

Correspondence to Nadim Mahmud.

Ethics declarations

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors. Institutional review board approval for this study was obtained from the Hospital of the University of Pennsylvania and the Philadelphia Veterans Affairs Hospital.

Compliance with ethical requirements

Karen Xiao, David Kaplan, Tamar Taddei, and Nadim Mahmud declare that they have no conflict of interest. Rebecca Hubbard has received research grants from Humana and Pfizer. David Goldberg has received research grants from Merck, AbbVie, and Gilead.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 232 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Xiao, K.Y., Hubbard, R.A., Kaplan, D.E. et al. Models for acute on chronic liver failure development and mortality in a veterans affairs cohort. Hepatol Int 14, 587–596 (2020).

Download citation


  • Asia pacific association of the study of the liver
  • Acute on chronic liver failure
  • Prediction model
  • Mortality model
  • Outcomes model
  • Veterans health
  • Cirrhosis
  • Liver transplant
  • Veterans outcomes and costs associated with liver disease