Background and purpose
The diagnosis of acute on chronic liver failure (ACLF) carries a high short-term mortality, making early identification of at-risk patients crucial. To date, there are no models that predict which patients with compensated cirrhosis will develop ACLF, and limited models exist to predict ACLF mortality. We sought to create novel risk prediction models using a large North American cohort.
We performed a retrospective study of 75,922 patients with compensated cirrhosis from the Veterans Outcomes and Costs Associated with Liver Disease (VOCAL) dataset. Using 70% derivation/30% validation sets, we identified ACLF patients using the Asian Pacific Association of Liver (APASL) definition. Multivariable logistic regression was used to derive prediction models (called VOCAL-Penn) for developing ACLF at 3, 6, and 12 months. We then created prediction models for ACLF mortality at 28 and 90 days.
The VOCAL-Penn models for ACLF development had very good discrimination [concordance (C) statistics of 0.93, 0.92, and 0.89 at 3, 6, and 12 months, respectively] and calibration. The mortality models also had good discrimination at 28 and 90 days (C statistics 0.89 and 0.88, respectively), outperforming the Model for End-stage Liver Disease (MELD), MELD-sodium, and the APASL ACLF Research Consortium ACLF scores.
We have developed novel tools for predicting development of ACLF in compensated cirrhosis patients, as well as for ACLF mortality. These tools may be used to proactively guide patient follow-up, prognostication, escalation of care, and transplant evaluation.
Receiver operating characteristic (ROC) curves for predicting development of APASL ACLF at 3 months (a), 6 months (b), and 1 year (c)
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Tapper EB, Parikh ND. Mortality due to cirrhosis and liver cancer in the United States, 1999–2016: observational study. BMJ. 2018;362:k2817. https://doi.org/10.1136/bmj.k2817.
Sarin SK, Choudhury A. Acute-on-chronic liver failure: terminology, mechanisms and management. Nat Rev Gastroenterol Hepatol. 2016;13(3):131–49. https://doi.org/10.1038/nrgastro.2015.219.
Moreau R, Jalan R, Gines P, et al. Acute-on-chronic liver failure is a distinct syndrome that develops in patients with acute decompensation of cirrhosis. Gastroenterology. 2013;144(7):1426–37. https://doi.org/10.1053/j.gastro.2013.02.042(1437.e1–9).
O’Grady J. Con: acute-on-chronic liver failure. Liver Transpl. 2017;23(10):1325–7. https://doi.org/10.1002/lt.24809.
Nadim M, Kaplan D, Taddei T, Goldberg DS. Reply to letters to the editor: Diverging definitions and dividing lines in acute‐on‐chronic liver failure. Hepatology. https://www-ncbi-nlm-nih-gov.proxy.library.upenn.edu/pubmed/31243784/. Accessed 23 Dec 2019.
Sarin SK, Choudhury A, Sharma MK, et al. Acute-on-chronic liver failure: consensus recommendations of the Asian Pacific association for the study of the liver (APASL): an update. Hepatol Int. 2019;13(4):353–90. https://doi.org/10.1007/s12072-019-09946-3.
Singh H, Pai CG. Defining acute-on-chronic liver failure: east, west or middle ground? World J Hepatol. 2015;7(25):2571–7. https://doi.org/10.4254/wjh.v7.i25.2571.
O’Leary J, Reddy K, Garcia-Tsao G, et al. NACSELD acute-on-chronic liver failure (NACSELD-ACLF) score predicts 30-day survival in hospitalized patients with cirrhosis. Hepatology. 2018;67(6):2367–74. https://doi.org/10.1002/hep.29773.
Hernaez R, Kramer JR, Liu Y, et al. Prevalence and short-term mortality of acute-on-chronic liver failure: a national cohort study from the USA. J Hepatol. 2019;70(4):639–47. https://doi.org/10.1016/j.jhep.2018.12.018.
Choudhury A, Jindal A, Maiwall R, et al. Liver failure determines the outcome in patients of acute-on-chronic liver failure (ACLF): comparison of APASL ACLF research consortium (AARC) and CLIF-SOFA models. Hepatol Int. 2017;11(5):461–71. https://doi.org/10.1007/s12072-017-9816-z.
Kaplan DE, Dai F, Aytaman A, et al. Development and performance of an algorithm to estimate the Child–Turcotte–Pugh score from a National Electronic Healthcare Database. Clin Gastroenterol Hepatol. 2015;13(13):2333.e1–41.e61.e6.e6. https://doi.org/10.1016/j.cgh.2015.07.010.
Goldberg DS, French B, Forde KA, et al. Association of distance from a transplant center with access to waitlist placement, receipt of liver transplantation, and survival among US veterans. JAMA. 2014;311(12):1234–43. https://doi.org/10.1001/jama.2014.2520.
Mahmud N, Kaplan DE, Taddei TH, Goldberg DS. Incidence and mortality of acute-on-chronic liver failure using two definitions in patients with compensated cirrhosis. Hepatology. 2019;69(5):2150–63. https://doi.org/10.1002/hep.30494.
Mahmud N, Sundaram V, Kaplan DE, Taddei TH, Goldberg DS. Grade 1 acute on chronic liver failure is a predictor for subsequent grade 3 failure. Hepatology. 2019. https://doi.org/10.1002/hep.31012.
Goldberg D, Lewis J, Halpern S, Weiner M, Lo RV. Validation of three coding algorithms to identify patients with end-stage liver disease in an administrative database. Pharmacoepidemiol Drug Saf. 2012;21(7):765–9. https://doi.org/10.1002/pds.3290.
Lapointe-Shaw L, Georgie F, Carlone D, et al. Identifying cirrhosis, decompensated cirrhosis and hepatocellular carcinoma in health administrative data: a validation study. PLoS ONE. 2018;13(8):e0201120. https://doi.org/10.1371/journal.pone.0201120.
Bradley KA, DeBenedetti AF, Volk RJ, Williams EC, Frank D, Kivlahan DR. AUDIT-C as a brief screen for alcohol misuse in primary care. Alcohol Clin Exp Res. 2007;31(7):1208–17. https://doi.org/10.1111/j.1530-0277.2007.00403.x.
Beste LA, Leipertz SL, Green PK, Dominitz JA, Ross D, Ioannou GN. Trends in burden of cirrhosis and hepatocellular carcinoma by underlying liver disease in US veterans, 2001–2013. Gastroenterology. 2015;149(6):1471.e5–82.e5. https://doi.org/10.1053/j.gastro.2015.07.056(quiz e17–18).
Smith GCS, Seaman SR, Wood AM, Royston P, White IR. Correcting for optimistic prediction in small data sets. Am J Epidemiol. 2014;180(3):318–24. https://doi.org/10.1093/aje/kwu140.
Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. 2019;110:12–22. https://doi.org/10.1016/j.jclinepi.2019.02.004.
Harrell FE Jr. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. Cham: Springer; 2015.
Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38. https://doi.org/10.1097/EDE.0b013e3181c30fb2.
Paul P, Pennell ML, Lemeshow S. Standardizing the power of the Hosmer–Lemeshow goodness of fit test in large data sets. Stat Med. 2013;32(1):67–80. https://doi.org/10.1002/sim.5525.
Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: The Hosmer–Lemeshow test revisited. Crit Care Med. 2007;35(9):2052–6. https://doi.org/10.1097/01.CCM.0000275267.64078.B0.
Kim HY, Jang JW. Sarcopenia in the prognosis of cirrhosis: going beyond the MELD score. World J Gastroenterol. 2015;21(25):7637–47. https://doi.org/10.3748/wjg.v21.i25.7637.
DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.
Mahmud N, Hubbard RA, Kaplan DE, Taddei TH, Goldberg DS. Risk prediction scores for acute on chronic liver failure development and mortality. Liver Int. 2019. https://doi.org/10.1111/liv.14328.
Garg H, Kumar A, Garg V, Sharma P, Sharma BC, Sarin SK. Clinical profile and predictors of mortality in patients of acute-on-chronic liver failure. Dig Liver Dis. 2012;44(2):166–71. https://doi.org/10.1016/j.dld.2011.08.029.
Agrawal S, Duseja A, Gupta T, Dhiman RK, Chawla Y. Simple organ failure count versus CANONIC grading system for predicting mortality in acute-on-chronic liver failure. J Gastroenterol Hepatol. 2015;30(3):575–81. https://doi.org/10.1111/jgh.12778.
Gustot T, Fernandez J, Garcia E, et al. Clinical Course of acute-on-chronic liver failure syndrome and effects on prognosis. Hepatology. 2015;62(1):243–52. https://doi.org/10.1002/hep.27849.
This work was supported by resources and facilities available through the Philadelphia Veterans Affairs Healthcare System and central data repositories maintained by the Veterans Affairs Information Resource Center. The views expressed herein do not reflect position or policy of the Department of Veterans Affairs or the United States government.
Nadim Mahmud is supported by a National Institutes of Health T32 Grant (2-T32-DK007740-21A1).
This article does not contain any studies with human participants or animals performed by any of the authors. Institutional review board approval for this study was obtained from the Hospital of the University of Pennsylvania and the Philadelphia Veterans Affairs Hospital.
Compliance with ethical requirements
Karen Xiao, David Kaplan, Tamar Taddei, and Nadim Mahmud declare that they have no conflict of interest. Rebecca Hubbard has received research grants from Humana and Pfizer. David Goldberg has received research grants from Merck, AbbVie, and Gilead.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Xiao, K.Y., Hubbard, R.A., Kaplan, D.E. et al. Models for acute on chronic liver failure development and mortality in a veterans affairs cohort. Hepatol Int 14, 587–596 (2020). https://doi.org/10.1007/s12072-020-10060-y
- Asia pacific association of the study of the liver
- Acute on chronic liver failure
- Prediction model
- Mortality model
- Outcomes model
- Veterans health
- Liver transplant
- Veterans outcomes and costs associated with liver disease