Evaluation of the ACS NSQIP Surgical Risk Calculator in Elderly Patients Undergoing Hepatectomy for Hepatocellular Carcinoma



The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) surgical risk calculator (SRC) aims to help predict patient-specific risk for morbidity and mortality. The performance of the SRC among an elderly population undergoing curative-intent hepatectomy for hepatocellular carcinoma (HCC) remains unknown.


Patients > 70 years of age who underwent hepatectomy for HCC between 1998 and 2017 were identified using a multi-institutional international database. To estimate the performance of SRC, 12 observed postoperative outcomes were compared with median SRC-predicted risk, and C-statistics and Brier scores were calculated.


Among 500 patients, median age was 75 years (IQR 72–78). Most patients (n = 324, 64.8%) underwent a minor hepatectomy, while 35.2% underwent a major hepatectomy. The observed incidence of venous thromboembolism (VTE) (3.2%) and renal failure (RF) (4.4%) exceeded the median predicted risk (VTE, 1.8%; IQR 1.5–3.1 and RF, 1.0%; IQR 0.5–2.0). In contrast, the observed incidence of 30-day readmission (7.0%) and non-home discharge (2.5%) was lower than median-predicted risk (30-day readmission, 9.4%; IQR 7.4–12.8 and non-home discharge, 5.7%; IQR 3.3–11.7). Only 57.8% and 71.2% of patients who experienced readmission (C-statistic, 0.578; 95%CI 0.468–0.688) or mortality (C-statistic, 0.712; 95%CI 0.508–0.917) were correctly identified by the model.


Among elderly patients undergoing hepatectomy for HCC, the SRC underestimated the risk of complications such as VTE and RF, while being no better than chance in estimating the risk of readmission. The ACS SRC has limited clinical applicability in estimating perioperative risk among elderly patients being considered for hepatic resection of HCC.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2


  1. 1.

    Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68(6):394–424.

    Google Scholar 

  2. 2.

    Bertuccio P, Turati F, Carioli G, Rodriguez T, La Vecchia C, Malvezzi M, et al. Global trends and predictions in hepatocellular carcinoma mortality. J Hepatol. 2017;67(2):302–9.

    PubMed  Google Scholar 

  3. 3.

    Beal EW, Tumin D, Kabir A, Moris D, Zhang XF, Chakedis J, et al. Trends in the Mortality of Hepatocellular Carcinoma in the United States. J Gastrointest Surg. 2017;21(12):2033–8.

    PubMed  Google Scholar 

  4. 4.

    White DL, Thrift AP, Kanwal F, Davila J, El-Serag HB. Incidence of Hepatocellular Carcinoma in All 50 United States, From 2000 Through 2012. Gastroenterology. 2017;152(4):812–20.e5.

    PubMed  Google Scholar 

  5. 5.

    Ueno M, Hayami S, Tani M, Kawai M, Hirono S, Yamaue H. Recent trends in hepatectomy for elderly patients with hepatocellular carcinoma. Surg Today. 2014;44(9):1651–9.

    PubMed  Google Scholar 

  6. 6.

    Rosero EB, Cheng GS, Khatri KP, Joshi GP. Evaluation of epidural analgesia for open major liver resection surgery from a US inpatient sample. Proceedings (Baylor University Medical Center). 2014;27(4):305–12.

    PubMed Central  Google Scholar 

  7. 7.

    Hankey BF, Ries LA, Kosary CL, Feuer EJ, Merrill RM, Clegg LX, et al. Partitioning linear trends in age-adjusted rates. Cancer causes & control : CCC. 2000;11(1):31–5.

    CAS  PubMed  Google Scholar 

  8. 8.

    Okinaga H, Yasunaga H, Hasegawa K, Fushimi K, Kokudo N. Short-Term Outcomes following Hepatectomy in Elderly Patients with Hepatocellular Carcinoma: An Analysis of 10,805 Septuagenarians and 2,381 Octo- and Nonagenarians in Japan. Liver Cancer. 2018;7(1):55–64.

    PubMed  Google Scholar 

  9. 9.

    Al-Refaie WB, Parsons HM, Henderson WG, Jensen EH, Tuttle TM, Vickers SM, et al. Major cancer surgery in the elderly: results from the American College of Surgeons National Surgical Quality Improvement Program. Annals of surgery. 2010;251(2):311–8.

    PubMed  Google Scholar 

  10. 10.

    Kneuertz PJ, Pitt HA, Bilimoria KY, Smiley JP, Cohen ME, Ko CY, et al. Risk of morbidity and mortality following hepato-pancreato-biliary surgery. Journal of gastrointestinal surgery : official journal of the Society for Surgery of the Alimentary Tract. 2012;16(9):1727–35.

    Google Scholar 

  11. 11.

    Oliveira-Cunha M, Malde DJ, Aldouri A, Morris-Stiff G, Menon KV, Smith AM. Results of pancreatic surgery in the elderly: is age a barrier? HPB : the official journal of the International Hepato Pancreato Biliary Association. 2013;15(1):24–30.

    Google Scholar 

  12. 12.

    Tzeng CW, Cooper AB, Vauthey JN, Curley SA, Aloia TA. Predictors of morbidity and mortality after hepatectomy in elderly patients: analysis of 7621 NSQIP patients. HPB : the official journal of the International Hepato Pancreato Biliary Association. 2014;16(5):459–68.

    Google Scholar 

  13. 13.

    Bergquist JR, Shubert CR, Ubl DS, Thiels CA, Kendrick ML, Truty MJ, et al. Risk by indication for pancreaticoduodenectomy in patients 80 years and older: a study from the American College of Surgeons National Surgical Quality Improvement Program. HPB : the official journal of the International Hepato Pancreato Biliary Association. 2016;18(11):900–7.

    Google Scholar 

  14. 14.

    Cho SW, Steel J, Tsung A, Marsh JW, Geller DA, Gamblin TC. Safety of Liver Resection in the Elderly: How Important Is Age? Annals of surgical oncology. 2011;18(4):1088–95.

    CAS  PubMed  Google Scholar 

  15. 15.

    Ruzzenente A, Conci S, Ciangherotti A, Campagnaro T, Valdegamberi A, Bertuzzo F, et al. Impact of age on short-term outcomes of liver surgery: Lessons learned in 10-years' experience in a tertiary referral hepato-pancreato-biliary center. Medicine. 2017;96(20):e6955.

    PubMed  PubMed Central  Google Scholar 

  16. 16.

    Merkow RP, Bilimoria KY, Tomlinson JS, Paruch JL, Fleming JB, Talamonti MS, et al. Postoperative Complications Reduce Adjuvant Chemotherapy Use in Resectable Pancreatic Cancer. Annals of surgery. 2014;260(2):372–7.

    PubMed  Google Scholar 

  17. 17.

    Wu W, He J, Cameron JL, Makary M, Soares K, Ahuja N, et al. The Impact of Postoperative Complications on the Administration of Adjuvant Therapy Following Pancreaticoduodenectomy for Adenocarcinoma. Annals of surgical oncology. 2014;21(9):2873–81.

    PubMed  PubMed Central  Google Scholar 

  18. 18.

    Grady D, Berkowitz SA. Why is a good clinical prediction rule so hard to find? Archives of Internal Medicine. 2011;171(19):1701–2.

    PubMed  Google Scholar 

  19. 19.

    Wilson PW, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97(18):1837–47.

    CAS  PubMed  Google Scholar 

  20. 20.

    Cologne KG, Keller DS, Liwanag L, Devaraj B, Senagore AJ. Use of the American College of Surgeons NSQIP Surgical Risk Calculator for Laparoscopic Colectomy: how good is it and how can we improve it? Journal of the American College of Surgeons. 2015;220(3):281–6.

    PubMed  Google Scholar 

  21. 21.

    Beal EW, Lyon E, Kearney J, Wei L, Ethun CG, Black SM, et al. Evaluating the American College of Surgeons National Surgical Quality Improvement project risk calculator: results from the U.S. Extrahepatic Biliary Malignancy Consortium. HPB : the official journal of the International Hepato Pancreato Biliary Association. 2017;19(12):1104–11.

    Google Scholar 

  22. 22.

    Pencina MJ, D'Agostino RB. Evaluating Discrimination of Risk Prediction Models: The C Statistic. JAMA. 2015;314(10):1063–4.

    CAS  PubMed  Google Scholar 

  23. 23.

    Austin PC, Steyerberg EW. Interpreting the concordance statistic of a logistic regression model: relation to the variance and odds ratio of a continuous explanatory variable. BMC Med Res Methodol. 2012;12:82.

    PubMed  PubMed Central  Google Scholar 

  24. 24.

    Rufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol. 2010;63(8):938–9; author reply 9.

    PubMed  Google Scholar 

  25. 25.

    Centers for Medicare and Medicaid Services. The Administration, Cost, and Impact of the Quality Improvement Organization Program for Medicare Beneficiaries for Fiscal Year 20172018. Available from: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityImprovementOrgs/Downloads/Annual-Report-to-Congress-QIO-Program-Fiscal-Year-2017.pdf. Accessed 4 Jan 2019.

  26. 26.

    Rix TE, Bates T. Pre-operative risk scores for the prediction of outcome in elderly people who require emergency surgery. World Journal of Emergency Surgery. 2007;2(1):16.

    PubMed  PubMed Central  Google Scholar 

  27. 27.

    Edelstein AI, Kwasny MJ, Suleiman LI, Khakhkhar RH, Moore MA, Beal MD, et al. Can the American College of Surgeons Risk Calculator Predict 30-Day Complications After Knee and Hip Arthroplasty? The Journal of arthroplasty. 2015;30(9 Suppl):5–10.

    PubMed  Google Scholar 

  28. 28.

    Adegboyega TO, Borgert AJ, Lambert PJ, Jarman BT. Applying the National Surgical Quality Improvement Program risk calculator to patients undergoing colorectal surgery: theory vs reality. American journal of surgery. 2017;213(1):30–5.

    PubMed  Google Scholar 

  29. 29.

    Teoh D, Halloway RN, Heim J, Vogel RI, Rivard C. Evaluation of the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator in Gynecologic Oncology Patients Undergoing Minimally Invasive Surgery. Journal of minimally invasive gynecology. 2017;24(1):48–54.

    PubMed  Google Scholar 

  30. 30.

    Mogal HD, Fino N, Clark C, Shen P. Comparison of observed to predicted outcomes using the ACS NSQIP risk calculator in patients undergoing pancreaticoduodenectomy. Journal of surgical oncology. 2016;114(2):157–62.

    PubMed  PubMed Central  Google Scholar 

  31. 31.

    Kishida N, Hibi T, Itano O, Okabayashi K, Shinoda M, Kitago M, et al. Validation of hepatectomy for elderly patients with hepatocellular carcinoma. Ann Surg Oncol. 2015;22(9):3094–101.

    PubMed  Google Scholar 

  32. 32.

    Joseph B, Zangbar B, Pandit V, Fain M, Mohler MJ, Kulvatunyou N, et al. Emergency General Surgery in the Elderly: Too Old or Too Frail? J Am Coll Surg. 2016;222(5):805–13.

    PubMed  Google Scholar 

  33. 33.

    Vitale A, Spolverato G, Bagante F, Gani F, Popescu I, Marques HP, et al. A multi-institutional analysis of elderly patients undergoing a liver resection for intrahepatic cholangiocarcinoma. J Surg Oncol. 2016;113(4):420–6.

    PubMed  Google Scholar 

  34. 34.

    Jin S, Fu Q, Wuyun G, Wuyun T. Management of post-hepatectomy complications. World J Gastroenterol. 2013;19(44):7983–91.

    PubMed  PubMed Central  Google Scholar 

  35. 35.

    Lafaro K, Buettner S, Maqsood H, Wagner D, Bagante F, Spolverato G, et al. Defining Post Hepatectomy Liver Insufficiency: Where do We stand? J Gastrointest Surg. 2015;19(11):2079–92.

    PubMed  Google Scholar 

  36. 36.

    Paredes AZ, Beal EW, Bagante F, Dillhoff ME, Cloyd JM, Pawlik TM. Patterns of readmission among the elderly after hepatopancreatobiliary surgery. American journal of surgery. 2018.

  37. 37.

    Adam R, Frilling A, Elias D, Laurent C, Ramos E, Capussotti L, et al. Liver resection of colorectal metastases in elderly patients. The British journal of surgery. 2010;97(3):366–76.

    CAS  PubMed  Google Scholar 

  38. 38.

    Yasunaga H, Horiguchi H, Matsuda S, Fushimi K, Hashimoto H, Ohe K, et al. Relationship between hospital volume and operative mortality for liver resection: Data from the Japanese Diagnosis Procedure Combination database. Hepatology research : the official journal of the Japan Society of Hepatology. 2012;42(11):1073–80.

    Google Scholar 

  39. 39.

    Saraiva MD, Karnakis T, Gil-Junior LA, Oliveira JC, Suemoto CK, Jacob-Filho W. Functional Status is a Predictor of Postoperative Complications After Cancer Surgery in the Very Old. Ann Surg Oncol. 2017;24(5):1159–64.

    PubMed  Google Scholar 

  40. 40.

    Wagner D, DeMarco MM, Amini N, Buttner S, Segev D, Gani F, et al. Role of frailty and sarcopenia in predicting outcomes among patients undergoing gastrointestinal surgery. World J Gastrointest Surg. 2016;8(1):27–40.

    PubMed  PubMed Central  Google Scholar 

  41. 41.

    Gani F, Cerullo M, Amini N, Buettner S, Margonis GA, Sasaki K, et al. Frailty as a Risk Predictor of Morbidity and Mortality Following Liver Surgery. J Gastrointest Surg. 2017;21(5):822–30.

    PubMed  Google Scholar 

  42. 42.

    Valero V, Amini N, Spolverato G, Weiss MJ, Hirose K, Dagher NN, et al. Sarcopenia adversely impacts postoperative complications following resection or transplantation in patients with primary liver tumors. J Gastrointest Surg. 2015;19(2):272–81.

    PubMed  Google Scholar 

  43. 43.

    Robinson TN, Eiseman B, Wallace JI, Church SD, McFann KK, Pfister SM, et al. Redefining geriatric preoperative assessment using frailty, disability and co-morbidity. Ann Surg. 2009;250(3):449–55.

    PubMed  Google Scholar 

  44. 44.

    Cohen ME, Liu Y, Ko CY, Hall BL. An Examination of American College of Surgeons NSQIP Surgical Risk Calculator Accuracy. J Am Coll Surg. 2017;224(5):787–95.e1.

    PubMed  Google Scholar 

  45. 45.

    Landefeld CS, Palmer RM, Kresevic DM, Fortinsky RH, Kowal J. A randomized trial of care in a hospital medical unit especially designed to improve the functional outcomes of acutely ill older patients. N Engl J Med. 1995;332(20):1338–44.

    CAS  PubMed  Google Scholar 

  46. 46.

    Carli F, Brown R, Kennepohl S. Prehabilitation to enhance postoperative recovery for an octogenarian following robotic-assisted hysterectomy with endometrial cancer. Can J Anaesth. 2012;59(8):779–84.

    PubMed  Google Scholar 

  47. 47.

    Cerullo M, Chen SY, Dillhoff M, Schmidt C, Canner JK, Pawlik TM. Association of Hospital Market Concentration With Costs of Complex Hepatopancreaticobiliary Surgery. JAMA surgery. 2017;152(9):e172158.

    PubMed  PubMed Central  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Timothy M. Pawlik.

Ethics declarations

The study was approved by the Institutional Review Boards of all participating institutions.

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sahara, K., Paredes, A.Z., Merath, K. et al. Evaluation of the ACS NSQIP Surgical Risk Calculator in Elderly Patients Undergoing Hepatectomy for Hepatocellular Carcinoma. J Gastrointest Surg 24, 551–559 (2020). https://doi.org/10.1007/s11605-019-04174-4

Download citation


  • Risk calculator
  • Elderly
  • Hepatectomy