Background

Preoperative renal impairment is a well-established predictor of adverse outcomes in patients undergoing coronary artery bypass grafting (CABG) [1,2,3]. In addition, with advances in the fields of nephrology, cardiology, and cardiac surgery, an increasing number of patients with renal dysfunction are being offered coronary revascularization [4].Therefore, accurate preoperative evaluation of renal function is recommended before CABG [5].

Estimated glomerular filtration rate (eGFR) is now considered a more sensitive marker of renal function than serum creatinine alone identifying patients with even mild renal impairment despite normal or nearly normal creatinine levels [6,7,8].

Different formulas to estimate eGFR have been implemented [1, 2, 9] and, amongst these, the Modification of Diet in Renal Disease (MDRD) equation (eGFRMDRD) [1, 2, 9], the more recently defined Chronic Kidney Disease-Epidemiology Collaboration (CKDEPI) equation (eGFRCKD-EPI) [10, 11]along with the Cockcroft-Gault equation formula of Creatinine Clearance (CCG)[7, 8, 12]are the most widely employed.

The predictive value of eGFR on mortality and morbidity following CABG has been widely demonstrated [9, 10, 11, 12]. Nonetheless, papers have concentrated on patients with serum creatinine or eGFR calculated by the CCG equation or MDRD [2, 13, 14] and, at the best of our knowledge, no study exists comparing eGFRMDRD and CCG. eGFRCKD-EPI in their predictive value of post-CABG mortality.

Therefore, in this study we test the reliability of these three formulae in predicting mortality after CABG and compare their discrimination and calibration power. In addition, discrimination and calibration of the three models were also evaluated in relation to factors that may influence the absolute bias of the equations [15].

Methods

This study was performed in accordance with the Declaration of Helsinki and following STROBE guidelines [16]. Consecutive patients undergoing isolated CABG at Careggi Hospital (Florence, Italy) between 2006 and 2017 were retrospectively enrolled in the study.

Definitions

Definitions and calculations were as in our previous research [17]. Kidney dysfunction was defined following the recently updated Kidney Disease Outcomes Quality Initiative (KDOQI) [18] and Kidney Disease Improving Global Outcomes (KDIGO) Guidelines [7].

The CCG [19], MDRD [20] and Chronic Kidney Disease (CKD)-EPI estimate of renal function were calculated as recommended [15, 21] and normalized to 1.73 m2 of the body surface area (BSA) [22] and expressed in ml/min/1.73m2. The body mass index (BMI) was calculated as body weight divided by the square of height, with body weight expressed in kg and height in meters.

Endpoint

The single endpoint was all-cause mortality within 30 days after CABG (n = 3880 cases, 79 deaths) or during index procedure hospitalization- in case of postoperative length of stay > 30 days (n = 528 cases, 36 deaths) which was reported via hospital records or registry information.

Statistical analysis

Continuous data were summarized as mean and standard deviation or median and twenty-fifth to seventy-fifth percentiles in case of skewed distributions. Frequencies were reported for categorical variables. The performances of CCG vs. eGFRCKD-EPI vs. eGFRMDRD were analyzed to determine their discrimination power and calibration [23, 24]. The discrimination performance was assessed by receiver operating characteristic (ROC) and the area under the curve (AUC) with 95% confidence intervals [25,26,27]. Curves were analyzed with De Long, bootstrap, and Venkatraman methods [27]. Furthermore, the model was tested by Somers’ test assuming predictions as perfectly discriminating when Dxy = 1 [28]. Moreover, we employed the Brier score and when it was equal to 0 the prediction could be considered perfect [29].

The calibration performance can be evaluated by generating calibration plots: the perfect calibrated predictions stay on the diagonal, whilst a curve below or above it, respectively, reflects overestimation and underestimation [23, 27, 30].

Agreement between observed frequency and predicted probabilities were tested with the Hosmer-Lemeshow (H-L) goodness-of-fit test, whereas the comparison of actual slope and intercept with the ideal value of 1 and 0 was performed with the U statistic and tested against a χ2 distribution with 2 degrees of freedom.

Discrimination and calibration performances were stratified by renal function, gender, age, body weight, and BMI due to the fact that these variables might influence the performance of the equations. Stratification of calculated eGFR (≥ 60 ml/min/1.73m2;45–59 ml/min/1.73m2, 30–44 ml/min/1.73m2 and ≤ 29 ml/min/1.73m2) was based on updated KDOQI and KDIGO [7, 19] and according to level of calculated EGFR, as well as on the basis of the estimates of the Cockcroft- Gault, MDRD, and CKD-EPI formulas. Using Cohen’s k we tested the agreement between calculated and estimated EGFR .

Clinical cutoffs were used for age (18 to 39, 40 to 59, and ≥ 60 years) [31] and body weight (≤59, 60 to 79, 80 to 99, and ≥ 100 kg) [12]. Stratification for BMI followed the World Health Organization guidelines [32]. To address missing values (Additional file 1), we used fully conditional specification [33] (FCS) multiple imputation (MI) method (1000 replications).

R, version 3.3.1 (R Foundation for Statistical Computing, Wien, Austria) with pROC, rms and Resource Selection packages was employed to carry out statistical analysis.

Significance for hypothesis testing was set at the 0.05 two-tailed level.

Results

Study population

After exclusion of subjects without an available plasma creatinine level (n = 86), body weight (n = 73) or height (n = 37) measurements, those undergoing preoperative dialysis (n = 18), who had undergone previous cardiac surgery (n = 108), who experienced significant (life-threatening) post-operative complications (n = 396) or with mitral insufficiency ≥ moderate (n = 174) the final population consisted of 4408 subjects who remained eligible for inclusion. Patient characteristics are presented in Table 1.

Table 1 Patient characteristics (n = 4408)
Full size table

Mortality

Overall early mortality was 2.6% (n = 115): it was 24/2440 (1%) in patients with EGFR ≥60 ml/min/1.73m2, 20/1170 (1.7%) in those with EGFR between 45 and 59 ml/min/1.73m2, 39/703 (5.5%) in subjects with EGFR ranging from 30 to 44 ml/min/1.73m2 and 32/95 (33.6%) in those with EGFR ≤29 ml/min/1.73m2.

Overall performance

Results of Predictive Performance, Discrimination Power and Calibration are shown in Tables 2 and 3. The c Statistic and the other measures of performance showed that only the CKD-EPI formula had any notable discriminatory power. The MDRD formula shows borderline significant discrimination, given the lower confidence limit for the C statistic is 0.50 The CG formula shows no evidence of being able to discriminate between those who died and those who did not. The ROC curves are plotted in Fig. 1 a-c: The AUC was higher in eGFRCKD-EPI than in the other two and all the comparisons amongst them showed significant differences between the three formulas with best performance by eGFRCKD-EPI.

Table 2 Predictive performance and discrimination power
Full size table
Table 3 Calibration
Full size table
Fig. 1
figure 1

Receiver operating characteristic curves with 95% confidence intervals for eGFRCKD-EPI (a) eGFRMDRD (b) and CCG (c). A curve lying on the diagonal line reflects the performance of a diagnostic test that is no better than chance level. The closer is the curve to the upper left-hand corner the greater is the discriminant testing capacity. Calibration plots of eGFRCKD-EPI (d) and eGFRMDRD (e) and CCG (f). The diagonal line represents the perfect calibration. If the line lies below the ideal curve, the EGFR formula overestimates the outcome, if it is above the ideal curve the formula underestimates the outcome

Full size image

The pattern of calibration (Fig. 1 d-f) was different between the three indices. Indeed, eGFRCKD-EPI was closer to the ideal line with a slight under-prediction when risk was higher but with non-significant p values for the calibration statistics (both, p = 0.40). In contrast, eGFRMDRD and CCG diverged significantly from the ideal diagonal with significant p values for the related summary statistics (both, p = 0.02).

Performance by kidney function

Patients with a higher eGFR were younger (− 0.13 years [95%CI 0.11–1.1] per ml/min/1.73m2 increment in eGFR) with a higher body weight (1.64 kg [95%CI 1.41–2.89]) and a higher BMI (0.62 kg/m2 [95%CI 0.48–0.96]). The analysis of discrimination in the subgroups demonstrated an overall worse performance of eGFRMDRD, (Fig. 2a-c) with significant differences when eGFR was > 29 ml/min/1.73m2. In contrast, when the eGFR was ≤29 ml/min/1.73m2 the performance of MDRD was superior (p = 0.14). CCG showed comparable performance of eGFRCKD-EPI when eGFR was > 44 ml/min/1.73m2.

Fig. 2
figure 2

Patients Stratified by Renal Function. Receiver operating characteristic curves with 95% confidence intervals for eGFRCKD-EPI (a) eGFRMDRD (b). and CCG (c) . Colored curve above the diagonal line perform progressively better the closer they are to the upper left-hand corner. Calibration plots of eGFRCKD-EPI (d) eGFRMDRD (e) and CCG (f). Lines below the ideal curve (dotted line) overestimate the outcome, if they lie above the ideal curve the outcome is underestimated

Full size image

The pattern of calibration was different in the different subgroups of patients (Fig. 2 e-f: eGFRCKD-EPI demonstrated a satisfactory calibration with eGFR > 29 ml/min/1.73m2 but with non-significant p values for the calibration statistics (p = 0.58, p = 0.78, p = 0.39, in the three groups with eGFR > 29 ml/min/1.73m2, respectively). However, it tended to under- influence of predict mortality when eGFR was ≤29 ml/min/1.73m2 (p = 0.04).

In contrast, eGFRMDRD was well calibrated at values of eGFR ≤29 ml/min/1.73m2 (p = 0.06) whereas it diverged significantly from perfect calibration when eGFR was higher than 29 ml/min/1.73m2 (p = 0.02, p = 0.03, p = 0.04, in the three groups with eGFR > 29 ml/min/1.73m2, respectively). Finally, CCG tended to over-prediction when eGFR was < 44 ml/min/1.73m2 (p = 0.03).

Performance by age

Older patients had lower eGFR (− 0.93 ml/min/1.73m2 [95%CI 0.71–1.13] per yearly increment in age), had a lower body weight (− 1.73 kg [95%CI 1.51–2.29]) and a lower BMI (− 0.56 kg/m2 [95%CI 0.48–0.75]). The AUC of the ROC curves (Fig. 3 a-c) was significantly higher for eGFRCKD-EPI in all subgroups. CCG performed better the MDRD equation in the range of 40–59 years whereas it showed the worst performance of the three groups < 40 years.

Fig. 3
figure 3

Patients Stratified by Age. Receiver operating characteristic curves with 95% confidence intervals for eGFRCKD-EPI (a) eGFRMDRD (b) and CCG (c) stratified by age. Colored curve above the diagonal line perform progressively better the closer they are to the upper left-hand corner. Calibration plots of eGFRCKD-EPI (d) and eGFRMDRD (e) and CCG (f). Lines below the ideal curve (dotted line) overestimate the outcome, if they lie above the ideal curve the outcome is underestimated

Full size image

The pattern of calibration was different amongst age subgroups (Fig. 3 d-f): eGFRCKD-EPI was close to the ideal diagonal in the oldest patients whereas it tended to slightly overestimate in the other age groups with non-significant p values for the calibration statistics (p = 0.69). The eGFRMDRD resulted to be well calibrated in the ≥60 year- subgroup (p = 0.53) whereas it demonstrated a significant tendency to over-estimation in the other age subgroups (all, p < 0.05). Finally, CCG tended to over-prediction in the ≥60 year- and 18–39 year-subgroups (both, p = 0.03).

Performance by gender

Compared with men, woman had a lower body weight (− 11.4 kg [95%CI 4.4–12.4]) and a lower BMI (− 1.65 kg/m2 [95%CI 1.14–6.65]).

The eGFRCKD-EPI equation showed a higher AUC in both genders (Fig. 4 a-c) with significant differences compared to the CCG and MDRD equations (p < 0.05 for all comparisons). CCG showed a worse performance compared to eGFRMDRD in both genders.

Fig. 4
figure 4

Patients Stratified by Gender. Receiver operating characteristic curves with 95% confidence intervals for eGFRCKD-EPI (a) eGFRMDRD (b) and CCG (c). . Colored curve above the diagonal line perform progressively better the closer they are to the upper left-hand corner. Calibration plots of eGFRCKD-EPI (d) eGFRMDRD (e) and CCG (f). Lines below the ideal curve (dotted line) overestimate the outcome, if they lie above the ideal curve the outcome is underestimated

Full size image

In men (Fig. 4 d-f) eGFRCKD-EPI reached maximum accuracy whereas it showed a tendency to overestimation in women although calibration statistics were not significant in (both,p = 0.1). In contrast, eGFRMDRD was accurate in women (p = 0.72) and tended to overestimation in men (p = 0.03) whereas CCG significantly overestimated in both sexes (both, p = 0.03).

Performance by BMI

Subjects with a higher BMI were younger (− 0.22 years [95%CI 0.09–0.44] each kg/m2), had a higher body weight (2.89 kg [95%CI 2.73–3.21] each kg/m2) and a higher eGFR (0.78 ml/min/1.73m2 [95%CI 0.56–0.91] each kg/m2).

The AUC of ROC curves (Fig. 5 a-c) were higher with eGFRCKD-EPI no matter what the BMI subgroup was (p < 0.05 for all comparisons). The calibration curves are shown in Fig. 5 D-F: the eGFRCKD-EPI equation was close to the ideal diagonal at any value of BMI with a slight lower accuracy in patients with BMI < 25.0 kg/m2 (p = 0.03). In contrast, the MDRD was more accurate in patients with BMI < 25.0 kg/m2 (p = 0.7) whereas it showed a trend to over-prediction in subjects with BMI ≥25.0 kg/m2 (p = 0.03 and p = 0.04 in patients with BMI 25–29 Kg/m2 and ≥ 30 Kg/m2, respectively). Finally, the CCG formula was the less accurate up to 18.5 kg/m2 (p = 0.03, p = 0.02 and p = 0.02 in patients with BMI > 30 Kg/m2, 25–29 Kg/m2 and ≥ 30 Kg/m2, respectively) with a tendency to over-prediction, while it was comparable to the MDRD formula when in patients with BMI < 25.0 kg/m2 (p = 0.5).

Fig. 5
figure 5

Patients Stratified by Body Mass Index (BMI). Receiver operating characteristic curves with 95% confidence intervals for eGFRCKD-EPI (a) and eGFRMDRD (b). and CCG (c). . Colored curve above the diagonal line perform progressively better the closer they are to the upper left-hand corner. Calibration plots of eGFRCKD-EPI (d) eGFRMDRD (e) and CCG (f). Lines below the ideal curve (dotted line) overestimate the outcome, if they lie above the ideal curve the outcome is underestimated

Full size image

Discussion

Patients with CAD and renal disease have a dismal prognosis [34, 35]. In addition, estimated glomerular filtration rate (eGFR) has a major impact on the outcome of patients undergoing coronary revascularization, either percutaneous coronary intervention or coronary artery bypass grafting (CABG) [10, 36] .

Reduced erythropoietin synthesis and consequent anemia and reduced 1,25(OH) vitamin D production, associated with increased parathyroid hormone levels and higher prevalence of vascular calcification and arteriosclerosis have been reported to explain the association between renal dysfunction and cardiovascular events [37, 38].

In addition, patients with reduced or impaired renal function face additional challenges in the setting of CABG for several reasons: 1) Concomitant factors such as including advanced age, low ejection fraction, history myocardial infarction, and stroke which are themselves determinants of poor outcomes [39]. 2) Detrimental cardiovascular effects by oxidative stress and high levels of homocysteine, hyperuricemia, hypercalcemia, and uremia associated with reduced renal function [40, 41]. 3) Higher incidence of multivessel disease and microvessel disease in such patients [2, 11].

However, little is known whether eGFRs calculated with different formulas have comparable predictive value on post-CABG mortality.

In our recent paper [42] we had shown that the eGFRCKD-EPI equation led to categorization with a significantly lower number of patients at risk for post- CABG complications and with cut-off values of eGFRCKD-EPI predicting early and late events significantly lower than accepted prediction threshold values for post- CABG unfavorable events [2, 41, 43].

In the present study our study we assessed the performance, in terms of discrimination and calibration, of the MDRD, CG formulas and CKD-EPI equations in predicting mortality after CABG in the whole patient population and across different subgroups of patients defined by eGFR, age, gender and body size.

The main findings of our study can be summarized as follows:

  1. 1)

    The overall performance of eGFRCKD-EPI in prediction of post-CABG death is significantly superior to both eGFRMDRD and CG formulas and its calibration curve is close to the ideal prediction over a wide range of thresholds for mortality risk prediction whereas the MDRD and CG equations, show a general trend towards over-prediction.

  2. 2)

    The CKD-EPI equation gave the best overall accuracy and agreement after classification in subgroups of GFR. Furthermore, it had a greater accuracy in patients with an eGFR > 30 ml/min/1.73m2 whereas it showed a trend towards under-predicting mortality when the eGFR fell below 30 ml/min/1.73m2. In contrast, eGFRMDRD confirmed [15] to be the most reliable in patients with highly compromised renal function whilst CCG showed comparable performance of eGFRCKD-EPI when eGFR was > 44 ml/min/1.73m2.

  3. 3)

    Previous studies have demonstrated that the performance of eGFR equations depends on the stage of CKD [44], thus being greatly influenced by the value of glomerular filtration rate [15]. In addition, the MDRD equation resulted in imprecise and underestimates of eGFR at higher renal function levels [45]. In our experience, the accurateness of the CKD- EPI formula in predicting post-CABG mortality was independent of age and gender whereas eGFRMDRD overestimated the prediction in younger patients and in men while it was accurate in women and patients ≥60 years and CCG tended to over-prediction in the ≥60 year- and 18–39 year-subgroups and in both genders. This might be related to the uncertain reliability of these formulas in reflecting the true renal function [46, 47]

  4. 4)

    Since all three formulas rely on serum creatinine as the indicator for the rate of glomerular filtration and because serum creatinine correlates with muscle mass and nutritional status, the performance of the formulas might be influenced by body composition. This was assessed by studying the influence of body mass or BMI on eGFR, which, in our experience, did not affect the CKD-EPI equation whose calibration curve was close to the ideal diagonal at any value of BMI. In contrast, the MDRD was accurate only in overweight patients and those with body mass ≥ 30.0 kg/m2. These results are in accordance with Michels et al. [15] who found that MDRD provided greatest accuracy in defining renal function (97.0%) in subjects with the highest body weight whereas other studies showed no relation or positive correlation concluding that no creatinine-based method is reliable in the obese [48]. Lastly, the CCG formula was the least accurate up to 18.5 kg/m2 while it was comparable to the MDRD formula in smaller patients.

Renal function is regularly included in all risk stratification models in cardiac surgical patients. Two well-recognized risk models assess cardiovascular outcomes of patients undergoing CABG: the EuroSCORE and the Society of Thoracic Surgeons (STS) National Adult Cardiac Database [49]. The first employs eGFR calculated with CCG formula and value ranges that are not concordant with National Kidney Foundation recommendations [50] whereas the STS risk score incorporates a continuous parameter for serum creatinine and a binary variable for hemodialysis [51]. Based on KDIGO clinical practice Guidelines [8] and previous evidence [15], it would be of great interest to test, in a broad patient population, the eGFRCKD-EPI formula incorporated into CABG risk prediction algorithm, re-estimating the weight for all the variables in the predictive tool, to compare the predictive performance of such a model to algorithms currently in use. At this point, in the absence of validation studies, it is impossible to understand whether the use of eGFRCKD-EPI in stratification models would make a valuable contribution to improve the predictive value of the algorithm. Further research is warranted.

Study limitations

This study has some limitation that should be highlighted. Firstly, its retrospective nature makes it impossible to draw final conclusions. Secondly, the population is relatively small, and assessment of the equations was carried out in a restricted study population (i.e. post CABG patients), limiting extrapolation of findings to other cohorts such as myocardial infarction, heart failure etc. Thirdly, the patient population has several variations from most CABG profiles: low number of female, low incidence of adult onset diabetes mellitus, unstable angina and MI < 30 days and high number of patients receiving 1–2 grafts. Fourthly, patients with associated procedures were excluded and this could introduce another bias. We wanted to test the three indices excluding as much as possible confounding factors. Fifthly, preoperative eGFR was calculated on a single measurement and therefore susceptible of being influenced by cardiac function and therapy. Sixthly, preoperative renal function was unknown which could have post-CABG survival. Seventhly, eGFRCKD-EPI still has the limitation of being related to muscle mass, thus other filtration markers such as serum cystatin might have helped us in overcoming this issue. Eighthly, data presented in this paper did not say anything about which equation is the better predictor of true GFR, but it was beyond the aim of the paper that was explore which eGFR formula is the best predictor of mortality. The two things may go hand-in-hand, but this cannot be concluded from the existing data and it will be object of upcoming research. Finally, neither we compare the performance of the three formulae within specific risk scores, nor did not test the performance of eGFRCKD-EPI on postoperative renal failure but these were beyond the aim of the present study.

Conclusions

In general, CKD-EPI gives the best prediction of death after CABG with unsatisfactory accuracy and calibration only in patients with severe CKD. In contrast, the CG and MDRD equations were inaccurate in predicting mortality in a clinically significant proportion of patients. eGFRCKD-EPI should be incorporated into CABG risk-assessment algorithms to provide patients and their family members the most accurate risk prediction.