National Cardiovascular Data Registry-Acute Kidney Injury (NCDR) vs. Mehran risk models for prediction of contrast-induced nephropathy and need for dialysis after coronary angiography in a German patient cohort

Background Contrast-induced nephropathy (CIN) is a major adverse event in patients undergoing coronary angiography. The Mehran risk model is the gold-standard for CIN risk prediction. However, its performance in comparison to more contemporary National Cardiovascular Data Registry-Acute Kidney Injury (NCDR-AKI) risk models remains unknown. We aimed to compare both in this study. Methods and results Predictions of Mehran and NCDR-AKI risk models and clinical events of CIN and need for dialysis were assessed in a total of 2067 patients undergoing coronary angiography with or without percutaneous coronary intervention. Risk models were compared regarding discrimination (receiver operating characteristic analysis), net reclassification improvement (NRI) and calibration (graphical and statistical analysis). The NCDR risk model showed superior risk discrimination for predicting CIN (NCDR c-index 0.75, 95% CI 0.72–0.78; vs. Mehran c-index 0.69, 95% CI 0.66–0.72, p < 0.01), and continuous NRI (0.22; 95% CI 0.12–0.32; p < 0.01) compared to the Mehran model. The NCDR risk model tended to underestimate the risk of CIN, while the Mehran model was more evenly calibrated. For the prediction of need for dialysis, NCDR-AKI-D also discriminated risk better (c-index 0.85, 95% CI 0.79–0.91; vs. Mehran c-index 0.75, 95% CI 0.66–0.84; pNCDRvsMehran < 0.01), but continuous NRI showed no benefit and calibration analysis revealed an underestimation of dialysis risk. Conclusion In German patients undergoing coronary angiography, the modern NCDR risk model for predicting contrast-induced nephropathy showed superior discrimination compared to the Mehran model while showing less accurate calibration. Results for the outcome ‘need for dialysis’ were equivocal. Graphic abstract Supplementary Information The online version contains supplementary material available at 10.1007/s40620-021-01124-9.


Introduction
Contrast-induced nephropathy (CIN) is a major adverse event for patients undergoing cardiac catheterization and is associated with increased mortality [1,2]. CIN is defined as an impairment of renal function measured by an absolute (> 0.5 mg/dl) or relative (> 25%) increase in serum creatinine within 48-72 h after application of iodinated contrast media [3]. CIN after coronary angiography heralds a higher patient risk for long-term decline in renal function [4] and thus has profound prognostic relevance.
Several risk factors have been found associated with an increased risk of developing CIN [5]. Taking the most important risk factors (shock, congestive heart failure, older age, anemia, diabetes, contrast media volume, use of an intraaortic balloon pump and chronic kidney disease) into account, in 2004 Mehran et al. developed a simple risk score to estimate the risk of developing CIN and need for dialysis [6]. To date, the original score has been validated in several cohorts [7,8] and is the most established risk model for predicting CIN and need for dialysis. However, the more contemporary risk models of the National Cardiovascular Data Registry-Acute kidney Injury (NCDR-AKI) and NCDR-AKI-Dialysis (NCDR-AKI-D) based on data derived from 900,000 patients undergoing percutaneous coronary interventions (PCI) in the NCDR Cath-PCI Registry [9] showed promising performance in our external validation study [10]. The NCDR-AKI and NCDR-AKI-D models stratify the patients' risk of developing contrast-CIN by the following variables; age, acute decompensated heart failure, glomerular filtration rate, and diabetes, history of cardiovascular disease and heart failure, cardiogenic shock, cardiac arrest, anemia, use of an intra-aortic balloon pump.
Overall, identifying patients at risk and adequate preventive management remain key factors for improving patient outcomes. We thus aimed to compare the modern NCDR risk models to the established Mehran model for predicting contrast-induced nephropathy in a contemporary German patient cohort undergoing cardiac catheterization procedures with and without percutaneous coronary intervention.

Patient characteristics, clinical settings and data collection
Patients undergoing invasive coronary angiography at University Hospital Duesseldorf between 2014 and 2018 for reasons ranging from elective cardiac catheterization procedures to presentation with acute non-ST-segment elevation myocardial infarction (NSTEMI) and ST-segment elevation myocardial infarction (STEMI) were included in the study. All patients were treated according to current European guideline-recommended clinical practice for acute and chronic coronary syndromes [11][12][13][14]. Patient and procedural characteristics, as well as symptoms and results diagnostic testing were extracted from medical records and recorded into a dedicated database. History of chronic kidney disease was defined as a previous decline in renal function according to the current Kidney Disease: Improving Global Outcomes (KDIGO) chronic kidney disease guidelines [15].

Clinical outcomes definitions
The main outcomes of interest were CIN and need for dialysis. CIN was defined as an absolute increase in creatinine of more than 0.5 mg/dl or more than 25% compared to the baseline value within 48-72 h after cardiac catheterization [6]. Need for dialysis was defined as renal replacement therapy due to insufficient urine output, high retention parameters, metabolic acidosis or relevant electrolyte disorders during hospitalization. CIN events in patients who underwent chronic renal replacement therapy were not considered. Secondary outcomes, e.g. in-hospital mortality, major bleeding or stroke are additionally reported (Table 2).

Statistics
Data collection and descriptive statistics were done using Access (Microsoft), Excel (Microsoft) and SPSS 26 (IBM). Ordinal/categorical variables are presented as counts and % of total, continuous data are presented as means ± standard deviation (SD). Statistical significance was assumed at a two-sided α probability < 0.05 for all analyses.

Missing data
Missing data were imputed only if they were relevant for risk model calculation, either to the most common value (binary/ordinal/categorical variables) or to the mean (continuous variables) of the respective clinical setting subgroup (NSTEMI, STEMI or elective). Details on missing variables as % of all model-relevant variables are reported in the Results section. Patients lacking information on relevant outcome variables (CIN or need for dialysis) were excluded. Sensitivity analysis of patients with complete datasets was additionally performed (no imputation).

Risk model calculation
The Mehran B risk model was calculated to predict CIN and the need for dialysis [6]. The NCDR-AKI risk model was calculated to predict CIN [9], while the NCDR-AKI-D model was used to predict the need for dialysis [9]. For the calculation of individual risk scores, all relevant scoring parameters (Suppl. Tables 1-3) were weighted according to risk model definitions and summarized to obtain an individual summary risk score for each patient. Summary risk scores were assigned to event probabilities according to risk model definitions ( [6,9], Suppl. Tables 1-3).

Risk model discrimination analysis
Risk model discrimination performance was analyzed using receiver operating characteristics (ROC) curves with area under the curve (AUC, c-index) comparisons using the non-parametric DeLong method [16]. All calculations were done using MedCalc v18.21 (MedCalc Software, Belgium). C-indices with 95% confidence intervals are reported [17,18].

Risk model calibration analysis
Risk model calibration/goodness-of-fit was graphically analyzed [GraphPad Prism 8 (GraphPad Software Inc.)] 1 3 comparing observed events for patient quintiles of predicted risk based on historical event probabilities from the original derivation cohorts [6,9]; it was formally tested with a logistic regression model and-where possiblethe Hosmer-Lemeshow goodness-of-fit test [21] (MedCalc v18.21 (MedCalc Software, Belgium). For the Mehran risk model for predicting contrast-induced nephropathy, assignment of score values to predicted risk were done according to extracted data from the central figure of the development cohort in the original publication [6].

Patient and procedural characteristics
A total of 2,067 patients undergoing coronary angiography between 2014 and 2018 with and without PCI at University Hospital Duesseldorf were included. Patient characteristics and NCDR risk model performance indices of a sub-cohort of these patients (n = 1637) have been previously published by our group [10]. Patients were divided into clinical setting subgroups of elective, NSTEMI and STEMI.
Mean age in the overall cohort was 69 ± 12.3 years, 27.1% of the patients had a history of chronic kidney disease, 2.9% were treated with chronic dialysis. PCI was performed in 59.5% of all patients, the rest underwent diagnostic procedures alone. Mean applied contrast volume was 154.7 ± 97.0 ml, most in STEMI and least in elective procedures. Mechanical circulatory support was used in 2.3% (Impella®) and 1.9% (Extracorporeal life support, ECLS), respectively. The predominant symptom at admission was angina pectoris. Patient and procedural characteristics for all patients and for respective subgroups are reported in Table 1.

Contrast-induced nephropathy outcomes
Contrast-induced nephropathy occurred in 15.3% (n = 317) of all patients, 1.5% (n = 31) consecutively required renal replacement therapy. While CIN was relatively common in patients presenting with acute coronary syndrome (21.3% in NSTEMI patients, 17.9% in STEMI patients), only 0.6% of all patients undergoing elective procedures developed CIN, none of whom required dialysis. In-hospital clinical outcomes are reported in Table 2.
Risk model discrimination ( Fig. 1 and Table 3) The NCDR risk models showed good discrimination of risk for CIN (c-index 0.75, 95% CI 0.72-0.78) and very accurate performance in predicting need for dialysis

Risk model calibration (Fig. 2 and Table 3)
In the graphical analysis ( Fig. 2A and C), the NCDR risk model tended to underestimate risk, especially in intermediate risk quintiles, while showing good calibration in high-risk patients. The Mehran risk model showed more accurate calibration in the graphical calibration analysis and only overestimated risk in the highest risk quintile. The Hosmer-Lemeshow goodness-of-fit test showed p < 0.01 for the NCDR-AKI as well as the Mehran risk model for predicting contrast-induced nephropathy.
Both risk models showed inaccurate calibration for the outcome of need for dialysis ( Fig. 2B and D). The Hosmer-Lemeshow goodness-of-fit test showed p < 0.01 for the NCDR-AKI-D model.   need for dialysis. Statistical comparisons were performed using the DeLong method [16], results are also reported in Table 3   Table 3 Comparative risk model performance analysis regarding model discrimination (a), reclassification (b) and calibration (c) for contrastinduced nephropathy and need for dialysis Model discrimination is reported as areas-under-curve (AUC, c-indices) of receiver-operating characteristic (ROC) analyses with 95% confidence intervals; reclassification is reported as continuous net reclassification improvement (NRI) with 95% confidence intervals; mean risk prediction is reported as mean ± standard deviation; p < 0.05 was considered statistically significant Over the past decades, interventional cardiology has undergone tremendous changes in technology [22], pharmacological therapy [23][24][25] and demographic characteristics of patients [26]. But indications for invasive coronary angiography are also changing: interventional management of chronic coronary syndromes has become an increasingly controversial topic, taking the latest results from the ISCHEMIA trial and a large meta-analysis into account [27,28]. Thus, adequate risk assessment and management is an important part of daily clinical practice to improve patient outcomes. Temporal and external validation of established risk scores is necessary to optimize risk management and retain risk model performance. Standard risk scores developed in patient cohorts almost two decades ago -like the  [6]. The initial validation cohort showed similar risk discrimination performance (c-index of 0.67 [6]) for contrast-induced nephropathy compared to our cohort (c-index of 0.69). For predicting need for dialysis, the Mehran risk model also showed a reasonable discrimination performance (c-index 0.75). Graphical analysis revealed slightly better calibration for predicting contrast-induced nephropathy compared to the NCDR-AKI risk model, potentially due to similar rates of CIN (15.3% in our cohort vs. 13.1% in the development cohort [6]). Reasons for the inaccurate calibration regarding the outcome 'need for dialysis' may potentially lie in a limited number of events in our cohort (n = 31), a higher number of patients presenting with acute myocardial infarction (75.8% vs. 35.7% in the development cohort) and an overall older patient cohort (69.2 years vs. 63.8 years in the development cohort).
The National Cardiovascular Data Registry risk models for predicting acute kidney injury and need for dialysis were developed in a patient cohort of over 900,000 patients [9]. NCDR-AKI and NCDR-AKI-D showed good discrimination performance in the initial validation cohort with c-indices of 0.71 for NCDR-AKI and of 0.89 for NCDR-AKI-D [9]. We herein observed comparable results for predicting CIN (c-index 0.75) and need for dialysis (c-index 0.85). Calibration showed an underestimation of risk in the models (Fig. 2), especially for dialysis risk prediction. On the one hand, this may be related to a difference in risk (more myocardial infarction) with higher event rates: Tsai et al. [9] observed only 7.1% acute kidney injury in the validation cohort, compared to 15.3% in our cohort. On the other hand, the CIN outcome definition of Mehran et al. [6] used here is slightly different from the one by Tsai et al. [9], which may explain an offset in calibration.
Calibration as one of the performance measures of risk models is an important aspect of individualized risk prediction since it refers to the ability of a risk model to precisely forecast the true risk of a patient and therefore may also be seen as an indicator of whether the risk model fits the test data well. In our study, we observed a graphically more accurate risk model calibration for the Mehran risk model compared to the NCDR risk model, while statistical testing rendered both models inaccurate regarding calibration. Volatile calibration performance is often observed in external risk model validation studies: both NCDR models were externally validated in a Japanese patient cohort with over 11,000 patients [29], with similarly good discrimination performance, but also offsets in calibration [29]. Previous work from our group also observed corrupted calibration [10,30]. The methodical work of Matheny et al. [31] shows that volatile calibration in risk models in interventional cardiology is related to small changes in event risk and data assessment, while retained discrimination shows the stability of risk factors over time. They conclude that continuous recalibration of risk models is mandatory to achieve perfect calibration. Additionally, slight differences in endpoint definitions between CIN according to Mehran (used here [6]), AKIN (used in the original NCDR population [9]) and the contrast-induced acute kidney injury definition (according to KDIGO [32,33]) may influence comparability of risk models regarding calibration.
Preprocedural risk assessment, as well as procedural and postprocedural risk management are critical stepping stones to improve patient outcomes. A major advantage of the NCDR risk models over the Mehran risk model is that both NCDR risk models allow calculation of risk from preprocedural characteristics-while Mehran is a postprocedural risk prediction model. Knowing the risk of CIN or dialysis before the procedure, the operator may undertake preventive measures (-periprocedural risk management). The clinical indication of the procedure can be critically reviewed and high-risk patients may benefit from more intense or prolonged postprocedural monitoring of urine output and retention parameters. Contrast media volume is a key factor for the development of CIN, especially in high-risk patients [34] and may be managed accordingly: the use of biplane angiography systems helps to reduce contrast media exposure [35]. There are technical devices which are efficient in reducing contrast media over-injection [36]. The use of intravascular ultrasound (IVUS) and physiological guidance [37] may help in decreasing contrast media volume exposure for highrisk patients. Periprocedural risk management also includes volume expansion, which is still common practice (European class IIa recommendation for preprocedural hydration with isotonic saline [14]), although several studies showed no significant difference in the rate of CIN in patients with and without prophylactic hydration [38,39]. Other studies investigated the influence on statin administration for the prevention of contrast-induced nephropathy with divergent results [40,41]. Taken together, knowing patient risk for CIN upfront may hold the key for initiating adequate preventive measures in high-risk patients. Future studies are needed to investigate potential outcome benefits from preprocedural risk assessment and periprocedural risk management.
Major limitations of our study are the retrospective single-center design, which limits generalizability to other patient cohorts, and the limited number of included patients and events. Therefore, results, especially for the outcome 'need for dialysis', have to be interpreted with caution, as they are not clearly in favor of the NCDR model. A further limitation is missing data and the simple method of data imputation, especially for the Mehran model. However, sensitivity analysis in patients with complete data verified the overall results. There were almost no missing data for the calculation of the NCDR score, which underlines its usefulness as a preprocedural risk model.
Prospective validation of both risk models in a contemporary patient cohort is necessary to verify our results and help to further improve CIN risk management.

Conclusion
In German patients undergoing coronary angiography, the modern NCDR risk model for predicting contrast-induced nephropathy showed superior discrimination performance compared to the established Mehran risk model. Both risk models showed inaccurate calibration, with slight advantages for the Mehran over the NCDR risk model for predicting contrast-induced nephropathy. Results for the outcome of 'need for dialysis' were equivocal. Further prospective studies are necessary to investigate patient outcome benefits from optimization in risk assessment and risk management.
Author contributions CP and GW conceived and designed the study. JQ, JT, SB, CP and LK collected and analyzed data. GW and CP analyzed and interpreted data and drafted the manuscript. MB, YL, VS and CJ supported data acquisition and analysis and critically revised the manuscript. MK and AI assumed project supervision, interpreted data and critically revised the manuscript. AS and AH supported data analysis and revision of the manuscript during the submission process. All authors read and accepted the submitted version of the manuscript.
Funding Open Access funding enabled and organized by Projekt DEAL. This work was supported by the Forschungskommission of the Medical Faculty of the Heinrich-Heine-University Düsseldorf for a Clinician Scientist track (No. 2018-32 to GW).

Declarations
Conflicts of interest All authors declare: no support from any industry or third-party organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval This study was positively evaluated by the Ethics committee of the Heinrich-Heine-University Düsseldorf (Study-No. 6050R, Registration-ID: 2017074349) and is registered at clinicaltrials. gov (NCT03671356). Informed consent was waived due to the retrospective design.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.