Background

Atrial fibrillation (AF) is a prevalent clinical arrhythmia, and AF and heart failure (HF) are common co-existing diseases [1]. More than one-third of newly diagnosed AF patients have HF, and more than half of newly diagnosed HF patients have AF [2]. The presence of HF and AF significantly contributed to cardiovascular morbidity and mortality in the general population, and portends worse outcomes [3]. Compared with patients with AF or HF, patients with both have a higher risk of death [4]. A previous meta-analysis data of more than 50 000 patients demonstrated that in patients with HF, AF is associated with 40% higher odds of death among patients included in randomized trials and 14% higher odds of death in patients in observational studies [5]. Early identification of HF and AF patients with high mortality risk is of great significance for the implementation of medical decision-making and the reduction of disease burden.

Previously, factors such as catheter ablation, drug treatments, and red blood cell distribution width (RDW) were reported to be associated with the risk of mortality of HF patients or AF patients [6,7,8]. Several scoring systems or models have been published for the prediction of mortality in HF patients [9,10,11]. A risk score for in-hospital mortality in patients hospitalized with HF using American Heart Association Get With the Guidelines-Heart Failure (GWTG-HF) program data was also identified and widely applied [12]. A meta-analysis revealed that the prediction effect of the existing model was mediocre, with an average C-index of about 0.66, and the included population was mainly from strictly screened randomized controlled trials or medical claim data, which had limited extension possibility to other populations [13, 14]. In addition, these prediction models mainly focus on HF patients, and few studies have constructed prediction models for the risk of mortality of HF patients with AF.

MIMIC-III database is a large open-access database comprising deidentified health-related data associated with over forty thousand patients who stayed in critical care units of the Beth Israel Deaconess Medical Center between 2001 and 2012. The database included information such as demographics, vital sign measurements made at the bedside, laboratory test results, procedures, medications, caregiver notes, imaging reports, and mortality (https://mimic.mit.edu/docs/iii/) [15]. MIMIC-IV database, constructed based on MIMIC-III, and incorporated numerous improvements over MIMIC-III (https://mimic.mit.edu/docs/iv/) [16]. MIMIC-IV contains over 70,000 ICU admissions across the United States collected from 2008 to 2019 including comprehensive patient information. The eICU Collaborative Research Database (eICU-CRD) is a multicenter database including more than 200,000 ICU admissions in the United States [17].

In view of the co-morbidification burden of HF and AF, this study intended to develop a prediction model for in-hospital mortality of HF patients with AF based on the data from the Medical Information Mart for Intensive Care (MIMIC)-III and MIMIC-IV database. And the data from eICU-CRD were used as an external validation set. The prediction performance of the prediction model was evaluated and compared with GWTG-HF risk score.

Methods

Study design and population

In total, this cohort study extracted the data of 13,183 patients diagnosed as HF with AF upon intensive care unit (ICU) admission who aged ≥ 18 years old from the MIMIC-III (n = 4679), MIMIC-IV database (n = 7097) and eICU-CRD (n = 1407). HF and AF were diagnosed based on the International Classification of Disease (IC) codes. HF were diagnosed according to ICD-9 (42,821, 42,822, 42,823, 42,831, 42,832, 42,833, 42,841, 42,842, and 42,843), and ICD-10 (I5021, I5022, I5023, I5031, I5032, I5033, I5041, I5042, I5043, I50811, I50812, and I50813). AF was diagnosed according to ICD-9 (42,731), and ICD-10 (I480, I481, I482, and I4891). MIMIC-III (Medical Information Mart for Intensive Care III) is a large, freely-available database comprising deidentified health-related data associated with over forty thousand patients who stayed in critical care units of the Beth Israel Deaconess Medical Center between 2001 and 2012, including information such as demographics, vital sign measurements made at the bedside (~ 1 data point per hour), laboratory test results, procedures, medications, caregiver notes, imaging reports, and mortality (https://mimic.mit.edu/docs/iii/) [15]. MIMIC-IV constructed based on MIMIC-III, and incorporated numerous improvements over MIMIC-III (https://mimic.mit.edu/docs/iv/) [16]. The eICU-CRD is publicly available database comprising de-identified health data associated with more than 200,000 admissions to ICUs across the United States between 2014 and 2015 (https://eICU-crd.mit.edu/about/eICU/) [18]. In our study, those without survival information and hospitalized in the ICU < 24 h were excluded. Finally, 11,455 patients were included [MIMIC-III (n = 4238), MIMIC-IV (n = 5998), and eICU-CRD (n = 1219)]. The requirement of ethical approval for this was waived by the Institutional Review Board of Putuo People’s Hospital affiliated to Tongji University, because the data was accessed from MIMIC (a publicly available database). The need for written informed consent was waived by the Institutional Review Board of Putuo People’s Hospital affiliated to Tongji University due to retrospective nature of the study. All methods were performed in accordance with the relevant guidelines and regulations.

Potential predictors

Demographic data including age (years), weight (kg), gender, race (Black, White, other or unknown), insurance (Medicare or other), and marital status (divorced, married, single, or widowed), laboratory data including heart rate (bpm), systolic blood pressure (mmHg), diastolic (mmHg), respiratory rate (bpm), temperature (℃), oxygen saturation (SpO2) (%), Charlson comorbidity index, the simplified acute physiology score (SAPS)-II, Glasgow coma scale (GCS), white blood cells (WBC) (K/uL), platelets (K/uL), hemoglobin (g/dL), RDW (%), creatinine (mg/dL), international normalized ratio (INR), prothrombin time (PT) (sec), partial thromboplastin time (PTT) (sec), blood urea nitrogen (BUN) (mg/dL), Glucose (mg/dL), anion gap (mEq/L), urine output (mL), and sodium (mEq/L), treatments during 24-h ICU admission including ventilation (no or yes), vasopressor (no or yes), renal replacement therapies (RRT) (no or yes), coronary artery bypass grafting (CABG) (no or yes), catheter (no or yes), antiarrhythmic (no or yes), antiplatelet (no or yes), anticoagulation (no or yes), beta-blocker (no or yes), and diuretic (no or yes), and other data including chronic obstructive pulmonary disease (COPD) (no or yes) first care unit [coronary care unit (CCU), cardiac vascular ICU (CVICU), medical ICU (MICU), surgical ICU (SICU) or other] were potential predictors analyzed in this study. All the data were collected within 24 h on admission to ICU, and the first measurement on ICU admission was applied for the prediction model construction.

Outcome variable

The outcome in this study was the mortality 24 h into the ICU visit until the hospital discharge, which was defined as mortality status from 24-h admission to the ICU to hospital discharge. The beginning of follow-up was considered 24 h of the patient’s ICU admission. The date of death was obtained from the US government’s Social Security Death Index records and should not exceed the discharge date from the hospital. The mean survival time of participants from MIMIC-III was 11.29 ± 10.05 days and the mean survival time of participants from MIMIC-IV was 10.56 ± 9.19 days.

Statistical analysis

Mean ± standard deviation (SD) was used to describe the measurement data subject to normal distribution, and t-test was used to compare the difference between the two groups. Medians and quartiles [M (Q1, Q3)] were employed to display the measurement data with abnormal distribution. Wilcoxon rank sum test was used to compare the difference between the two groups. Enumeration data were expressed as number of cases and percentages [n (%)], and differences between groups were compared using Chi-square test or Fisher’s exact probability method. The subjects from MIMIC-IV were divided into the training set to construct the prediction model, and the testing set to verify the performance of the model. The samples from MIMIC-III database were included as the internal validation set and the samples from eICU-CRD were included as the external validation set to validate the predictive value of the model. Univariate and multivariable Logistic regression analyses were used to explore predictors for in-hospital death in patients with HF and AF. The odd ratios (OR) and 95% confidence interval (95%CI) were applied as effect size. The receiver operator characteristic (ROC), calibration curves and the decision curve analysis (DCA) curves were plotted to evaluate the predictive values of the model. The area under the curve (AUC), sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV) and accuracy of the models for predicting the risk of in-hospital death in patients with HF and AF were measured. The confidence level alpha = 0.05. Data analysis, ROC curve plotting, difference comparison, construction of the prediction model, and Delong test were completed by SAS 9.4 (SAS Institute Inc., Cary, NC, USA). Visualization of nomogram and DCA curves were done by R version 4.2.1 (2022–06-23 ucrt). P < 0.05 was considered statistically significance.

Results

Comparisons between the characteristics of subjects in the survival group and death group

In our study, 13,183 patients with HF and AF who aged ≥ 18 years old from the MIMIC-III (n = 4679), MIMIC-IV database (n = 7097) and eICU-CRD (n = 1407) were included. There were 12 people lost survival information in eICU-CRD. In total, 463 subjects from MIMIC-III database, 1099 participants from MIMIC-IV database and 176 patients from eICU-CRD who hospitalized in the ICU < 24 h were excluded. Finally, 11,455 patients were included with 4238 from MIMIC-III, 5998 from MIMIC-IV and 1219 from eICU-CRD. The screen process was presented in Fig. 1.

Fig. 1
figure 1

The screen process of participants

Full size image

In the samples from MIMIC-III database, there were 680 (16.05%) participants died at the end of the follow-up. The percentages of participants receiving ventilation (51.85% vs 68.82%), vasopressor (41.09% vs 58.68%), RRT (6.46% vs 17.79%), antiarrhythmic (17.79% vs 6.49%), and beta-blocker (8.53% vs 3.34%) in the survival group were lower than the death group. The mean age of the survival group was higher than the death group (78.00 years vs 75.77 years). In the samples from MIMIC-IV database, 5094 (84.93%) subjects were survived at the end of the follow-up. The percentages of patients receiving vasopressor (67.26% vs 46.54%), RRT (20.91% vs 8.11%), antiarrhythmic (26.00% vs 17.37%), and beta-blocker (21.90% vs 7.99%) in the survival group were lower than the death group. Participants in the survival group had lower age than the death group (76.73 years vs 74.05 years). The detailed information of participants was presented in Table 1. In the samples from eICU-CRD, 996 participants were survived and 233 were dead. The percentages of people received ventilation in the survival group was lower than the death group (27.51% vs 46.19%). The mean age of the survival group was lower than the death group (73.50 years vs 75.93 years) (Supplementary Table 1).

Table 1 The characteristics of patients from MIMIC-III and MIMIC-IV in the survival group and death group
Full size table

Construction of the prediction model for in-hospital death in patients with HF and AF

All the samples from MIMIC-IV database were randomly divided into the training set and the testing set at a ratio of 7:3. The baseline data of the participants in the training set and testing set were shown in Table 2. The results of univariate logistical regression model revealed that ventilation, vasopressors, first care unit, race, insurance, RRT, antiarrhythmic, antiplatelet, anticoagulation, beta-blocker, CABG, age, heart rate, systolic blood pressure, respiratory rate, temperature, SpO2, Charlson comorbidity index, SAPSII, WBC, platelet, RDW, creatinine, INR, PT, PTT, BUN, glucose, anion gap, urine output and COPD might be predictors for in-hospital death in patients with HF and AF. The final formula of prediction model was shown as follows:.

Table 2 The characteristics of patients in the survival group and death group of the training set and testing set
Full size table
$$\begin{array}{c}1\mathrm n\left(\frac{\mathrm p}{1-\mathrm p}\right)=-5.5310+0.0474\times\mathrm{SAPSII}+0.1121\times\mathrm{RDW}-0.4192\times\mathrm{Beta}-\mathrm{blocker}\;\left(\mathrm{no}\right)-0.4175\times\mathrm{Race}\;(\mathrm{Black})\\-0.1878\times\mathrm{Race}\;\left(\mathrm{others}\right)+0.6578\times\mathrm{Race}\;\left(\mathrm{unknown}\right)+0.0308\times\mathrm{Respiratoryrate}\\\begin{array}{c}-0.00022\times\mathrm{Urineoutput}+0.7242\times\mathrm{CABG}\;\left(\mathrm{no}\right)+0.0967\times\mathrm{Charlsoncomorbidityindex}\\\begin{array}{c}-0.0239\times\mathrm{Spo}2+0.00333\times\mathrm{PPT}-0.1961\times\mathrm{Antiarrhytmic}\;\left(\mathrm{no}\right)+0.0133\times\mathrm{Age}\\\begin{array}{c}-0.3853\times\mathrm{RRT}\;\left(\mathrm{no}\right)-0.2248\times\mathrm{Creatinine}+0.0538\times\mathrm{Anion gap}\\-0.1747\times\mathrm{Anticoagulation}\end{array}\end{array}\end{array}\end{array}$$

The AUC, specificity, NPV and accuracy of our prediction model in the training set were 0.810 (95%CI: 0.791–0.828), 0.755 (95%CI: 0.740–0.769), 0.940 (95%CI: 0.931–0.948), and 0.750 (95%CI: 0.736–0.763). The AUC, specificity, and NPV of our prediction model in the testing set were 0.757 (95%CI: 0.729–0.786), 0.760 (95%CI: 0.738–0.782), and 0.906 (95%CI: 0.889–0.922). The AUC of our prediction model in the internal validation set was 0.792 (95%CI: 0.774–0.810) and 0.724 (95%CI: 0.687–0.762) in the external validation set (Table 3). The ROC curves of our prediction model and the previous risk score in the training set, testing set, internal validation set, and external validation set were exhibited in Figs. 2, 3, 4 and 5, respectively. The calibration curves of the model in the training set (Supplementary Fig. 1), testing set (Supplementary Fig. 2), internal validation set (Supplementary Fig. 3), and external validation set (Supplementary Fig. 4) revealed that the predictive probabilities of our model for the in-hospital death in patients with HF and AF deviated slightly from the ideal model. The DCA curves revealed that the use of our prediction model increased the net benefit than use no model, suggesting that the model might help the clinicians quickly identify those at high risk of in-hospital mortality (Supplementary Figs. 5, 6, 7 and 8). The nomogram of the prediction model was plotted (Fig. 6). Delong test depicted that the AUCs of our model in the training set, testing set, and internal validation set were higher than the previous risk score (Table 4).

Table 3 The predictive values of the models
Full size table
Fig. 2
figure 2

The ROC curve of our prediction model and the previous risk score in the training set

Full size image
Fig. 3
figure 3

The ROC curve of our prediction model and the previous risk score in the testing set

Full size image
Fig. 4
figure 4

The ROC curve of our prediction model and the previous risk score in the internal validation set

Full size image
Fig. 5
figure 5

The ROC curve of our prediction model and the previous risk score in the external validation set

Full size image
Fig. 6
figure 6

The nomogram of our prediction model

Full size image
Table 4 The results of Delong test comparing the predictive value of our model with GWTG-HF risk score
Full size table

Discussion

In the current study, a prediction model for in-hospital mortality of HF patients with AF was established based on the predictors including race, RRT, antiarrhythmic, anticoagulation, beta-blocker, CABG, age, respiratory rate, SpO2, Charlson comorbidity index, SAPSII, RDW, creatinine, PTT, anion gap, and urine output. The prediction model showed good predictive performance with AUC of 0.810 in the training set, 0.757 in the testing set, 0.792 in the internal validation set and 0.724 in the external validation set. The prediction model might provide a useful tool to early identify patients complicated with HF and AF who were at high risk of in-hospital death, and offer timely interventions to improve their prognosis.

At present, several models were established for predicting the mortality of HF patients. Li et al. established a prediction model for in-hospital mortality in ICU patients with HF using machine learning methods, showing good predictive performance [19]. Another multivariable prediction model for the mortality of patients with HF had a C-index of 0.70 [20]. The GWTG-HF risk score is constructed to predict the risk of in-hospital mortality for patients hospitalized with HF based on information concerning patient age, SBP, BUN, HR, serum sodium, COPD and non-African American ethnicity [21]. The GWTG-HF risk score is widely applied to be a prognostic tool for evaluating the mortality of HF patients [22, 23]. The GWTG-HF risk score for participants in this study was also calculated, and the discrimination performance were validated in the samples of our study. The predictive performance of the model in the current study was superior to the GWTG-HF score. Several other risk scores including the Intermountain Risk Score (IMRS) and the Naples score (NS) were established for other heart diseases such as cardiogenic shock or myocardial infarction [24, 25]. Compared with previous models and risk score, our model could quickly identify patients with both HF and AF who were at high risk of in-hospital mortality. We also verify the predictive performance of the model in the internal validation set using the data from MIMIC-III database and external validation set using the data from eICU-CRD. The model had good discrimination ability for HF and AF patients with high risk of in-hospital death. This model combined fast and routinely available variables including demographic characteristic and laboratory characteristics, which seemed to be a promising tool for early and accurate risk stratification in the ICUs. The nomogram was also plotted, and the probability of in-hospital mortality of patients with HF and AF could quickly be obtained. For clinicians, special interventions and care should be applied to those with high risk of mortality in patients with HF and AF.

A previous nomogram revealed that age, AG ≥ 20 mEq/L, RDW ≥ 15.5%, and beta‐blocker were important predictors for the in‐hospital mortality of patients with congestive HF and chronic kidney disease [26]. Wussler et al. conducted a review exploring recent advances and remaining uncertainties regarding risk stratification in acute HF, which identified that age, respiratory rate, oxygen saturation, and creatinine were the most commonly used predictor variables in the described risk scores [27]. There was evidence indicated that antiarrhythmic [28, 29], beta-blockers [30, 31], and anticoagulation [32] were correlated with the prognosis of patients with HF or patients with HF and AF. Charlson comorbidity index, the most extensively studied comorbidity index, was found to be associated with the clinical outcome in patients with HF [33] and an important predictor for 30-day readmission in patients with HF exacerbation and AF [34]. Age and creatinine were also demonstrated to have predictive value for in-hospital mortality in patients with cardiogenic shock [25]. These findings supported the results in the present study, which showed that race, RRT, antiarrhythmic, anticoagulation, beta-blocker, age, respiratory rate, SpO2, Charlson comorbidity index, RDW, creatinine, anion gap, and urine output were essential predictors for in-hospital death in patients with HF and AF.

Several limitations were found in this study. Firstly, due to the limitation of the MIMIC database, the data on left ventricular ejection fraction, and lactate level were not reported, which might affect the results. Secondly, electrocardiograms scores have been applied for the prediction of diastolic dysfunction and other diseases in previous studies [35, 36], but the data on electrocardiograms could not been obtained from MIMIC database. Thirdly, patients with missing death information were excluded, which might impact the generalizability of the model. In the future, more studies are needed to verify the findings of our study.

Conclusions

The present study established a prediction model for in-hospital death mortality of patients with HF complicated with AF. The prediction model had good discriminative ability, and might provide a tool to quickly identify patients with HF complicated with AF who were at high risk of in-hospital mortality.