Background

In-hospital cardiac arrest (IHCA) is an acute disease with a high fatality rate that burdens individuals, society, and the economy [1, 2]. There are approximately 290,000 cases of IHCA in the United States annually, with only 25% of such cases surviving and being discharged from the hospital [3]. A study [4] of IHCA in China observed that the incidence of IHCA was 17.5 per 1000 admissions, and the rates of return of spontaneous circulation and survival to hospital discharge were 35.5% and 9.1%, respectively. Thus, the current situation of IHCA in China is still concerning. Although technologies, such as mild hypothermia and extracorporeal membrane oxygenation, are increasingly used in cardiac arrest therapy [5, 6], patient prognosis remains poor. Therefore, identifying patients with high risk of IHCA is crucial for early intervention.

Machine learning has been demonstrated as a powerful tool that could detect unnoticed data trends and patterns in the use of conventional statistical models [7]. Recently, machine learning methods are increasingly applied to predict IHCA in hospitalized patients, emergency department patients, and intensive care unit patients [8,9,10,11]. However, to the best of our knowledge, a few studies explored approaches for predicting IHCA in patients treated with rescue therapy. Compared with previous study patients, patients treated with rescue therapy are in worse condition and need more efficient risk assessment. Furthermore, laboratory parameter are subject to strict quality control and have been identified as independent risk factors of poor patient outcomes [12, 13]. However, previous studies mainly focused on the performance of the model, with limited emphasis on the predictive factors [14,15,16]. Moreover, most previous studies only included blood cell counts and biochemical markers but overlooked the predictive value of coagulation markers [9, 11, 17, 18]. A study conducted by Deng et al. reported that D-dimer was associated with immediate mortality in patients with IHCA, while other markers related to coagulation were not analyzed [19]. Meanwhile, a number of former studies included subjective and unstructured variables in prediction modeling, which need manual discrimination or data conversion that might not be applicable to a rescue setting [8, 11, 20,21,22,23]. Laboratory results are objective and readily available; however, no previous studies have used machine learning to predict cardiac arrest solely based on routine laboratory parameters.

Therefore, we aimed to develop an appreciable model solely using routine laboratory data obtained from hospital information system (including blood counts, biochemical markers, and coagulation markers) to predict incident IHCA in patients requiring rescue therapy.

Results

Population characteristics

A total of 11,308 hospitalized patients receiving rescue therapy were included in this study, and 3529 patients were excluded according to the exclusion criteria. Among 7779 patients included in the present analysis, 1796 patients experienced IHCA (positive samples) and 5983 patients did not experience IHCA (negative samples) (Fig. 1). The incidence rate of IHCA did not differ by sex (63.73% vs 64.70%, P = 0.453). Patients with IHCA were older (80 years old vs 63 years old, P < 0.001) and had a higher proportion of comorbidities, such as hypertension (48.55% vs 38.84%, P < 0.001) and diabetes (25.84% vs 21.16%, P < 0.001), compared with those without IHCA (Table 1).

Fig. 1
figure 1

The screening phase flowchart. OHCA out-of-hospital cardiac arrest, IHCA in-hospital cardiac arrest

Full size image
Table 1 Baseline characteristics of the non-IHCA cohort and the IHCA cohort
Full size table

After undersampling, 1796 patients with IHCA and 1796 patients without IHCA were included in the model development. Basic characteristics are shown in Additional file 2: Table S1. After randomly grouping at a ratio of 8:2, the training set included 2873 patients and the testing set included 719 patients. The basic characteristics of the training set and testing set are listed in Table 2. There was no statistically significant difference between training set and testing set.

Table 2 Baseline characteristics of the undersampled training cohort and the testing cohort
Full size table

Model development and validation

We used a heatmap to present the correlation coefficient between all variables (Fig. 2), and the results showed that hemoglobin and red blood cell, direct bilirubin, and total bilirubin had high correlation coefficient (> 0.8), respectively. The variable importance ranked by extra trees classifier (ETC) algorithm showed that red blood cell and total bilirubin had lower variable importance compared with their counterparts. Therefore, we discarded red blood cell and total bilirubin in the model development.

Fig. 2
figure 2

Correlation between variables. WBC white blood cell, PT prothrombin time, PTA prothrombin activity, TT thrombin time, LDH lumbar disc herniation, ALT alanine aminotransferase, NT-proBNP N-terminal pro-BNP, RBC red blood cells, APTT activated partial thromboplastin time, TnT troponin T, AST aspartate aminotransferase, CRP C-reactive protein

Full size image

ETC, logistic regression, random forest, support vector machine (SVM), and decision tree algorithms were used to develop the prediction models of IHCA in patients treated with rescue therapy. In the training group, we observed that ETC algorithm showed better performance compared with the other four algorithms (area under curve [AUC], 0.939 in ETC vs. 0.896 in logistic regression, 0.938 in random forest, 0.829 in SVM, and 0.871 in decision tree; P for comparison < 0.01) (Table 3 and Fig. 3). In the testing group, ETC algorithm also showed the best performance among five algorithms (AUC, 0.920 in ETC vs. 0.895 in logistic regression, 0.877 in random forest, 0.864 in SVM, and 0.843 in decision tree; P for comparison < 0.01) (Table 3 and Fig. 3). After fivefold cross-validation, the ETC algorithm generally provided the best overall performance regarding the AUC, accuracy, specificity, sensitivity, and F1-score compared with the other four algorithms (Table 3 and Fig. 4), indicating the consistency and robustness of our model. After considering these scores, especially the AUCs, we chose ETC model as the final prediction model. The hyperparameters for the ETC model as selected are summarized as follows: number of trees (n_estimators) = 100, supported criteria (criterion) = gini, maximum tree depth (max_depth) = 12, minimum number of samples leaf (min_samples_leaf) = 1, and minimum number of samples split (min_samples_split) = 2.

Table 3 Summary of model performance of five machine learning algorithms
Full size table
Fig. 3
figure 3

The ROC curve of different machine learning algorithms predicting IHCA in the training group and testing group. ROC receiver-operating characteristic, SVM support vector machine

Full size image
Fig. 4
figure 4

The ROC curve of different machine learning algorithms predicting the IHCA in the fivefold cross-validation. ROC receiver-operating characteristic, SVM support vector machine

Full size image

Model interpretation

As shown in Fig. 5a, the mean absolute SHapley Additive exPlanation (SHAP) value indicates individual feature importance in the ETC model, and the top ten variables were prothrombin activity (PTA), platelets (PLT), hemoglobin, N-terminal pro-BNP (NT-proBNP), neutrophils, prothrombin time (PT), serum albumin, sodium, activated partial thromboplastin time (APTT), and potassium. According to the summary plot (Fig. 5b), PTA, PLT, hemoglobin, and serum albumin were negatively correlated with IHCA occurrence. For example, a low PTA increases the importance of IHCA prediction, whereas a high PTA reduces the importance of IHCA prediction. In contrast, NT-proBNP, neutrophils, PT, sodium, APTT, and potassium were positively correlated with IHCA occurrence. The dependence plots of the SHAP value of top ten important variables are shown in Additional file 1: Fig. S1. The difference of abnormalities of top ten important variables between patients with and without IHCA is shown in Additional file 2: Table S2 [24,25,26,27,28,29,30]

Fig. 5
figure 5

SHapley Additive exPlanations (SHAP) results. a SHAP feature importance; b SHAP summary plot of the top 20 variables. PTA prothrombin activity, NT-proBNP N-terminal pro-BNP, PT prothrombin time, APTT activated partial thromboplastin time, CRP C-reactive protein, WBC white blood cell, LDH lumbar disc herniation, AST aspartate aminotransferase, TT thrombin time, ALT alanine aminotransferase, TnT troponin T

Full size image

Discussion

In this study, we developed a machine learning-derived prediction model of IHCA in patients treated with rescue therapy. After fivefold cross-validation, the prediction model of IHCA based on the ETC algorithm showed the best performance among five algorithms used for model development. The SHAP interpreted the output of machine learning model and estimated the positive and negative contributions of each feature to the model prediction. The top ten important variables included PTA, PLT, hemoglobin, NT-proBNP, neutrophils, PT, serum albumin, sodium, APTT, and potassium, which are important predictors for IHCA in rescue-treated patients and provide valuable information for early intervention in rescue-treated patients to prevent IHCA.

ETC algorithm is an ensemble classifier that uses unpruned decision trees from the training datasets to construct an extremely randomized or extra tree classifier [31]. In this study, the ETC model showed excellent performance and better predictive power than the other four machine learning models. Notably, this model could predict the occurrence of IHCA in 24 h after testing laboratory parameters, and all variables were easy to obtain and under strict laboratory quality control. Therefore, this model is suitable for clinical practice in large tertiary hospitals.

Coagulopathy often occurs after resuscitation from cardiac arrest or during cardiac arrest [32]. Previous studies [33] have suggested that consumptive coagulopathy played a central role in the pathogenesis of cardiac arrest and the activation of the coagulation system was persistent during cardiac arrest. Among rescue-treated patients, sepsis-related coagulation dysfunction is one of the most common causes of death [34,35,36]. Because of the excessive production of plasminogen activator inhibitor-1, sepsis-related disseminated intravascular coagulation (DIC) causes excessive inhibition of fibrinolysis and may result in related prothrombotic effects, leading to reduced tissue perfusion, organ dysfunction, and poor outcomes [37, 38]. PT, APTT, and PLT are generally used to evaluate blood coagulation function, and have a high predictive value for DIC prediction [39]. Moreover, previous studies [40, 41] have indicated that PT prolongation and PLT decline are associated with increased mortality in patients with sepsis. Consistently, our study showed that coagulation markers are important predictors for IHCA. PT and APTT were positively correlated with the occurrence of IHCA, whereas PTA and PLT were negatively correlated with the occurrence of IHCA. 16.09% of patients with IHCA had abnormally prolonged PT by more than 3 s, and 42.20% of patients with IHCA had a PLT < 100 × 109/L. However, clinicians always ignored coagulation disorders at an early stage, because these coagulation parameters were easily affected by multiple diseases and the fluctuation range is large [39, 42]. The findings of our study indicated that more attention should be paid to the progressive deterioration of coagulation parameters. It is necessary to monitor coagulation parameters in patients treated with rescue therapy and intervene coagulation disorders as early as possible.

As a commonly used infection index in clinics, neutrophils have been proven as biomarkers of sepsis [43, 44], which explains the finding that higher neutrophils was positively correlated with the occurrence of IHCA. Consistently, patients with cardiac arrest were frequently accompanied with infection, as shown in Additional file 1: Fig. S1. Albumin is the one of the most important components in human plasma, which can reflect the nutritional status and maintain osmotic pressure. Hofer et al. [45] found that the plasma albumin level in septic patients was significantly lower than that in non-septic patients. Our study found that serum albumin was negatively correlated with incident IHCA, which supports the effort of early management on nutritional status to prevent the occurrence of IHCA.

Previous studies [3] suggested that cardiovascular problems are the most common cause of cardiac arrest (50–60%). Higher brain natriuretic peptide is significantly associated with more severe cardiac injury and poorer prognosis. Pfister et al. [46]showed that NT‐proBNP was associated with both death and cardiovascular composite outcomes in a clinical population of patients with cardiovascular disease or chronic kidney disease. Similar to these findings, the SHAP results in our study showed that NT-proBNP is a strong predictor of IHCA and those with high NT-proBNP were more likely to suffer IHCA. In addition, 64.25% of patients with IHCA had an NT-proBNP level > 450 pg/mL, as shown in Additional file 2: Table S2.

Electrolyte disorder is one of the potential cause of cardiac arrest, among which potassium disorder is closely related to life-threatening arrhythmia [47]. Potassium is the main intracellular cation in the human body, which regulates the cardiac function, bones’ metabolism, and acid–base balance. Abnormality in potassium concentration can lead to serious complications. Patients with hyperkalemia are in extremely high risk of life-threatening cardiac arrest. Meanwhile, our study found that the level of sodium is positively correlated with the occurrence of IHCA in rescue-treated patients. A former study suggested that hypernatremia (> 145 mmol/L) can increase the risk of mortality [25]. Coppini et al. [48] found that the change in intracellular calcium homeostasis and the increase of late sodium current promoted arrhythmia. Additionally, Wu et al. [11] found that low level of hemoglobin is related to incident cardiac arrest. Taken together, these findings explain the importance of electrolyte and hemoglobin in the prediction of IHCA and suggest that the blood concentration of electrolyte and hemoglobin should be monitored frequently in rescue-treated patients to better prevent the occurrence of IHCA.

The major strength of our study is that we developed a reliable machine learning model to predict IHCA in rescue-treated patients solely using readily available laboratory parameters, which minimums the potential subjective bias that is common in self-reported data and medical texts, and enables our model to be more robust and applicable. This model promotes accurate prediction for IHCA in rescue-treated patients, which may further reduce the burden in frontline healthcare and improve the rescue success rate. However, several limitations should be considered. First, the major limitation of our study is that we only included patients from a single hospital, lacking external validation, and due to issues like missing data, we were unable to compare our results with the existing models. This may limit the generalizability of our model to other hospitals or regions. However, we only used objective laboratory test data in model development. The homogeneous nature of laboratory test data could help to reduce potential confounding due to region and health care disparities. Second, our study was based on a retrospective design, which may cause selection bias. Future prospective study and external validation are still warranted to further validate the model of our study.

Conclusion

We developed an appreciable ETC model to predict IHCA in rescue-treated patients solely using routine laboratory parameters. The model showed that the major risk factors for IHCA in rescue-treated patients were PTA, PLT, hemoglobin, NT-proBNP, neutrophils, PT, serum albumin, sodium, APTT, and potassium. During the hospitalization of rescue-treated patients, physicians should attach great importance to frequently monitoring these parameters to prevent the occurrence of IHCA as possible.

Methods

Study population

This study was reviewed and approved by the ethics committee of the First Medical Center of Chinese PLA General Hospital (Ethics approval number: S2023-282-01). Rescue-treated patients were retrospectively identified from the hospital information system at the First Medical Center of the PLA General Hospital in Beijing, China, from January 2016 to December 2020. The inclusion criteria [11, 49, 50] were: (1) age ≥ 18 years and (2) length of hospital stay > 24 h; the exclusion criteria [11, 49, 50] were: (1) patients who had IHCA before rescue treatment, (2) patients who had history of out-of-hospital cardiac arrest (OHCA), (3) patients who had IHCA during surgery, and (4) patients with missing data on laboratory test.

Prediction outcome

The primary outcome measure was the incidence of IHCA. This study defined IHCA as hospitalized patients whose pulse disappeared and required chest compression or defibrillation because of electrical defibrillation/cardioversion events. Only the first cardiac arrest that occurred in the hospital was analyzed [3, 49, 51].

Candidate features

We abstracted data of the rescue-treated patients from the hospital information system. The following variables were collected: (1) demographic data: sex, age, body mass index (BMI), drinking, and smoking history; (2) basic diseases: hypertension and diabetes; (3) laboratory results: blood counts, biochemical markers, and coagulation markers, and C-reactive protein (CRP). For patients who experienced IHCA (positive samples), we collected laboratory data that were tested within the 24 h before incident IHCA. For patients who did not experienced IHCA (negative samples), we collected laboratory data that were tested within the 24 h after admission.

Data processing

Candidate variables with missing values greater than 40% were excluded [52]. We calculated correlation coefficient between all variables and identified those pairs with high correlation coefficient (> 0.8). In the pair of variables with high correlation coefficient, the one with lower variable importance would be excluded. We discarded outliers of each variable, which were defined as values whose difference with mean was greater than threefold standard deviation. Missing values were handled using means substitution method [53, 54]. Finally, given that the negative samples are several times of positive samples in our study, we conducted undersampling using k-means clustering algorithm to balance the imbalanced data sets. The k-means algorithm recognizes each negative sample as an eigenvector and divides all negative samples into n eigenvector datasets with similar features, where n is the same with the number of positive samples. For each dataset, the k-means algorithm selects 1 eigenvector, which is the closest to the mean of the eigenvectors in the dataset, as a negative sample, and ultimately forms a dataset with n negative samples to create sample balance [55].

Model development and validation

The dataset used for model development was randomly partitioned into two samples: 80% of the participants for model training and parameter learning and 20% of the participants for model performance evaluation and comparison. We used the ETC, logistic regression, random forest, SVM, and decision tree algorithms to develop a model for early prediction of IHCA. The performance metrics of the five models were evaluated by calculating the AUC of the receiver-operating characteristic curve, accuracy, specificity, sensitivity, and F1-score. Furthermore, we performed fivefold cross-validation to test the consistency and robustness of the model. To interpret the final predictive model, we used SHAP to explain the output of the model by evaluating the variable importance and the impact direction of variables [56]. Model development and validation were conducted using Python Version 3.8 (Python Software Foundation, Wilmington, DE, USA).

Statistical analysis

Continuous variables with a normal distribution were presented as the mean ± standard deviation, and t tests were used for comparison between groups. Furthermore, non-normally distributed continuous variables were presented as median (interquartile range). Moreover, the Mann–Whitney U test was used for comparisons between groups. Categorical variables were presented as percentages (%), and the Chi-square test was used for comparisons between groups. To enhance the interpretability of our model, we provided the difference of abnormalities of parameters with high feature importance between patients with and without IHCA. All statistical analyses were conducted using SPSS statistical software (version 26.0; IBM Corp., Armonk, NY, USA). P value < 0.05 was considered statistically significant.