Introduction

In-hospital cardiac arrest (IHCA) is a devastating complication among critically ill children admitted to the cardiac intensive care unit (CICU). Less than half who suffer from an IHCA will survive [1]. Survivors are often left with important morbidity, including severe neurologic impairment [2]. Neonates suffer a disproportionate number of the total IHCA in the CICU, with almost four times the odds of IHCA [3]. Implementation of processes to prevent IHCA in neonatal cardiac surgical patients represents a tangible opportunity to reduce patient harm and improve outcomes, as recent evidence suggests some IHCAs in this cohort are avoidable [4, 5].

The multicenter Pediatric Cardiac Critical Care Consortium (PC4) cardiac arrest prevention (CAP) project demonstrated that targeting high-risk CICU cohorts, including postoperative neonates, with a simple clinical practice bundle can decrease the IHCA rate [4]. However, the evidenced-based criteria used to identify the high-risk cohorts in the CAP project [3] lacked precision for predicting IHCA; the CAP bundle needed to be initiated in many children to prevent one IHCA. Similar to past initiatives, CAP project high-risk cohorts are largely limited to patient demographics and treatment characteristics [6, 7]. The inclusion of physiologic and laboratory data into models may be an opportunity to improve the precision of IHCA prediction in high-risk cohorts [8,9,10].

In this context, we performed a pilot study to determine if we could identify a cohort at even higher risk for IHCA within the already high-risk cohort of neonatal cardiopulmonary bypass (CPB). Specifically, we sought to determine the association between postoperative admission physiologic and laboratory data and the risk of early IHCA after neonatal cardiac surgery. In addition, we aimed to derive a clinically useful model that would identify the highest-risk cohort for early IHCA, such that future IHCA prevention initiatives could be more focused.

Methods

Patients

The University of Alabama at Birmingham Institutional Review Board approved this study (IRB-300001204) on March 29, 2018. The need for informed consent was waived. The data for this manuscript represents a retrospective, secondary analysis of a data set collected for a previous study evaluating the correlation of inferior and superior vena cava oxygen saturations (SVC O2) [11]. Original inclusion criteria were consecutive neonates with concurrent inferior vena cava (IVC) and superior vena cava (SVC) catheters who underwent cardiac surgery with cardiopulmonary bypass (CPB) between February 2008 and January 2014 at the University of Alabama at Birmingham and Children’s of Alabama. None of the 118 original patients were excluded. As there was a clinical practice pattern for increased monitoring in higher acuity cardiac operations, our cohort encompassed more single ventricle operations (most often the Norwood operation) than the typical neonatal CPB cohort. To increase applicability to most North American neonatal cardiac surgery programs, only venous blood measurements from the SVC catheter were used in the current study. IHCA details were obtained from CICU IHCA quality improvement clinical database; the etiology of IHCA was determined by a unit-based IHCA review team.

Perioperative management

All patients were admitted to the CICU directly from the operating room (OR). All patients who underwent CPB were mechanically ventilated, sedated, and under neuromuscular blockade at the time of admission. Adjustments of vasoactive and inotropic medications were made under the direction of the CICU physicians and advanced practitioners who used ongoing hemodynamic assessment to inform their decision-making. Standard baseline inotropes of epinephrine, vasopressin, and/or milrinone (0.5 mcg/kg/min) were initiated prior to leaving the OR. Intravenous fluid boluses and packed red blood cells were administered for hypotension and/or bleeding at the discretion of the CICU team. Local practice included goal hemoglobin of 9–11 mg/dL for two ventricle patients and 13–15 mg/dL for single ventricle patients. Laboratory monitoring was obtained every 2 h in the immediate postoperative period for most patients.

Data collection and definitions

We collected data at two time points: admission to the CICU from the OR and at 2 h after admission to the CICU. Admission data was as follows: vital signs, SVC O2 and arterial oxygen saturation (SaO2) co-oximetry (Radiometer America ABL 90 INC., Brea, CA), forehead and flank near-infrared spectroscopy (cNIRS and rNIRS, respectively) (INVOS, Medtronic, Minneapolis, MN). All oximetry values were measured and not calculated. Laboratory data were available on all 118 patients. Data collected at 2 h post-admission was limited to arterial serum lactate and modified vasoactive-inotropic score (VIS) [12]. Arteriovenous oxygen difference (AVO2) was calculated by subtracting SVC O2 from admission SaO2.

Outcomes

IHCA, the primary outcome of interest, was defined according to the American Heart Association’s (AHA) Get with the Guidelines-Resuscitation: any hypoperfusion episode requiring any chest compressions and/or defibrillation, eliciting a resuscitation response by facility personnel with documentation of a resuscitation record [13]. Early IHCA was defined as IHCA occurring within 48 h of cardiac surgery.

Statistical analysis

Data were analyzed using SAS for Windows version 9.4 (Cary, NC). Descriptive analyses are presented using mean (± standard deviation, SD) or median (interquartile range, IQR) and categorical data using number, proportion, or percentage (%). Chi-square, Fisher exact, Mann–Whitney U, and Student’s t-tests were used, as appropriate, to compare categorical and continuous variables between groups. All hypothesis tests were two-tailed, and we used a p value (p) < 0.05 to indicate statistical significance with correction for multiple comparisons testing where appropriate.

Risk-prediction modeling

Minitab Software (PA, USA) with a predictive analytics module with an automated machine learning tool was used to discover the best model among TreeNet®, Random Forests®, Classification and Regression Tree (CART®), and logistic regression models. Candidate prediction variables for the models were as follows: admission vital signs, demographics, and physiologic data as included in Table 1. The top performing model was determined based on the minimum misclassification rate and maximum area under the receiver operating curve (AUROC). TreeNet® consistently provided the least misclassification and was chosen for model derivation. TreeNet® models, which rely on stochastic gradient boosting, consist of several hundred CART® trees with a limited number of terminal nodes. Iterative steps using recursive data sampling are used to grow additional trees to explain residual error from previous trees. While CART® classification only captures interactions of predictor variables in very specific combinations that influence the outcome together, TreeNet® allows for the capture of the overall effect of one predictor variable over another. The models were weighted to ensure equal sample size across classes to overcome unequal distribution of class sizes (number of patients with and without cardiac arrest) in the training dataset. All order interactions between physiological variables were allowed. fivefold cross-validation was used in the test set. Relative variable importance, defined as percent improvement with respect to the top predictor, was used to select variables to develop a simplified model. Test characteristics of risk prediction models including AUROCs, positive and negative predictive values, and likelihood ratios were determined. Given the black box nature of TreeNet® models, alternative CART® models were presented to promote open science and to allow for external validation. Briefly, models were weighted to match sample frequencies, and minimum misclassification cost was chosen to select the optimal tree. Class probability method and fivefold cross-validation were used. CART® trees were pruned to ensure that terminal nodes had > 5% of patients of the root node. Brier scores, which summarize model calibration and discrimination, were used to estimate the accuracy of probabilistic predictions of models developed in the cohort using the R studio package “Scoring.” Brier score values can range between 0 and 1 with lower values indicating better calibration and higher accuracy [14].

Table 1 Univariate associations between predictor variables and in-hospital cardiac arrest (IHCA) among neonates undergoing congenital heart surgery
Full size table

Results

IHCA

Of 118 neonates in the cohort, 12 (10%) patients experienced early IHCA at a median of 8 h after postoperative admission (range 1–44 h), 8 of whom had single ventricle palliative surgery. Episodes occurred at 0, 1, 3, 3, 4, 6, 6, 7, 9, 11, 20, and 42 h after 2-h data was attained. The etiology of IHCA was low cardiac output/hypotension in 7 patients, cardiac tamponade in 2 patients, hypoxemia due to impaired pulmonary blood flow in 2 patients, and pulmonary hypertension in 1 patient. Extracorporeal membrane oxygenation (ECMO) was initiated in eight patients. There was a return of spontaneous circulation (ROSC) in the other four. Four patients had residual cardiac lesions requiring catheterization or surgical intervention. Six patients experiencing IHCA survived hospital discharge. Univariate association between demographic and physiological variables at 2 h after admission to the CICU and odds of IHCA are shown in Table 1.

Prediction models

Discrimination metrics for all models tested are shown in Supplemental File 1. The best performing model was derived using TreeNet® modeling. Of the candidate variables, only 3 served as prediction variables in the model; VIS at 2 h, admission lactate concentration, and change in VIS from admission to 2 h. Supplemental File 2 demonstrates the relative importance of variables included in the TreeNet® model, with VIS collected 2 h after admission being the most important predictor of IHCA. Supplemental File 3 shows one predictor partial dependence plot for the three prediction variables in the model. The AUROC for the TreeNet® model, hereafter referred to as neoIHCA-TN, was 0.99 (95% CI 0.99–1). The model had a sensitivity of 91.7% and a specificity of 98.1%. The model performed comparably upon fivefold cross-validation with an AUROC of 0.92 (95% CI 0.76–1), sensitivity of 75.0%, and specificity of 97.2% (the AUROC curve is displayed in Supplemental File 4). Full test characteristics are shown in Table 2. The neoIHCA-TN model had a Brier score of 0.037 when comparing predicted probabilities of cardiac arrest relative to the occurrence of events in the cohort.

Table 2 Test characteristics for the neoIHCA-TreeNet® prediction model
Full size table

A 3-terminal node (TN) CART tree, henceforth referred to as neoIHCA-CART, to predict the risk of IHCA in the cohort is shown in Fig. 1. There were 2 low-risk nodes (TN-1 and TN-2) with the risk of IHCA being < 3%. Terminal node-3 (TN3) was high-risk with 90% of patients suffering from IHCA. Consistent with the neoIHCA-TN model, the same predictor variables influenced the classification of patients. The VIS score at 2 h was the most important decision point followed by serum lactate, with a 2-h VIS > 17.5 and serum lactate > 5.9 mmol/L being associated with a high risk of IHCA in the cohort. The AUROC for the neoIHCA-CART model was 0.88 (95% CI 0.37–1.00) and 0.76 (95% CI 0.57–0.93) in the training and test set, respectively. Full test characteristics for neoIHCA-CART are shown in Table 3. The neo-IHCA CART model had a Brier score of 0.025 when comparing predicted probabilities of cardiac arrest relative to the occurrence of events in the cohort.

Fig. 1
figure 1

Classification and regression tree (CART) model for prediction of in-hospital cardiac arrest among neonates after cardiac surgery. The three-terminal node (TN) classification tree, referred to as neoIHCA-CART, utilizes two clinically based decision rules (VIS 2 and admission lactate) to predict the risk of IHCA in the cohort. There were 2 low-risk nodes (TN-1 and TN-2) with the risk of IHCA being < 3%. Terminal node-3 (TN3) was high-risk with 90% of patients suffering from IHCA. VIS 2: vasoactive-inotropic score at 2 h after admission to the cardiac intensive care unit (CICU); Lactate: lactate level in mmol/L measured at the time of admission to the CICU

Full size image
Table 3 Test characteristics for the neoIHCA-CART prediction model
Full size table

Discussion

In this study, we demonstrate that it is possible to use common clinical and laboratory data obtained in the first 2 h of admission after neonatal cardiac surgery to identify physiologic derangements that may contribute to early postoperative IHCA. Furthermore, using just arterial lactate and VIS score, we were able to create a simple model that is predictive of IHCA within this dataset. Our results show promise to further risk stratify patients with increased odds of IHCA within the already high-risk neonatal cardiac surgery cohort, which may facilitate a more targeted approach for cardiac arrest prevention processes. However, external and prospective validation are essential before we can support these results for use in clinical trials and other IHCA prevention initiatives.

This pilot study demonstrates the potential to identify neonates most likely to have IHCA within a previously described high-risk cohort. The importance lies in the potential to create a real-time risk assessment score based on the inclusion of early, readily available postoperative physiologic data that could be provided on a dashboard or the patient’s electronic health record (EHR). Identification of the highest-risk patients may enable clinicians to increase vigilance and ensure a shared mental model regarding IHCA risk, including initiation of specific IHCA prevention processes. For instance, the CICU team may discuss and prepare for early recognition and mitigation of reversible causes of suboptimal oxygen delivery/cardiac output before deterioration to potential IHCA occurs (i.e., titration of vasoactive agents, optimize preload, provide sedation, and evaluate for cardiac dysfunction or residual cardiac lesions). It remains to be seen if real-time identification of IHCA risk with physiologic data can facilitate incremental improvement in the reduction of IHCA rate possible with the previously published CAP practice bundle. The postoperative IHCA rate in this neonatal cohort, while over 3 times the odds of other cohorts, is still only 7.3% [15]. At a minimum, we significantly improved the probability of identifying those at the highest risk of IHCA in this neonatal cohort, which would enable a more focused application of prevention processes, decreasing the time and effort necessary to prevent some IHCA [4, 5]. Perhaps most importantly, these data suggest it may be possible to identify this very high-risk cohort within the first 2 h of admission—some episodes which did not occur until several hours later—enabling time to initiate targeted cardiac arrest prevention processes and team IHCA situational awareness.

We demonstrate the clinical utility of decision tree-based model creation within our dataset. Though the majority of the physiologic variables evaluated demonstrated strong association with IHCA, TreeNet analysis yielded a simple model with excellent ability to identify the group at highest risk for IHCA using just 3 variables, as demonstrated by high sensitivity, positive predictive value, and likelihood ratio. NeoIHCA-CART performed exceptionally well in identifying the subset of neonates who were unlikely to suffer from IHCA but only had a modest ability to predict the most at-risk population. For the predictor variables appearing in both models (VIS at 2 h and admission lactate), cut-off points for bedside use were similar, with neo-CART yielding a cut-point of 17 for VIS at 2 h and admission lactate of 6 mmol/L. For clinical context, VIS of 17 at our institution would be high; (as example: VIS of 17 = epinephrine at 0.07 mcg/kg/min, vasopressin 0.5 milliunits/kg/min, milrinone 0.5 mcg/kg/min), while lactate > 6 mmol/L is approximately the 75th percentile for the cohort. These cut-points are certain values that are reflective of low cardiac output and/or impairment in oxygen delivery, as it is expected that patients in these pathophysiologic states would have higher vasoactive requirements and lactate levels, particularly when nearing a peri-arrest state.

The results of this study are consistent with previous retrospective studies focused on the use of physiologic markers of low cardiac output to predict the risk of cardiac arrest in the postoperative cardiac surgery population. Elevated postoperative lactate has been shown to be associated with morbidity, including IHCA, after pediatric cardiac surgery in multiple studies [16,17,18,19,20,21]. While the validity of admission VIS as a surrogate of impaired cardiac output may be impacted by baseline treatment bias, it is hard to argue that escalating vasoactive agents is performed in the absence of impaired physiology. A patient that met both high-risk cut-points of 2 h VIS and changes in VIS score would achieve a VIS score that is consistent with that shown to be associated with a composite poor outcome, including IHCA, after pediatric cardiac surgery [12, 22]. The combination of therapeutic interventions, such as titration of vasoactive-inotropic agents, clinical signs of impaired cardiac output, and laboratory data to predict poor outcomes have previously been described by Ulate and colleagues to create the low cardiac output syndrome score [9, 23]. Similarly, machine learning algorithms use a combination of demographic, continuous physiologic, and clinical data to predict cardiac arrest in several populations, including neonates, admitted for medical and surgical reasons to CICUs [10, 24,25,26]. Compared to the available models and scoring systems, our model is simple, can be used in real-time at one very early time point (2 h after admission from the operating room), and does not require technology that may or may not be available at all institutions. Additionally, our model only includes postoperative neonates. However, as with all the other models described above, ours will need to be validated in a larger, multicenter cohort.

We have previously shown that IHCA etiologies, IHCA high-risk cohorts, and IHCA rates vary among centers [4]. There will also likely be important variation among the admission absolute values of physiologic variables such as lactate and inotrope score. Therefore, we do not presume these cut-points in this analysis will be directly transferrable to other neonatal cardiac surgery programs. Hospitals could create center-specific models or, alternatively, use local consensus-derived cutoffs of key physiologic variables, such as lactate and VIS, as targets in quality improvement initiatives aimed at decreasing local IHCA.

Limitations

Including those discussed above, this study is limited by those inherent with any single-center, retrospective study. We do not have the ability to identify all treatments initiated as a result of admission data (i.e., fluid boluses) that might have impacted the outcomes. These data are primarily admission data. We do not know if findings would be similar for other time points, including time further into the postoperative admission or other times prior to IHCA. This model was developed in a high acuity neonatal CPB cohort (44% single ventricle physiology)—we cannot comment on whether these data would be reflective of other cohorts outside this specific cohort. Additionally, due to the low incidence of cardiac arrest compared to cohort size, overfitting is a significant concern in predictive models developed. We proactively sought to address this by weighting to overcome the unequal distribution of classes and pruning the CART tree to ensure that each terminal node consisted of no less than 5% of the cohort. Although we lacked a distinct dataset to validate our models, we used cross-validation approaches and calibration to ensure robust classification of patients. In this regard, despite having a lower AUROC in training and test sets, the neoIHCA-CART model had better calibration than the TreeNet model. Future external validation of this model in a large contemporary multicenter cohort of cardiac arrest is warranted.

Conclusion

Using center-specific physiologic data, those at highest risk for IHCA after neonatal cardiac surgery with CPB can be identified. Validating this pilot data in a multicenter neonatal cohort is the next step to improving postoperative outcomes—with our goal to combine a validated postoperative physiologic model with high-risk demographic and treatment data to improve the precision for early identification of the highest-risk patients in which IHCA prevention initiatives can be targeted.