Introduction

In the past decades, the global survival rate of out-of-hospital cardiac arrest (OHCA) patients who received cardiopulmonary resuscitation (CPR) has increased to approximately 8.8%1. In 2015, the survival rate of OHCA patients in South Korea was 9.3%, and in 2019, it was 9.5% in Seoul2,3. The low survival rate and resource allocation of emergency medical services (EMS) raise the issue of termination of resuscitation (TOR) to reduce futile transport to the emergency department (ED). The American Heart Association guidelines for CPR and emergency cardiovascular care suggest the basic life support (BLS) TOR rule for adult OHCA4. All three of the following criteria must be present before moving the patient to the ambulance for transport: (1) arrest was not witnessed by an EMS provider or first responder, (2) no return of spontaneous circulation (ROSC) after three full rounds of CPR and automated external defibrillator (AED) analysis, and (3) no AED shocks were delivered before transport.

However, studies from Asian countries to validate the TOR rule reported a high false-positive rate (FPR) and lower positive predictive value (PPV) than reports from North America5,6. A study from Japan reported developing their own TOR rule7. In Korea, a study by Yoon et al. assessed the predictive performance of the BLS TOR rule for a nationwide cohort in 2019 and reported a high FPR8. The study also proposed a new rule called the Korean Cardiac Arrest Research Consortium (KoCARC) TOR rule. It consists of three criteria of the BLS TOR rule and two other criteria, age (> 60 years) and initial pre-hospital rhythm (asystole). However, this study had some limitations. Most of the data were from a high-level ED, and the data demonstrated less possibility of including study populations from rural areas. The time interval, such as the scene time interval, was not considered a criterion due to missing data.

The KoCARC TOR rule was derived from a nationwide cohort, but whether this rule can be applied to a local population was not validated. In particular, the predictive performance of the new rule must be examined in an OHCA population, including a rural area and those transported to a low-level ED, and is considered a limitation. The rule should be validated again in the group described as a limitation because Korea demonstrates a gap in the survival rate of EMS-assessed OHCA patients according to the level of urbanization9,10. In Korea’s EMS system, EMS providers cannot declare death at the pre-hospital stage. Emergency physicians sometimes refer to the chest compression period in the pre-hospital stage as the criterion for discontinuing resuscitation efforts. Therefore, this study aimed to validate the predictive performance of the BLS and KoCARC TOR rules and modify the KoCARC TOR rule in adult OHCA patient groups in mixed urban–rural areas in South Korea for emergency physicians.

Methods

Study design and setting

This study was a retrospective observational cohort study that analyzed the data of a previous investigation of adult OHCA patients attended by EMS providers in the Busan, Ulsan, Gyeongnam, and Changwon regions in South Korea from November 1, 2019, to January 31, 2020, and from November 1, 2020, to January 31, 2021. We retrospectively evaluated the predictive performance of the BLS and KoCARC TOR rules using FPR and PPV in the study population. We identified the modifiable criteria applicable to the local study population. We examined the time value for the compression time interval (CTI) as a new criterion for the KoCARC TOR rule.

Busan, Ulsan, Gyeongnam, and Changwon regions are located along the coast in the southeastern part of South Korea. Busan and Ulsan are two metropolitan cities, and Changwon is the capital city of Gyeongnam province. The population and area of Busan, Ulsan, Changwon, and Gyeongnam are 3.429 million and 770 km2; 1.166 million and 1057 km2; 1.07 million and 736.3; and 3.448 million and 10,533 km2, respectively. Busan, Ulsan, and Changwon are classified as urban areas, and Gyeongnam as a rural area.

The EMS system in the region, which is government-based and single-tiered, provides basic to intermediate levels of EMS from fire agency headquarters. The EMS resuscitation protocol introduces multiple dispatches (two or more ambulance teams), provides on-site CPR, and transports patients to an ED in an ambulance with ongoing CPR11. EMS providers cannot stop CPR unless spontaneous circulation returns in the patient, either on-site or during transportation to the ED. Only physicians in hospital EDs can declare death12.

Most of the EMS teams in urban areas consist of three EMS providers, usually including one emergency medical technician. However, EMS teams in rural areas often consist of two EMS providers. Most EMS providers possess registered nurse or first/second-grade emergency medical technician certifications. Generally, ambulances staffed with a physician are not available. In the practices of EMS providers, advanced airway management, intravenous access, fluid and drug administration, and withholding/withdrawal of resuscitation must be overseen by medical directors who are mostly emergency physicians in the area. There is one fire agency headquarters in Busan, Ulsan, Changwon, and Gyeongnam, and one medical director council is in charge of the medical oversight of the four regions. According to local EMS guidelines, EMS providers are advised not to exceed 15 min on the scene.

Study population and data collection

The inclusion criteria were all patients with OHCA attended by EMS providers during the study periods. The exclusion criteria were patients under 18 years of age, those arrested by trauma, intoxication, or drowning, and those with obvious death or valid do-not-resuscitate orders. Patients were excluded if the arrest occurred in health care facility staffed with a physician or in an ambulance.

There was no integrated cardiac arrest registration system in the study regions. Pre-hospital and hospital data were collected and then matched and merged. Pre-hospital data on all EMS dispatches are collected and managed by regional fire agencies electronically from scene-dispatched EMS providers. This study collected anonymous pre-hospital data from the four fire agency headquarters. Hospital data were collected from 76 treating hospital EDs in the study area, from low-level EDs to high-level EDs.

As patient variables, information about age and sex was collected. As bystander variables, information regarding whether the bystander witnessed, whether the bystander performed CPR and whether the bystander applied an AED was collected. As EMS variables, data on initial rhythm at the scene, advanced airway, epinephrine administration by the EMS provider, and the EMS process time interval, consisting of the response time interval, scene time interval, and transport time interval, were collected. EMS response, scene, and transport time intervals were defined as the time elapsed from the call to EMS arrival at the scene, from EMS arrival at the scene to EMS departure from the scene, and from EMS departure from the scene to EMS arrival at the ED, respectively. The CTI was defined as the sum of the scene and transport time intervals when the EMS providers performed chest compression. As hospital variables, information on survival to discharge and favorable neurological outcomes were collected. Survival to discharge was defined as the case in which a patient survived until hospital discharge. Favorable neurological outcome was determined as cerebral performance category (CPC) 1 or 2, among the following categories: CPC 1 (good cerebral performance), CPC 2 (moderate cerebral disability), CPC 3 (severe cerebral disability), CPC 4 (coma or vegetative state), and CPC 5 (dead)13.

Outcome measures

The primary endpoint of this study was the predictive performance (FPR and PPV) of the BLS and KoCARC TOR rules for death, and poor neurological outcomes at hospital discharge of the study population. The secondary endpoint was to identify the modifiable criteria to supplement the KoCARC TOR rule and examine the time cutoff value of CTI as a new criterion of the KoCARC TOR rules.

Statistical analysis

We performed a descriptive analysis to examine the study population. Continuous variables are presented as the mean and standard deviation (SD) or median and interquartile range (IQR), and categorical variables are presented as frequencies and proportions. To compare the two groups (the KoCARC TOR rule unmet group and met group), we conducted the Student’s t-test or Mann–Whitney test for numerical variables and the Chi-square test or Fisher’s exact tests for categorical variables after testing for normality. We calculated the predictive performance of the KoCARC TOR rule by sensitivity, specificity, positive and negative predictive value (PPV and NPV), and FPR with a 95% confidence interval (CI) for death and poor neurological outcomes at hospital discharge. The criteria for whether the TOR rule was accurate and reliable were defined when the FPR was close to 0% (less than 1%) with a narrow 95% CI, and the PPV was higher than 99.0%14,15. There are three subtypes of the KoCARC TOR rule, depending on the combination of criteria: (1) KoCARC TOR rule I includes the BLS TOR rule and asystole as an initial pre-hospital rhythm; (2) KoCARC TOR rule II includes the BLS TOR rule and age over 60 years; and (3) KoCARC TOR rule III includes the BLS TOR rule, asystole as an initial pre-hospital rhythm, and age over 60 years. The KoCARC TOR rule III is known as the KoCARC TOR rule.

We verified all four rules in the study population. We constructed a receiver operating characteristic (ROC) curve and calculated the area under the ROC curve (AUC) for all the continuous variables to determine their discriminating performance for death and poor neurological outcomes. The optimal cutoff value was determined using the Youden index, and variables with an AUC over 0.7 were considered modifiable criteria. We performed a sensitivity analysis of the predictive performance to determine the CTI cutoff value. All statistical analyses were performed using MedCalc® Statistical Software version 20.027 (MedCalc Software Ltd, Ostend, Belgium; https://www.medcalc.org; 2022). A two-sided P-value of < 0.05 was considered statistically significant.

Ethics statement

The present study protocol was reviewed and approved by the Institutional Review Board of Dong-A University Hospital (approval No. DAUHIRB-EXP-22-045). Informed consent was waived because the present study was a retrospective analysis of existing data that did not contain personal patient information when the data were collected. All methods were conducted in compliance with relevant guidelines and regulations outlined in the Declaration of Helsinki.

Results

A total of 2884 OHCA patients were eligible during the study period. Of these, 588 patients were excluded based on the exclusion criteria, and 59 patients were excluded as cases in healthcare facilities with physicians. Two hundred sixty-three more patients were excluded due to duplicate data, missing in-hospital data, refusal to provide hospital data and incomplete data for TOR criteria. Finally, 1827 patients were categorized into two groups: patients who met all criteria of the KoCARC TOR rule (n = 530) and those who did not (n = 1297) (Fig. 1).

Figure 1
figure 1

Definition of the study cohort. OHCA, out-of-hospital cardiac arrest; EMS, emergency medical service; KoCARC, Korean Cardiac Arrest Research Consortium; TOR, termination of resuscitation; DNR, do not resuscitate.

Full size image

The demographics, pre-hospital characteristics, and hospital outcomes of the study population are presented in Table 1. The mean age of the total study population was 70.8 years, and 63.3% were male. There was no significant difference in the proportion of urban and rural patients between the two groups (P = 0.38).

Table 1 Demographics, pre-hospital characteristics, and hospital outcomes of the study population.
Full size table

Predictive performance of the BLS and KoCARC TOR rules for death and poor neurological outcomes at hospital discharge

The predictive performance of the TOR rules for death and poor neurological outcomes are presented in Table 2. The BLS TOR rule showed the highest FPR (11.7%, 95% CI 5.9–17.5%) in predicting death but the lowest FPR (3.6%, 0.0–7.8%) in predicting poor neurological outcomes at hospital discharge.

Table 2 Predictive performance of TOR rules for death and poor neurological outcomes at hospital discharge.
Full size table

Modification of the BLS and KoCARC TOR rules for death and poor neurological outcomes at hospital discharge, incorporating age and compression time interval

In the continuous variables, only age had an AUC over 0.7, and the cutoff value was over 68 years (Fig. 2). In the group that met the BLS TOR rule, the CTI cutoff for death was not determined due to its high FPR, but the cutoff for poor neurological outcomes at hospital discharge with less than 1% of the FPR was 21 min (Table 3). In the group that met the KoCARC TOR rule, the CTI cutoff for death and poor neurological outcomes at hospital discharge with less than 1% of the FPR was 25 min and 21 min, respectively (Table 4). When the additional criteria of age over 68 years and the CTI were applied to the study group, no patient for whom the rule predicted the discontinuation of resuscitation survived (Table 5).

Figure 2
figure 2

Receiver operating characteristic curve of age for predicting death at hospital discharge. In this figure, the optimal cutoff value of age for predicting survival was over 68 years with a sensitivity of 64.9%, specificity of 75.0%, AUC of 0.723, P-value of < 0.001), and 95% CI of 0.701 to 0.743). AUC, the area under the curve; CI, confidence interval.

Full size image
Table 3 The cutoff compression time interval in the group met the BLS TOR rule and its predictive performance for death and poor neurological outcomes at hospital discharge.
Full size table
Table 4 The cutoff compression time interval in the group met the KoCARC TOR rule and its predictive performance for death and poor neurological outcomes at hospital discharge.
Full size table
Table 5 Predictive performance of revised KoCARC TOR rule for death and poor neurological outcomes at hospital discharge.
Full size table

Discussion

This study aimed to validate whether the BLS and KoCARC TOR rules could be used in a local cohort and identify the modifiable criterion and CTI cutoff for modifying the TOR rule in adult OHCA patient groups in mixed urban–rural areas in Korea. The predictive performance of all the TOR rules was not identified as an appropriate decision-making tool to terminate the resuscitation effort due to the high FPR. Age was the only modifiable criterion, and the cutoff was over 68 years old. The CTI cutoff for mortality was 25 min in the group that met the KoCARC TOR criterion, and for poor neurological outcomes, it was 21 min in both groups. This study demonstrated strength in verifying its applicability and reliability by reapplying the KoCARC TOR rule derived from a national cohort to a local patient group.

The BLS TOR rule showed a relatively low predictive performance with the highest FPR in this study. This finding is consistent with other studies that reported the predictive performance of the BLS TOR rule in Korea. A study using data from Seoul reported 69.6% specificity and 95.9% PPV for predicting death16. In another study in Daegu, located in the middle of South Korea, the BLS TOR rule showed 85% specificity and 99% PPV for predicting unfavorable survival outcomes17. In contrast, studies from North America reported more than 99% PPV and less than 1% FPR for a clinical prediction of the BLS TOR rule18,19,20. This gap is due to the difference in the EMS system and cultural background. In Korea, resuscitation is rarely withdrawn at the pre-hospital stage, and almost all OHCA patients are transported to the ED. It is rare for patients to clarify their intention to do-not-resuscitation in advance, although it has improved since the enforcement of the Life-sustaining Treatment Decision Act in 201821,22.

The age gap in both the study populations is hypothesized as the main reason for the poor performance of the KoCARC rule. Our study population was older (70.8 ± 15.0 years) than that in the KoCARC study (67.43 ± 15.80 years). In the KoCARC study, the cutoff value for the age criterion was over 60 years, whereas it was 68 years older in our research (in the ROC curve). However, when we changed the age criterion to 68 years, the FPR was 1.7% (1.1–2.3), and the PPV was 99.6% (99.3–99.9), indicating that the values were still inappropriate. The TOR rule should minimize the inclusion of potential survivors in the rule. However, considering our study's results, the age criterion's cutoff value seemed to vary depending on the study population. In addition, estimating the patient’s age only by appearance may be inaccurate at the pre-hospital and hospital stages. Moreover, there are many cases where physiological and chronological age is not matched. Thus, age should be approached carefully as a criterion for discontinuing resuscitation efforts.

It can be assumed that our study group was older than the KoCARC group because it included patients from rural areas. However, this assumption was not true. In the group that met the KoCARC TOR rule, the proportion of patients in rural areas was 31.9%, which was less than that in the group in which the KoCARC TOR rule was not met (34.0%). There was no significant difference in the proportion of patients in rural areas in the two groups. Actually, in this study, the mean ± SD and median age and IQR of the patients were 70.8 (± 15.0) and 74 (61.0–81.0) years in urban areas and 71.8 (± 14.9) and 76.0 (61.0–83.0) years in rural areas, respectively. There was no significant difference in age (P = 0.132). Further research is needed to prove if any insignificant difference in the age of OHCA patients in urban and rural areas was limited to this study area or is similar in other areas.

In this study, the CTI was added to supplement the predictive performance of the TOR rules. However, we could not derive the CTI cutoff value for the BLS TOR rule that predicted mortality as no improvement in the FPR was observed. The probable reason could be the high FPR of the BLS TOR rule. In this scenario, the BLS TOR rule was identified as inappropriate for decision-making by either EMS providers at the pre-hospital stage or emergency physicians in the ED. Kim et al. reported using the STI as an indicator of TOR in Seoul, Korea, in 201514. However, we could not apply the STI as the criterion of the TOR rule for this study population. The local guidelines for medical direction recommend that the STI should not exceed 15 min23. The CTI was the actual period when the EMS providers performed chest compression and ventilation. There may be cases where bystander CPR was either performed or not. However, we did not consider this point because even if bystander CPR was performed, the quality of CPR could not always be guaranteed. In some cases, the EMS processing time interval may affect decision-making for discontinuing the resuscitation effort. Accordingly, we analyzed the processing time interval cutoff value for death and poor neurological outcomes at hospital discharge, which was 42 min and 28 min, respectively (Supplementary 1).

Several limitations of our study must be acknowledged. First, we validated the BLS TOR rule in our research. Still, in some patients, advanced cardiovascular life support (ACLS) was performed using epinephrine at the pre-hospital stage as part of a national pilot project. However, we included this study population because local emergency physicians may use the BLS TOR (terminate resuscitation) rule to decide when to stop resuscitation efforts, even if the patient received epinephrine in the pre-hospital setting. This is because the pre-hospital ACLS TOR rule has yet to be established in Korea. Moreover, this is not standard ACLS, and epinephrine administration did not improve the survival rate in our previous study24. Thus, no significant impact on the study findings is expected. Second, we did not consider in-hospital resuscitation time and interventions, such as coronary reperfusion therapy and targeted temperature management. Moreover, many emergency physicians persist on ACLS for at least 20 min in OHCA patients regarding the in-hospital resuscitation time. Still, we have yet to verify whether this aspect was maintained even in low-level EDs. Third, the CTI criterion was added to the KoCARC TOR rule. Decision-making rules should be as simple as possible to apply. Subsequent studies on the feasibility and reliability of the revised TOR rule are needed.

In conclusion, the BLS and KoCARC TOR rules showed inappropriate predictive performance with high FPR for death and poor neurological outcomes at hospital discharge in adult OHCA patients from mixed urban–rural areas in South Korea. However, it is proposed that the predictive performance could be supplemented by modifying the age criterion of the KoCARC TOR rule from 60 to 68 years and adding the CTI cutoff criterion of 25 min for death and 21 min for poor neurological outcomes. This modified TOR rule was derived retrospectively, and further studies are needed to determine whether this TOR rule could be used in actual patients.