Introduction

Rising global temperatures lead to heat waves, directly causing heat-related illnesses1,2. Severe heat waves, such as those that occurred in Europe in 2003, India in 2010, and Japan in 2018, lead to large numbers of patients with heat-related illnesses and high mortality3,4,5. Moreover, an increase in the prevalence of heat-related illness is associated with the increase in the age of the population, as older individuals are more susceptible to a hot environment6. During the past 20 years, there has been a 53.7% increase in heat-related mortality in people older than 65 years, reaching a total of 296,000 deaths in 2018 worldwide7. Thus, heat-related illnesses are becoming a more common emergency condition with life-threatening consequences. Among heat-related illnesses, heatstroke is the most hazardous condition, in which a person experiences an extremely high core temperature (usually > 40.5 °C), resulting in central nervous system (CNS) dysfunction, multiorgan failure, and disseminated intravascular coagulation (DIC)8,9,10. Previously, we developed a tool, derived using a Japanese nationwide database (in 2010 and 2012) of heat-related illnesses, to predict the disposition and mortality of patients with heat-related illnesses11. The early risk assessment tool for detecting clinical outcomes in patients with heat-related illness, called the J-ERATO score, was defined as the sum of six binary components in the prehospital setting (respiratory rate ≥ 22 /min, Glasgow Coma Scale score < 15, systolic blood pressure ≤ 100 mmHg, heart rate ≥ 100 bpm, body temperature ≥ 38 °C, and age ≥ 65 years), with a total score ranging from 0 to 6 (Supplemental Table 1). We demonstrated that the J-ERATO score is accurate in predicting ICU admission and in-hospital mortality11,12. Furthermore, the usefulness of the J-ERATO score has been successfully validated in South India, which has a population with a different age distribution to Japan13.

Early recognition of and treatment for heatstroke can reduce morbidity and mortality irrespective of its etiology8,14,15. Rapid whole-body cooling followed by advanced care in the ICU is essential to improve survival after exertional heatstroke16,17,18. As most cases of heat-related illness are encountered in the prehospital phase in emergency care settings, rapid recognition, and assessment of patients with clinical manifestations of heatstroke in the prehospital setting play pivotal roles in the timely initiation of interventions. The best practices require rapid cooling of the patient to a temperature less than the threshold for critical cell damage (~104.5°F) in less than 30 min from the time of collapse. Cold water immersion is considered the gold standard treatment, but unfortunately, there are many situations in which this method is unavailable17. Clinical prediction scores and laboratory measures related to outcomes could be useful to prepare for possible clinical deterioration. However, no precise clinical score or biomarker has been adopted for the prediction of clinical deterioration and prognosis in patients hospitalized for heat-related illnesses. To address this clinical gap, we evaluated the relationship between the J-ERATO score and clinical deterioration after hospital admission in a cohort of patients who required hospitalization for heat-related illnesses.

The purpose of this study was to evaluate the ability of the J-ERATO score, which was obtained by emergency medical service (EMS) personnel at the scene, to predict mortality and the development of DIC and multiorgan dysfunction among patients hospitalized for heat-related illnesses.

Methods

Study design and settings

This analysis was conducted using a registered database of a prospective, multicenter, observational study (the Heatstroke Study) in Japan from July 1st to September 30th in 2019, 2020, and 2021. Briefly, the Japanese Association for Acute Medicine (JAAM; Heatstroke and Hypothermia Surveillance Committee) established the Heatstroke Study in 2006. It involved a survey of patients with presumed heat-related illnesses who were transferred to emergency hospitals by EMS personnel11,19,20,21,22,23. Since 2017, this registry has been used to gather information regarding only patients admitted to the hospital owing to a diagnosis of heat-related illness. Approximately 142 emergency hospitals from all over Japan took part in the registry during the study period12. Data were manually recorded by the staff at each participating hospital by using standardized record sheets. The study protocol was approved by the Teikyo University Ethical Review Board for Medical and Health Research (approval number; 17-021-5, board name; “Heatstroke STUDY”, approval date; May 21st, 2020), which waived the requirement for informed patient consent owing to participant anonymity. The procedures followed were in accordance with the ethical standards of the institution’s responsible committee on human experimentation and with the Helsinki Declaration of 1975, as most recently amended.

Definition of heat-related illnesses

Heat-related illnesses were defined as conditions in the spectrum of illnesses progressing from heat exhaustion to heatstroke during exercise or exposure to environmental heat stress. Heat-related illness was diagnosed by the attending physician in the emergency department where the patient was admitted.

Selection of participants

This study included adult patients (aged ≥ 18 years) with a diagnosis of heat-related illness who were admitted to the hospital. The heat-related illness was diagnosed by the modified definition of heatstroke (mJAAM) criteria19. Briefly, the mJAAM criteria consist of factors such as CNS manifestations, hepatic/renal dysfunction, and DIC, but not body temperature. Patients who were not directly transferred from the occurrent site to the participant’s hospital, for whom transportation data were missing, or who suffered cardiopulmonary arrest on arrival were excluded from this study. In addition, patients for whom data on the primary outcome, J-ERATO score, event location, and/or the circumstances of the occurrence were missing were excluded from the analyses.

Data collection

Patient demographics, prehospital information collected by the EMS personnel, and in-hospital information were prospectively recorded (i.e., event location, circumstances of the occurrence, transportation, pre-existing functional dependency, age, sex, prehospital vital signs [Glasgow Coma Scale score, pulse rate, non-invasive blood pressure, respiratory rate, body temperature, and peripheral pulse oximetry], medical history [liver disease, cerebrovascular disease, respiratory disease, chronic kidney disease, immunocompromised disease, psychological disorder, diabetes mellitus with organ dysfunction, and previous heat-related illness], physical findings assessed by the EMS at the scene (seizure, dry skin, skin redness, and skin hotness to touch), in-hospital information [the location of admission, i.e., the ICU or a non-ICU general ward), and survival to hospital discharge. In addition, we gathered information about the patients’ Sequential Organ Failure Assessment (SOFA) score24 on the first day after admission (day 1) and the presence of DIC on day 1. Owing to the lack of a specific mortality prediction tool for heat-related illnesses, the SOFA score, which is a general scoring system for critically ill patients, is commonly used to estimate the severity of heat-related illnesses25. DIC was diagnosed according to the JAAM DIC diagnostic criteria26,27, with a total score ≥ 4 establishing a diagnosis of DIC. Scoring is based on peripheral blood platelet counts, prothrombin time, fibrinogen/fibrin degradation products or D-dimer levels, and the presence of systemic inflammatory response syndrome.

Outcomes

The primary outcome was survival to hospital discharge. The secondary outcomes were occurrences of DIC and SOFA score on day 1.

Statistical analysis

Baseline characteristics were summarized using medians and interquartile ranges for continuous variables and frequencies (%) for categorical variables. The differences between groups were tested using the Mann–Whitney U test. The chi-square or Fisher’s exact test was used to compare binary variables. We evaluated the relationship between the J-ERATO and SOFA scores using Spearman's rank correlation test.

Multivariable logistic regression was used to adjust for the selected covariates to determine whether the J-ERATO is related to the prognosis of patients hospitalized for heat-related illnesses. The following independent variables were selected for the model: the J-ERATO score, event location, circumstances of the occurrence, medical history of psychiatric disorder, and age. A set of selected covariates was chosen a priori based on biological plausibility and a priori knowledge28,29,30,31,32.

We calculated the areas under the receiver operating characteristic curve (AUCs) and their 95% confidence intervals (CIs) to evaluate the predictive ability of J-ERATO score to differentiate between survival and non-survival at hospital discharge, and between patients with and those without DIC on day 1. The best cut-off point for the J-ERATO score to determine the highest mortality risk was based on Youden’s index. We also performed multivariable analyses in the subgroup of ICU-admitted patients, adjusting for the same variables as in the overall analyses.

All statistical analyses were carried out with a two-sided significance level of 5% via JMP PRO 16.0.0 (http://www.jmp.com).

Results

Characteristics of study subjects

A total of 2474 hospitalized patients diagnosed with heat-related illnesses were registered during the study period. After excluding 1137 patients, 1337 patients were analyzed in this study (Fig. 1).

Figure 1
figure 1

Patient flowchart. CA, cardiac arrest.

Full size image

Of the 1337 eligible patients, 1244 (93.0%) survived to hospital discharge. Table 1 summarizes the patient characteristics of the survivor and non-survivor cohorts. Survivors were younger (P = 0.03) and more commonly experienced events outdoors (P < 0.0001) than non-survivors. Compared with survivors, non-survivors exhibited a significant deterioration of prehospital vital signs, with lower systolic and diastolic blood pressures and SpO2, and a higher pulse rate, respiratory rate, body temperature, and disorientation rate (all P < 0.0001). Non-survivors had a higher proportion of seizures at the scene than did survivors.

Table 1 Patients’ characteristics.
Full size table

Table 2 summarizes the duration of hospitalization, outcomes, and severity scores among the survivors and non-survivors. Survivors had a lower proportion of ICU admission (P < 0.0001) than non-survivors. Among the patients admitted to ICU (560 of 1337 eligible patients [41.9%]), survivors had significantly longer ventilator-free (P < 0.0001) and ICU-free (P < 0.0001) days than non-survivors. On day 1, fewer survivors had DIC (P < 0.0001), and survivors had lower SOFA scores (P < 0.0001) than non-survivors.

Table 2 Duration of hospitalization, outcomes, and severity scores between survivors and non-survivors.
Full size table

Main results

Table 3 demonstrates the association between the J-ERATO score and primary outcome among all patients. The J-ERATO score was an independent negative predictor of survival at hospital discharge. Table 4 demonstrates the association between the J-ERATO score and the occurrence of DIC among all patients. The J-ERATO score was an independent positive predictor of DIC on day 1.

Table 3 Association of survival at hospital discharge with the J-ERATO score among all study subjects.
Full size table
Table 4 Association of presence of DIC on day 1 with the J-ERATO score among all study subjects.
Full size table

We assessed the heterogeneity of our results by conducting subgroup analyses of ICU-admitted patients. The subgroup results were similar to the main results, emphasizing the role of the J-ERATO score on both survival outcome at discharge and development of DIC on day 1 (Supplemental Table 2).

We further evaluated the sensitivity and specificity of J-ERATO score cut-off values for the entire cohort. The receiver operating characteristic analyses revealed a J-ERATO score cut-off of 5 points, providing optimal sensitivity and specificity to predict mortality at discharge (AUC, 0.742; 95% CI 0.691–0.787; sensitivity, 74.2%; specificity, 64.4%) (Fig. 2, left). For calibration, the predicted probability and actual observation by score are illustrated (Fig. 2, right).

Figure 2
figure 2

The area under the receiver operating characteristic curve of the J-ERATO score to predict mortality after heat-related illness. (A, left) The J-ERATO score cut-off of 5 points provided optimal sensitivity and specificity to predict mortality at discharge (AUC [95%CI], 0.742 [0.691–0.787]; sensitivity, 74.2% [64.5–82.0]; specificity, 64.4% [61.7–67.0]; PPV, 13.5% [10.8–16.7]; NPV, 97.1% [95.7–98.0]; positive likelihood ratio, 2.1 [1.8–2.4]; negative likelihood ratio, 0.4 [0.3–0.6]). (A, right) Calibration plot. Left: X-axis: predicted probability, Y-axis: observed probability. AUC, the area under the receiver operating characteristic curve; CI, confidence interval; DIC, disseminated intravascular coagulation; PPV, positive predictive values; NPV, negative predictive values.

Full size image

The J-ERATO score cut-off of 5 points also provided optimal sensitivity and specificity to predict DIC development on day 1 (AUC 0.723; 95% CI 0.684–0.758; sensitivity, 70.1%; specificity, 64.0%) (Fig. 3).

Figure 3
figure 3

The area under the receiver operating characteristic curve of the J-ERATO score to predict the presence of DIC on day 1. The J-ERATO score cut-off of 5 points provided optimal sensitivity and specificity to predict the presence of DIC on day 1 (AUC [95%CI], 0.723 [0.684–0.758]; sensitivity, 70.1% [63.1–76.3]; specificity, 64.0% [60.2–67.6]; PPV, 35.8% [31.1–40.9]; NPV, 88.2% [84.9–90.8]; positive likelihood ratio, 1.9 [1.7–2.2]; negative likelihood ratio,0.5 [0.4–0.6]). AUC, the area under the receiver operating characteristic curve; CI, confidence interval; DIC, disseminated intravascular coagulation; PPV, positive predictive values; NPV, negative predictive values.

Full size image

In this study, a large amount SOFA score data was missing, both in the entire cohort (29.9% [400 of 1337]) and the ICU-admitted cohort (missing, 25.0% [140 of 560]). Figure 4 shows the association between the J-ERATO and SOFA scores on day 1 in the entire cohort. An increased J-ERATO score was significantly associated with an increase in the SOFA score in both cohorts (both P < 0.0001). There was a positive association between the J-ERATO and SOFA scores on day 1 (Spearman’s rank correlation coefficient, 0.418; P < 0.0001). Subgroup analysis of ICU-admitted patients revealed a similar association between the J-ERATO and SOFA scores (Supplemental Figure 1).

Figure 4
figure 4

Boxplots displaying the association between the J-ERATO and SOFA scores on the first day after hospital admission. In total, 937 patients were included in the analysis. The numbers of patients in the J-ERATO score groups of 0, 1, 2, 3, 4, 5, and 6 were 19, 75, 97, 140, 224, 302, and 80, respectively. A significant linear trend was observed between the J-ERATO and SOFA scores on day 1 (Spearman’s rank correlation coefficient [rS] = 0.418, P < 0.0001). J-ERATO, early risk assessment tool for detecting clinical outcomes in patients with heat-related illness; SOFA: Sequential Organ Failure Assessment.

Full size image

We addressed the issue of missing SOFA score data by conducting additional analyses. Accordingly, there were no differences in age, event location, occurrence circumstances, prehospital blood pressure, ICU admission rate, or mortality between patients with and those without SOFA data. Patients without SOFA data had a lower body temperature, a lower proportion of prehospital CNS abnormalities, and a lower incidence of ICU admissions (Supplemental Table 3).

Discussion

This study verified the clinical usefulness of the J-ERATO score11 for patients with heat-related illnesses who were transferred to the hospital by EMS personnel. To the best of our knowledge, this is the first study to demonstrate that the J-ERATO score is an independent predictive marker for survival to discharge as well as the occurrence of DIC on day 1, and that there is a significantly positive correlation between the J-ERATO score and the severity of organ dysfunction (the SOFA score) on day 1. Given that the J-ERATO score is calculated with only physical information regarding prehospital vital signs and patient age, our observations are very important to improve outcomes in patients with heat-related illnesses.

While risk factors for mortality have been investigated in patients admitted to the ICU with heatstroke33,34,35, few studies have focused on early screening tools for the prediction of clinical outcomes in patients with heat-related illnesses in the prehospital setting, emphasizing the importance of the J-ERATO score. Furthermore, the J-ERATO score has been validated in South India13. Ninan et al. reported that a higher J-ERATO score at hospital presentation was an independent predictor of underlying multiple organ dysfunction syndromes, defined as a SOFA score > 2 points (P < 0.029)13. On the one hand, we previously reported the potential of machine learning-based mortality prediction models for heat-related illnesses (AUC, 0.92), using 24 variables at hospital arrival22. However, at present, such a model is not clinically feasible in most cases where heat-related illnesses are encountered, as machine learning algorithms require certain computer equipment for calculation. In general, a cumbersome tool is unlikely to be used consistently by busy physicians. Thus, a simple bedside tool for use at the time of presentation of heat-related illness is necessary. Furthermore, there is strong evidence that immediate and aggressive cooling after heatstroke ensures survival with limited sequelae8,15,16,36,37,38, highlighting the need for appropriate prehospital care. Aggressive treatment in the prehospital setting according to the severity of heat-related illness, as determined with the J-ERATO score, is a promising strategy.

Although no universally accepted definition of heatstroke exists in clinical settings, Bouchama's definition is most commonly used globally9. They defined heatstroke as severe illness characterized by a core temperature of > 40 °C and CNS abnormalities resulting from exposure to environmental heat (classic heatstroke) or strenuous physical exercise (exertional heatstroke). However, mortality in heat-related illness is attributable not only to CNS abnormalities but also to organ dysfunction. Of note, the maximum body temperature recorded upon hospital admission after EMS transferal in several fatal cases was below 40 °C19. Mortality after heat-related illness increased significantly to approximately 5% at body temperatures above 38.1 °C19. To address this clinical weak point in Bouchama's definition, the JAAM first established criteria for heat-related illnesses in 2014 and modified the definition of heatstroke (mJAAM) in 201619. Apart from body temperature, the mJAAM criteria for heatstroke include all the components of Bouchama’s criteria21. Kondo et al. used the previous JAAM heatstroke database in 2014 and evaluated the differences between Bouchama’s criteria and mJAAM criteria in terms of diagnosis and identification of mortality. A total of 317 patients were included and divided into the Bouchama, mJAAM, and non-heatstroke groups, each consisting of 97, 302, and 15 patients, respectively. The sensitivity for death was 1.0 (95% CI 0.87–1.0) with the mJAAM criteria and only 0.29 (95% CI 0.14–0.49) with Bouchama’s criteria. On the other hand, both the Bouchama and mJAAM criteria could not predict in-hospital mortality (AUC: 0.52 for both criteria). The median SOFAs were 5 and 3 in the Bouchama and mJAAM criteria, respectively21. Taken together with our current results, the diagnosis of heatstroke with the mJAAM criteria in combination with prognostication with the J-ERATO score, especially more than 5 points of J-ERATO score, could enable physicians to optimize treatment and allocate proper medical resources to patients with heat-related illnesses.

DIC has a reported incidence of ≥ 48% among patients with heatstroke10,39,40. In the current study, DIC was diagnosed based on the JAAM DIC diagnostic criteria, with a total score ≥ 4 indicating a diagnosis of DIC26,41. Gando et al. reported that the JAAM DIC scoring system exhibited superior prognostic value over the International Society on Thrombosis and Haemostasis' overt DIC scoring system for the prediction of multiple organ dysfunction syndromes and poor prognoses in patients with severe sepsis41. Hifumi et al. reported that mortality worsened significantly as the JAAM DIC score increased and was approximately 10% even at a DIC score of only 2 in patients with heatstroke according to the mJAAM criteria19. In the current study, higher J-ERATO scores were associated with a higher prevalence of DIC and mortality. The AUC of the J-ERATO cut-off of 5 was greater than 0.7, which is regarded as an acceptable predictive accuracy42. Taken together, early evaluation of the patient’s status should be helpful in evaluating the severity of heatstroke and determining the optimal treatment at the patient’s presentation.

The current study had several limitations. First, 594/2474 (24.0%) patients in the registry were excluded from the analyses because of missing data, which may confer a risk of bias and affect our results. Especially, the high percentage of missing data may have affected the results of the AUC value. Further evaluation will be needed. Further, 512/1337 (38.3%) patients had missing data regarding the occurrence of DIC and 400/1337 (29.9%) regarding the SOFA score on day 1, for unknown reasons. We addressed the latter issue by comparing parameters between patients with and those without SOFA score data. Our results indicate that the usefulness of the J-ERATO score may be limited to patients with more severe heat-related illness. Second, we did not impute missing values because of the large amount of missing data that would need to be imputed, which might have generated bias. Thus, our findings should be interpreted with caution. Third, although neurological sequelae after heatstroke are clinically important complications43, we were not able to analyze the association between the J-ERATO score and neurological dysfunction because of the small number of events. Thus, the role of the J-ERATO score on neurocognitive function after heat-related illness remains undetermined. Fourth, although data were collected prospectively in this study, the J-ERATO score was analyzed in a retrospective manner. Therefore, we could not evaluate the usefulness of the treatment strategy according to the J-ERATO score. In particular, patients were treated according to the discretion of the physician in charge, which may have been based on clinical information, including the J-ERATO score, resulting in bias.

We found that the J-ERATO score has the potential to predict patients' severity expressing after transportation to the hospitals. Therefore, it is expected that EMS personnel calculate the J-ERATO score and start first aid, including aggressive body temperature control. On the other hand, the reconstitution of the database system to decrease the missing data may be needed in the future. Based on the new database, re-evaluating the accuracy of J-ERATO score for the outcomes despite patients’ severity would be desirable. Further, the construction of new transportation systems or prehospital treatment strategies according to the J-ERATO score needs to be considered.

Conclusion

The J-ERATO score which can be determined by EMS personnel at the scene was an independent predictor of survival at hospital discharge and DIC incidence on day 1 and was positively associated with more severe organ dysfunction on day 1. The J-ERATO score may be helpful in predicting the severity and mortality of patients with heat-related illnesses.