Background

Acute poisoning is a global health problem, and prevention of mortality is essential in both intentional and accidental poisonings. Prediction of prognosis in patients with acute poisoning has clinical significance, i.e., it helps in timely and appropriate treatment. However, toxicology research lacks a well-accepted method for assessing the severity of poisoning [1,2,3]. The Poisoning Severity Score (PSS), which has been used in toxicology as a disease-specific scoring system, is used infrequently [4]. Further, it has been misused or modified from its original form [4]. Currently, it has limited clinical utility and is not likely to be applied to many cases owing to their unique clinical circumstances [4].

Mortality prediction in acute poisoning cases has been explored by applying various clinical scoring systems used in critical care [5, 6]. The Acute Physiology and Chronic Health Evaluation (APACHE) score and Simplified Acute Physiology Score (SAPS) are commonly applied tools in the intensive care unit; they are used for predicting the outcomes in specific poisoning cases [7, 8]. The mortality of patients depends on their physiological conditions and unique characteristics of the poisoning. The type of substance, route of exposure, and intent of poisoning affect the outcomes in patients with acute poisoning. Additionally, the toxic substances and their lethality are often unknown. A prediction model of mortality for patients with acute poisoning has to consider both poisoning-related characteristics and patients’ physiological conditions; moreover, it must be applicable to patients of all ages. The objective of this study was to develop a scoring system for predicting mortality in patients with acute poisoning at the emergency department (ED). This work will assist in treatment allocation and therapeutic decision making at early stages for patients with acute poisoning.

Methods

Study design and selection of study patients

This study was a retrospective analysis of a prospective cohort (from 23 EDs), namely the Injury Surveillance Cohort, which was generated by the Korea Center for Disease Control and Prevention (KCDC) from 2011 to 2018. This registry comprised of prospectively collected data on epidemiology and outcome variables of patients with injuries presented at the ED [9]. The registry included cases of poisoning as those with a type of injury. We selected patients with poisoning from this cohort. This selected registry included the baseline characteristics of patients with poisoning: age; sex; time-related factors, such as ED admission time and poison exposure time; poisoning-related variables, such as the intent of poisoning, route of exposure, type of substance (seven categories and 44 types of substances); and initial vital signs at the ED, such as systolic blood pressure (SBP), heart rate (HR), respiration rate (RR), body temperature (BT); and AVPU (A-alert, V-verbal response, P-pain response, U-unresponse) scale of mental status. The registry also contained outcome-related variables, such as mortality at ED or after hospitalization.

Patients who were transferred from the initial ED to another hospital and those who had incomplete data on poisoning-related variables, initial physiological condition-related variables or outcome-related variables, and death on arrival (DOA) at the ED were excluded from this study (Fig. 1).

Fig. 1
figure 1

Selection of study patients. PMS poisoning mortality score

Full size image

The selected study population was divided into two groups: the derivation group for predicting in-hospital mortality and the validation group for external validation of the prediction model developed (Fig. 1).

The Institutional Review Board of the Korea University Hospital approved this study (IRB No. 2020AN0195).

Data analysis

The primary outcome was in-hospital mortality. We compared the characteristics of poisoning patients between the derivation and validation groups (Table 1). Age, sex, time interval from poison exposure to ED admission, classes of substances, intent of poisoning, route of exposure, vital signs of the patients at the ED, and in-hospital mortality were analyzed (Table 1). For analysis, variables related to poisoning characteristics were categorized as follows: intent of poisoning: (1) unintentional, (2) intentional, and (3) unknown and route of exposure: (1) dermal, ocular, or contact; (2) oral; and (3) inhalation. There were 44 kinds of toxic substances that were classified into eight categories from A to H. For categorization, we considered classification of the types of substances. Further, we categorized the substances under the same classification according to the mortality index (MI) of each substance: (A) pharmaceutical agents with MI of less than 0.5%, (B) pharmaceutics with MI of 0.5–5%, (C) artificial toxic substances with MI less than 1.0%, (D) artificial toxic substances or pesticides with MI of 1.0–10.0%, (E) artificial toxic substances or pesticides with MI of 11.0–20.0%, (F) paraquat with MI of 52.5%, (G) gases with MI less than 1.0%, and (H) natural toxic substances with MI less than 1.0% (Table 2) ( Additional file 1 shows this in more detail [see Additional file 1]). MI is estimated by dividing the number of deaths by the total number of patients exposed to the specific substance. Next, it is multiplied by 100 and expressed as percentage. A patient’s physiological variables included age, SBP, HR, RR, BT, and mental status (AVPU scale). They were categorized in accordance with the predictors in SAPS-II [10]. However, SAPS-II does not include RR score. Thus, we categorized RR according to the normal range (12–24 breaths/min).

Table 1 Comparison of patient characteristics between the derivation and validation groups
Full size table
Table 2 Category of exposed of substances according to the class of the substance and the mortality index in the derivation group
Full size table

Development of the new poisoning mortality scoring system

We developed a new-Poisoning Mortality Scoring system (new-PMS) to generate a prediction model for the derivation group (2011–2017 data of the KCDC cohort) (Fig. 1). In the derivation group, we compared demographics, poisoning-related variables, and initial vital signs between the patients at ED who survived and were discharged (survivor subgroup) and those who died at the hospital (in-hospital death subgroup) (Table 3). We selected variables that had statistical and clinical significance in acute poisoning as predictors for developing the new-PMS [11]. Points for categories of each predictor were computed using multivariable logistic regression. The regression coefficient for each category was converted into points by dividing the smallest regression coefficient in the model (Table 4) [12]. Sum of the points for categories in the predictors was the new-PMS.

Table 3 Comparison of characteristics between the survivor and in-hospital death subgroups in the derivation group
Full size table
Table 4 Multivariable logistic regression for the calculation of the new-poisoning mortality scores (PMSs) for each of category of each variable in the acute poisoning patients
Full size table

Evaluation of the new-PMS model performance

Statistical performance of the new-PMS was assessed in terms of calibration and discrimination. Hosmer–Lemeshow goodness-of-fit test was used to evaluate the agreement between observed and predicted mortalities with respect to calibration ability of the new-PMS. The discrimination performance of the new-PMS was evaluated using sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUROC). The optimal cutoff value for calculating sensitivity, specificity, and accuracy was the shortest distance between each point on the ROC curve and the upper left corner. External temporal validation was achieved by calculating AUROC in the validation group (2018 data of the KCDC cohort).

For simple interpretation in a clinical setting, we created risk groups. First, we created 10 score groups of equal sizes from the Hosmer–Lemeshow test. Next, we categorized them into the following four risk groups according to morality of the score groups: very low for less than 0.1%, low for 0.1–0.9%, intermediate for 1.0–9.9%, and high risk for 10.0% or higher [13]. AUROC and observed mortalities were investigated in the derivation and validation groups, respectively [11, 13]. Additionally, we introduced an equation for calculating the predicted mortality from logistic regression of the new-PMS.

Statistical analyses

Continuous variables were reported as medians with interquartile ranges. Differences in the medians were compared using the Mann–Whitney U test. Categorical variables were compared using the Chi-square test. Sensitivity, specificity, accuracy, and AUROC were reported with 95% confidence intervals (CIs). All statistical analyses were performed using SPSS version 20.0 (IBM Corp., Armonk, NY, USA). Two-tailed p values less than 0.05 were considered statistically significant.

Results

Selection of the study population and outcomes

Of the 57,326 poisoning cases, 14,758 (25.7%) were excluded (Fig. 1). Of the 42,568 included patients, 34,352 (80.7%) and 8216 (19.3%) were enrolled in the derivation and validation groups, respectively (Fig. 1). Among the study population, the median time from poison exposure to ED presentation was 2.0 h (interquartile range 1.0–2.0 h). The incidence of in-hospital mortality was 909 (2.6%) and 135 (1.6%) for the derivation and validation groups, respectively (p < 0.001) (Table 1). Characteristics of the derivation and validation groups are presented in Table 1.

Development of the new-PMS in the derivation group

We compared characteristics between the survivor and in-hospital death subgroups within the derivation group (Table 3). The demographics, poisoning-related variables, and initial vital signs of each subgroup are shown in Table 3. Patients of the in-hospital death subgroup showed higher likelihood of being older, male, undergoing intentional poisoning, oral ingestion, and pesticide poisoning; they also initially presented low SBPs, high HRs, high RRs, and altered mental statuses compared with those in the survivor subgroup. Time from poison exposure to ED presentation was not significantly different between the survivor and in-hospital death subgroups (p = 0.057).

We selected 10 predictors from these variables considering clinical reasoning and statistical significance. The 10 predictors (age, sex, type of substance, intent of poisoning, route of poisoning, SBP, HR, RR, BT, and AVPU scale) and categories of each predictor are presented in Table 4. Multivariable logistic regression was used to calculate the points for categories of each predictor. First, we estimated the regression coefficient (B) for categories of each predictor in the multivariable logistic regression model. Next, base constant B was selected as the smallest B in the model. The base constant B was 0.124 in the multivariable model (Table 4). We converted the B category of each predictor into points using the formula B/0.124 (Table 3) [11]. Points for each predictor are displayed in Table 4. The new-PMS was the sum of points of each predictor. The minimum-to-maximum possible range of the new-PSS was 0–137 points. Real ranges of the new-PSS were 0–117 points and 8–115 points in the derivation and validation groups, respectively.

Performance evaluation of the new-PMS

The AUROC of 0.941 (95% CI: 0.934–0.949) in the derivation group was significantly high (p < 0.001) (Fig. 2a). The optimal cutoff value was 49 points. Statistical performance of the new-PMS in predicting in-hospital mortality for acute poisoning cases is shown in Table 5. External temporal validation analysis of the new-PMS also showed a significantly high AUROC of 0.946 (95% CI 0.929–0.964) (p < 0.001) in the validation group (Fig. 2b).

Fig. 2
figure 2

AUROCs of the new-PMS for predicting in-hospital mortality in the derivation (a) and validation groups (b). AUROC area under the receiver operating characteristic curve, PMS poisoning mortality score

Full size image
Table 5 The performance of the new-PMS for predicting mortality
Full size table

During risk grouping for simple interpretation in clinical settings, patients were classified according to the new-PMS into four categories: 0–27 points, very low-risk; 28–40 points, low risk; 41–55 points, intermediate risk; and 56 points or more, high risk. Table 6 presents the observed mortality according to the four risk groups. AUROCs of risk grouping were 0.920 (95% CI 0.912–0.928, p < 0.001) and 0.927 (95% CI 0.909–0.946, p < 0.001) in the derivation and validation groups, respectively. The equation for predicting in-hospital mortality was as follows: Predicted mortality = 1/(1 + e−z), z = − 9.763 + 0.126 × new-PMS.

Table 6 Risk groups within the derivation and validation groups
Full size table

Discussion

Outcome prediction systems for patients with poisoning are rarely studied. Thus, we developed the new-PMS to predict the probability of mortality among patients with acute poisoning. The new-PMS is a simplified scoring system that has several benefits, namely usage of objective predictors, rapid assessment of mortality risk, and early applicability in clinical settings.

Several models for severity of illness that have been used in intensive care units (ICUs) can be applied to patients with acute poisoning. Silakhori [14] reported that the APACHE-II, APACHE-IV, SAPS-II, and Sequential Organ Failure Assessment have acceptable discriminatory power for patients with poisoning, and APACHE II can be used for mortality prediction during early days of admission. However, on the first day of admission, AUROC of APACHE-II was only 0.77 in their study. They did not consider poisoning-related factors. While predicting the outcomes of acute poisoning in clinical settings, we have to consider both the physiological condition of the patient and unique characteristics of the poisoning. The new-PMS reflected two major characteristics of acute poisoning among patients, namely the characteristics of poisoning and the early physiological condition after poisoning.

Lionte [2] developed a risk-prediction nomogram for in-hospital mortality among adults poisoned with drugs and non-pharmaceutical agents. The AUROC of their nomogram was 0.949 for the validation group, which was similar to our AUROC (0.946) for the validation group. However, Lionte’s study used some test variables as predictors and included only those patients who were in the ICU or non-ICU ward, which had a small sample size. Our study was different from Lionte’s study because we performed risk grouping and introduced an equation (for clinical use) for predicting in-hospital mortality instead of using a nomogram. Further, our study included all patients with acute poisoning hospitalized, discharged from the ED with a large sample size of 23 ED-based cohort. We expect that the new-PMS can be applied for very mild to serious acute poisoning cases using the risk group and calculation of predicted mortality at the ED.

Given the unique characteristics of individual xenobiotics, many researchers have attempted to apply physiological scoring systems in patients with specific xenobiotic poisoning [7, 8, 15,16,17,18]. Peter [7] compared the performance of APACHE-II, SAPS-II, and PSS in acute organophosphate poisoning. In their study, AUROC was 0.77, 0.75, and 0.67 for APACHE-II, SAPS-II, and PSS, respectively. Previous prediction outcome models for specific toxic substances have limited value when they are applied to a wide range of patients with poisoning. In the current study, the new-PMS showed excellent performance in predicting mortality, with an AUROC of over 0.9 in all patients with acute poisoning, regardless of the cause of poisoning, type of substance, age, and sex. The present study was an attempt to develop a new scoring system (as an alternative to PSS) for outcome prediction in patients with poisoning.

We used multivariable logistic regression method to assign points for categories of each predictor. This method is commonly used for developing prognosis prediction models [11, 12]. This approach has been used in numerous studies to create a risk scoring system [19, 20]. The reference category of each predictor was determined considering the lowest mortality in SAPS II as the reference category or normal physiological variable value [10]. For example, the mortality of 40-year-olds was 0.04%, which was the lowest among all age-groups, and the point in SAPS II was 0 (Table 2).

Performance of the new-PMS was excellent according to the general guideline of AUROC in both the derivation and validation groups [21]. In simulation studies, the external validation of a prediction model requires a minimum of 100 events of the primary outcome because a small external validation study is unreliable and inaccurate [22,23,24]. Our validation group had 135 mortality cases from a total of 8216 poisoning cases.

For ease of use in clinical settings, we constructed four risk groups according to the new-PMS. This risk group showed a high AUROC (0.927) in the validation group. The observed mortalities increased according to the grade of risk score and showed agreement with the probability of death in the derivation and validation groups. We expect the new-PMS to be useful for objective discrimination between very-low-risk or low-risk patients, which can reduce unnecessary hospitalization. Moreover, patients with high scores can be transferred to the poisoning treatment center at early stages of treatment. Furthermore, the risk of mortality sharply increased in patients with acute poisoning with intermediate- and high-risk scores of the new-PMS. These results suggested that toxicological-specific treatment and early hemodynamic stabilization for intermediate- and high-risk patients at the ED may improve their clinical outcomes. The new PMS will contribute to clinical decision making and therapeutic guidance for patients with acute poisoning.

Limitations

First, in this study, we excluded cases that had missing values of poisoning, outcome, and vital signs-related variables. The traditional “complete cases” analysis may lead to selection bias of subjects and statistically inefficient results [11]. Additionally, we excluded patients with DOA from this study because we considered that these patients required no specific treatments for acute poisoning. Second, the amount of exposure in cases of oral ingestion and the duration of exposure in cases of inhalation or surface absorption are important for predicting the outcomes of acute poisoning among patients. Unfortunately, our cohort did not have data on the amount of exposure and envenomation, such as animal bites. However, the new-PMS developed in this study included the unique characteristics of poisoning, such as intent and route of poisoning. Third, we categorized the toxic substances into eight categories comprising 44 specific substances. This is because all the 44 substances could not be included in the multivariable logistic regression. The clinical severity of poisoning can range from asymptomatic to lethal, depending on specificities of the toxin. For example, specific toxic substances such as paraquat are known to have high mortality irrespective of other predictors [25]. In this study, we considered paraquat as a separate category of substance. Machine learning systems have the potential to predict mortality or carry out early detection of diseases among patients in the ED [26]. Thus, we can try using several machine learning techniques with the 44 special substances without categorization for better performance of prediction models in the future. Lastly, the observed mortalities in this study were as low as 2.6% and 1.6% for the derivation and validation groups, respectively. There is a risk of overestimating/over-fitting the predictive performance of the model if the number of predictors is much larger than the number of outcome events [11].

Conclusions

We developed a new PMS system based on demographic, poisoning-related variables, and vital signs among patients at the ED. The new-PMS showed good performance for predicting in-hospital mortality in both the derivation and validation groups, which is objective and is applicable at an early stage of poisoning. The new-PMS will contribute to clinical decision making for patients with acute poisoning at the ED.