Introduction

Reliability can be described as the absence of unwanted variance in performance, or it can be considered a measure of failure-free operations over time [1]. Failures in the processes of medical care do occur and can lead to adverse events (AEs) that may cause patients harm and even death [2,3,4]. Intensive care units are complex environments where many decisions and interventions are performed in short periods of time [5]. They have been recognized as one of the settings most prone to errors, AEs, and preventable harm to patients [2, 6, 7].

The methods commonly used to detect AEs are voluntary incident reports, retrospective patient chart reviews, and direct observation of clinical practice. The most frequently used method is the retrospective chart review using trigger tools [8, 9]. Each method has different strengths and weaknesses; different methods identify different AEs, and different methods estimate different rates of AEs in the same patient population [10,11,12]. Despite these methodological limitations, in the specific field of pediatric intensive care, the rates, types, severity, and factors associated with AEs have been demonstrated [12,13,14,15,16,17,18]. Nevertheless, although it has been reported that adult patients experiencing AEs have worse outcomes than those with no AEs there is still insufficient data regarding the impact of AEs on pediatric intensive care patient outcome [19].

Therefore, the primary goal of this study is to evaluate the association between the occurrence of AEs and morbidity–mortality in the population of children admitted to the pediatric intensive care unit (PICU) of Hospital Italiano de Buenos Aires (HIBA).

Materials and methods

Setting

The PICU at HIBA is a twenty-bed multidisciplinary unit inserted in a general university hospital that obtained Joint Commission International accreditation in 2015. The PICU has about 800 admissions a year and assists all kinds of critically ill patients. It is staffed by eight pediatric intensivists and four specialty fellows covering the unit 24 h every day. There is one nurse for every two patients. They are all registered nurses, and most are dedicated solely to pediatric intensive care. The unit is also staffed by pharmacists 24 h a day, who participate in medical rounds and validate medical prescriptions.

Population, study period, and design of the study

In this study, a prospective cohort was investigated. Data were collected over 12 months regarding all children admitted to the PICU from August 2016 to July 2017. The observation units were AEs related to the active delivery of care.

Variables

Adverse clinical incidents

An ACI is defined as any incident in which there is an undesirable change in patient status [20].

Adverse events

Any injury or harm that is associated with a healthcare-related incident and cannot be explained by the patient’s disease or condition [21,22,23]. For an ACI to be considered an adverse event, two conditions must be met as follows:

  • The association between the event and the healthcare provided. To explore this association, a six-point Likert scale was used (1, no evidence of association and 6, strong evidence of association). ACIs rated higher than 3 were assumed to be associated with healthcare [3].

  • The existence of harm to the patient (E, F, G, H, and I) [24]

Harm

Any unintended physical injury resulting from medical care that requires additional monitoring, treatment, hospitalization, or that results in death [24].

Severity of harm

The Institute for Healthcare Improvement adapted a classification from the National Coordinating Council for Medication Error Reporting and Prevention Index for Categorizing Errors. This classification includes nine categories from A to I. The first four (A, B, C, and D) consider errors that do not result in harm. The last five (E, F, G, H, and I) refer to incidents that cause harm to patients [25]. Although originally developed for categorizing medication errors, these definitions were applied to any type of error or adverse event [24]. For the purpose of this study, we used categories E to I, because they described the severity of harm as follows:

  • Category E: Temporary harm to the patient and required intervention

  • Category F: Temporary harm to the patient and required initial or prolonged hospitalization

  • Category G: Permanent patient harm

  • Category H: Intervention required to sustain life

  • Category I: Patient death

Types of adverse events

They were classified in seven types as follows [20]:

  • Healthcare-associated infections, for example, ventilator-associated pneumonia

  • Procedural complications, for example, pneumothorax associated with central line insertion

  • Adverse drug events, for example, hypoglycemia secondary to insulin administration

  • Surgical complications, for example, bleeding after surgery

  • Therapeutic adverse events, for example, thrombosis associated with a central line

  • Diagnostic adverse events, for example, interventions associated with misdiagnosis

  • System adverse events, for example, delay in a surgical procedure that causes patient deterioration

Frequency of adverse events

Indicators used to estimate the frequency of AEs were:

  • Adverse events per 100 patient days

  • Percent of admissions with an adverse event [24].

Severity of illness

The Pediatric Index of Mortality 2 (PIM2) score was used as a surrogate of the severity of clinical condition at admission to PICU [26]. It was considered a relevant variable for controlling the association of interest (occurrence of AEs and morbidity–mortality). This instrument has been previously validated in our population [27, 28].

Outcome variable

We used a combined outcome variable that included two components:

  • Hospital mortality

  • Hospital morbidity. It was estimated using the Functional Status Scale (FSS) [29]. PICU-acquired morbidity was considered when there was an increase between PICU admission and hospital discharge FSS greater than or equal to 3 [30].

Identifying adverse events

The direct observation strategy was selected to identify AEs. This choice was made because a small number of incidents were usually obtained from the hospital voluntary reporting system, and the completeness of electronic medical records was considered inadequate to perform comprehensive chart reviews.

A two-step process was designed to define AEs. First, ACI were detected by PICU physicians [20, 31]. Second, the director of the patient safety committee (author of the study, JMCC, a neonatologist with extensive experience in patient safety) evaluated every ACI and defined which would be classified as AEs. The steps of this process were as follows:

Identification of adverse clinical incidents

Two pediatric intensivists and two PICU fellows monitored the patients. When they were directly involved in patient care (office hours every day of the week and four to five nights of the week), they identified ACI by direct observation. Incidents that occurred in the PICU outside of the hours of direct observation were identified by active surveillance. This consisted of open discussion at rounds on the subsequent day and interactions with PICU staff, including nurses, physicians, pharmacists, respiratory therapists, and clerks. These interactions included questions oriented to the practitioners’ knowledge of any potential adverse clinical occurrences or the spontaneous volunteering of such information. Every time an ACI was detected, it was registered in a specific database. A description of the ACI, its probable cause, and consequences to the patient were included in the form (Appendix 1).

Identification and classification of adverse events

All ACI registered were evaluated every 2 weeks by the director of the hospital patient safety committee. If it was established that the incident was associated with the healthcare provided, and it also caused harm to the patient, the ACI was considered an AE. Once an AE was determined, it was classified as one of the seven types previously described. Its preventability was assessed using a 4-point Likert scale (1, definitely not preventable; 2, probably not preventable; 3, probably preventable; and 4, definitely preventable). AEs were considered preventable when categorized as 3 or 4 in this scale [32]. If extra information was needed, two PICU physicians (authors of the study, PE and PA) would complete the requested data but did not categorize the incident. The main purpose of this decision was to minimize risks of bias. In this way, the PICU observers collecting data were not responsible for analyzing them.

Statistical analysis

For descriptive statistics, we used either means and standard deviations, medians and interquartile ranges, or proportions, depending on the characteristics and distribution of the variables.

A multivariate logistic regression analysis was conducted to explore the association between the adverse event variable and combined outcome A univariate logistic regression was performed first. Results from the univariate analysis were used to identify variables for the multivariate model. Statistically significant variables having a p value < 0.1 were introduced into the multivariate model. Variables considered clinically relevant in the univariate analysis, for example, PIM2, were introduced into the model for controlling the association of interest, independently of their p value. All variables were introduced manually. Dichotomous data were introduced in the same way, categorical data were transformed into dummy variables, and continuous data were either introduced in the same way (age) or transformed into dichotomous data (PICU length of stay). Variables with p < 0.05 in the Wald test were kept in the multivariate model. The ratio between the independent variables and the number of outcomes was 1:10. Discrimination of the final model was assessed by the area under the receiver operating characteristic (ROC) curve and calibration by the Hosmer–Lemeshow test in deciles of risk. A p value > 0.05 was considered indicative of an adequate fit [33]. Statistical analysis was done with Stata 9.1 (StataCorp, College Station, TX).

Results

During the study period, August 1, 2016, to July 31, 2017, 842 patients were admitted to the PICU. Their characteristics are reported in Table 1. In total, 467 ACIs were registered, and 249 (53.31%) were considered AEs. There were 142 (16.86%) patients that suffered at least one AE. The median (25–75) AE per patient was 0 (0–0). The rate of AE was 4.27 (249/5827) every 100 patient days. AEs characteristics are summarized in Table 2. There were 227 (91.2%) AEs considered preventable: 118 were classified as probably preventable and 109 as definitely preventable. Appendix 2 includes descriptions of significant AEs.

Table 1 Population characteristics (842 patients)
Full size table
Table 2 Description of adverse events (n = 249)
Full size table

The outcome morbidity–mortality was present in 39 (4.63%) of these patients: 33 died (3.92%) and 6 (0.71%) were discharged with PICU-acquired morbidity. These six patients were sent home with tracheostomy tubes that were not present at the time of admission. Four of them (4/6, 67%) also required mechanical ventilation.

The univariate analysis is reported in Table 3 and the multivariate analysis in Table 4. The regression model demonstrated that three variables were independently associated with the morbidity–mortality outcome: young patient age (OR 0.99, CI95% 0.986–0.999, p = 0.036), high severity of the clinical condition at admission to the PICU evaluated through categories of risk in the PIM2 score, occurrence of AEs during PICU hospitalization (OR 5.70 CI95% 2.58–12.58, p = 0.001).

Table 3 Univariate analysis
Full size table
Table 4 Multivariate analysis
Full size table

The discrimination and calibration of the regression model were adequate. The discrimination assessed by the area under the ROC curve was 0.9243 (CI95% 0.903–0.940). The Hosmer–Lemeshow test showed a p value of 0.8933, and the number of observed and estimated events in each decile of risk is summarized in Appendix Table 5.

Discussion

This study demonstrates that the occurrence of AEs is associated with poor patient outcomes. Experiencing an AE was significantly associated with morbidity–mortality independently of the age and the severity of patients’ clinical condition at admission to the PICU.

The methods commonly used to detect AEs may influence the results obtained. Analyzing the data via incident reporting databases is relatively easy; however, the disadvantages to this method include variable quality of data, the underreporting of AEs, and the description of primarily minor AEs [15, 34, 35]. The retrospective chart review method is the most precise approach to detecting AEs [34]. It has been extensively used in different clinical settings in different countries [8, 9]. However, chart reviews can be difficult, time-consuming, and certainly; the quality of data obtained is dependent on the completeness of medical records [34, 35]. The direct observation method has been found to be the most effective for detecting medication errors [36]. In addition, it has been used to detect any type of AEs in the critical care setting [20, 31]. Nevertheless, it is resource intensive and the validity of the observational data may be compromised by observer bias [34, 35].

Considering the particular characteristics of each method and the distinctive features of our healthcare organizational culture, we decided that direct observation would be the most suitable tool to detect AEs in this setting. The information included in medical records was often incomplete and unreliable to reveal significant healthcare-associated events. Furthermore, the number of incidents voluntary and anonymously reported to the hospital quality committee was insufficient to be considered an accurate estimation of the magnitude of this problem. The risk of bias associated with the direct observation method was of particular concern. For this reason, we decided that PICU physicians would be responsible only for registering ACIs and the patient safety committee would analyze which ACIs would be considered AEs. With PICU observers collecting data, and non-PICU physicians categorizing it, we assumed risk of bias would be acceptable.

The frequency of events in any cohort study can modify the presence or the magnitude of the association of interest. Several authors reported variable frequencies of AEs in the field of pediatric intensive care. The proportion of affected patients varied between 22 and 76% [12, 15]. Reported rates of AEs fluctuated among 2.7, 6, 8, 19, 28.6, and 50.8 AEs per 100 hospital days [12,13,14, 16,17,18]. The frequency of severe or catastrophic AEs variated between 3 and 13% [14, 16]. In this study, almost 17% of children suffered healthcare-associated harm and the rate of AEs (4.27 every 100 patient days) was low compared with previous reports. However, the frequency of severe AEs was higher than previously reported; almost 20% (18.5%) of AEs were considered to cause significant harm that required urgent interventions to save lives (category H), and 4% were associated with permanent injuries or death (categories G and I). The low rate of AEs and the many severe AEs identified may be explained by the method used to detect AEs. We believe that severe AEs could have been underestimated if we used chart review or voluntary reporting in this patient population. We agree with Silas et al. that different methods capture different AEs; therefore, different incidences are expected [15]. Considering that the main goal of this study was to evaluate the impact of AEs on morbidity–mortality, the low rate of AEs reported avoids the possibility of an overestimation in the association found between the occurrence of AEs and poor patient outcomes.

Most AEs detected were categorized as preventable: 47.4% were classified as probably preventable and 43.8% as definitely preventable. This frequency is significantly higher than 36% or 45% reported in previous pediatric intensive care studies [13, 14]. This difference may be related to the instruments used to estimate preventability. The most frequently used tools have been Likert-type scales and expert opinions [13, 14, 23]. To the best of our knowledge, no study has reported preventability based on objective deviations from a well-recognized standard of care or policy. Klein et al. found poor reproducibility for preventability when reviewers re-examined patient data. An international consensus of what exactly constitutes preventability of AEs and agreement on a clear definition is necessary [37].

The multivariate logistic regression model showed three variables independently associated with the outcome of morbidity–mortality: the younger the age, the occurrence of AEs during PICU hospitalization, and the severity of the clinical condition on admission estimated through categories of predicted risk of death by PIM2 score. Severity of clinical condition was the variable most strongly associated with the outcome. Therefore, although patients of this cohort were more likely to die or survive with morbidity if the clinical condition on admission was worse, their chance of intact survival was also significantly reduced if they experienced a healthcare-associated AE during pediatric intensive care hospitalization.

This study has limitations. First, the external validity of the reported findings. The results may not be generalizable because the study was carried out in a single pediatric intensive care unit. Second, the setting where the direct observation strategy was used. Although it has not been used as extensively as retrospective chart reviews, direct observation has been used in critical care settings. Third, the potential risk of introducing the observer bias. We discussed this issue when we designed the protocol of the study, and we decided that the observers collecting data would not be responsible for categorizing them. Fourth, the potential underreporting of AEs, based on the finding that the rate of AEs in this cohort was lower than reported in previous studies. However, as the main goal of the study was to evaluate the association between AEs and morbidity–mortality, we considered that the rate of AEs found could have only underestimated the association found. Moreover, the significant proportion of severe AEs reported suggests that, if underreporting did occur, only minor events were missed. Fifth, there is the potential for overestimation of the preventability of AEs. The use of a Likert-type scale and the lack of a definition of preventability based on objective deviations from standards of care may explain this finding.

Conclusions

This prospective cohort study shows that almost 17% of children suffer healthcare-associated harm during pediatric intensive care hospitalization and that experiencing AEs significantly increases the risk of morbidity and mortality independent of age and severity of illness at admission.