Introduction

Patients admitted to an intensive care units (ICU) are characterized specifically by their need for more advanced level of monitoring than can be provided in other wards of a hospital. However, precisely because of the greater complexity of the healthcare that must be provided to these patients, the risk of patient safety-related incidents is greater in the ICU than in other hospital wards [1, 2].

Errors in healthcare may occur due to an unintended act or by omission. Those resulting from the former are more visible and therefore more easily detectable. Errors of omission are more insidious and more difficult to identify and include, as an example, the lack of adherence to good clinical practice guidelines [3] which occurs, paradoxically, in more severe patients [4]. Explanations for such omissions are a lack of knowledge of good clinical practice guidelines and the presence of barriers that prevent their use, such as the lack of time and/or resources, organizational aspects of the ICU or even resistance to changing work habits [5].

The aims of health risk management are to detect, analyse and prevent patient safety-related incidents using reactive or proactive tools. These tools are complementary to each other. Proactive tools have been proposed as a simple and useful method to avoid errors of act and omission in critically ill patients [6, 7]. Although the implement of proactive tools has been shown to be a promising strategy to prevent adverse events (AEs) [8] and are current used in various aspects of healthcare quality [9], they have not been free of criticism in terms of the permanence of their effect over time and the (increased) consumption of staff resources and time [10]. Among the various proactive methods which have been proposed, the use of random safety audits stands out [6, 8] because rather than attempting to monitor all potential errors all the time, random process auditing systematically selects a subset of error-prone points to monitor at any given moment. Our group has recently developed and validated such a tool—the real time random safety audits (in Spanish: Análisis Aleatorios de Seguridad en Tiempo Real, AASTRE)—and found it to be effective in detecting and remedying errors of omission in real time, thereby improving adherence to guidelines [11].

The implementation, development and effectiveness of tools used to improve healthcare quality and safety are influenced by the leadership and culture of the institution, suggesting that not all tools are equally effective and critically depend upon the environment in which they are applied [12]. The aims of this study are, first, to analyse the feasibility and utility (improve the care process) of implementing a new real time random safety tool and, second, to explore the utility of this tool in core hospitals (those participating in tool design) versus non-core hospitals.

Materials and methods

Methodology for the implementation of AASTRE

Design and description of the checklist

The checklist, previously validated [11], consists of 37 safety measures which are grouped into ten blocks of different areas of care [13–15]: mechanical ventilation, haemodynamics, renal function and continuous renal replacement techniques (CRRT), sedation and analgesia, treatment (two blocks), nutrition, techniques and tests, nursing care and structure. AASTRE is standardly performed three days per week, with randomization of 50 % of the safety measures and 50 % of the ICU patients on the each day of analysis. Each safety measure has a specific definition, assessment criteria and a specific methodology for verification [see Electronic Supplementary Material (ESM)].

The definition of an “eligible patient” is a patient who meets the assessment criteria for each of the measures selected on the day of analysis. If the patient does not meet the assessment criteria, the measure is deemed not applicable.

Role and training of prompters

The safety audits are always carried out after the ICU daily round and require the participation of a prompter and the healthcare professionals directly responsible for patient care (attending physician, residents and nurses). The prompter is one of the two senior attending physicians of each ICU (not directly caring for the patient) who have received the education and training required by the study and who are responsible for verifying and/or promoting the safety measures.

At all centres, training sessions were held on the theoretical aspects and the methodology used in the AASTRE. In addition, all prompters were trained in the goals of the study and in the use of the tool via online. Moreover, practical training was also required, by reproducing at least three safety rounds in a core center prior to the start of the study.

Safety audits

Many of the measures included in the checklist are routinely carried out by healthcare professionals during the ICU daily round. The purpose of the safety audits is to verify that they indeed have actually been carried out. If this were not the case (error of omission), the prompter reminds the healthcare professionals that they should be carried out. In this framework, the possible responses during the audits are: (1) “Yes”—when the measure analysed had been taken/performed on the ICU daily round; (2) “Yes, after AASTRE”—when the safety audit was used to detect an error of omission that has been corrected; (3) “No”—when the measure analysed could not be changed despite the audit; (4) “Not applicable”—when the patient did not meet the assessment criteria.

The checklist and the responses of the evaluations are entered into a web platform (http://www.aastre.es). Safety audits were performed with a tablet at the bedside to facilitate implementation.

Study design and participating centres

This was a prospective study in which six Spanish ICUs participated during a study period of 4 months (February to May 2013). The organization of healthcare did not differ between the participating hospitals. During ICU daily round, the attending physician, resident and nurse responsible for the patient were present.

Table 1 shows the characteristics of the centres and the most relevant initiatives implemented in terms of patient safety (register of AEs, voluntary reporting of AEs and National Nosocomial Infections Surveillance Study). Hospitals that participated in the design of the AASTRE tool and in its pilot study were defined as core hospitals (Hospitals 1 and 2).

Table 1 Characteristics of the centres and safety-related initiatives
Full size table

Variables considered

The variables (and their definition) considered in this study are as follows:

  • Number of patient-days was the number of patients assessed in the total number of days on which safety audits were carried out in the six hospitals.

  • Feasibility was determined by the number of times the AASTRE was completed compared with the number of days on which it was scheduled and for the average evaluation time spent each day. The average time spent each day that the AASTRE was conducted was evaluated at a single core centre.

  • Utility was evaluated as the proportion of changes in the care process carried out as a result of verification. In particular, for each safety measure, a quantitative variable was defined to analyse it according to the following formula (improvement proportion related to AASTRE, IPR-AASTRE):

    $$ {\text{IPR-AASTRE}} = \frac{{{\text{Number of occasions on which the AASTRE changed care process}}\, ({"\text{yes,}}\,{\text{after the AASTRE}}")}}{{{\text{Number of occasions on which the measure was selected }} - {\text{number of occasions on which the patient was not eligible}}}}\, \times \, 100 $$

  • IPR-AASTRE-G is the proportion of changes in the care process carried out as a result of the verification of the safety measures over the total AASTRE.

As the safety measures were distributed according to blocks, and each block analyses common aspects of a particular area of the critically ill patient, a variable was created to reflect the IPR of each block (IPR-AASTRE-B):

$$ {\text{IPR-AASTRE-B}} = \frac{\text{Sum of the number of occasions on which the AASTRE changed care process in each block}}{{{\text{Number of occasions on which the measure was selected in each block}} - {\text{number of occasions on which the patient was not eligible in each block}}}} \, \times \,100 $$

This formula helped simplify the assessment of the impact of different variables on utility. These variables are: the core hospital/non-core hospital, staffing ratio [patient:nurse ratio (≤2:1 vs. >2:1), patient:physician ratio (≤3:1 vs. >3:1)], the Sequential Organ Failure Assessment (SOFA) score, type of patient (medical, surgical, neurocritical and trauma), and length of stay (length of stay at the time of safety audits (<7, 7–14, >14 days).

Ethical aspects

The study was approved by the Ethics and Clinical Research Committee of each investigating centre. Given the characteristics of the study and the anonymity of the data, it was deemed unnecessary to obtain informed consent.

Statistical analysis

All variables were described using the absolute number (N) and with the relative frequency percentage for categorical variables and the mean and standard deviation for continuous variables. Univariate analysis was performed to compare the groups (core hospitals vs. non-core hospitals), and the chi-square test was used for categorical variables. Multivariate analysis was performed to ascertain the impact of the ten blocks of different variables on the IPR-AASTRE-B and with the aim of adjusting possible confounding effects; multiple logistic regression, fixed model and likelihood ratio method analyses were performed for possible confounding effect. The results were expressed as odds ratio and their 95 % confidence interval. The acceptable level of statistical significance was set at p ≤ 0.05. All data analysis was performed using the SPSS version 15 statistical package (SPSS Inc., Chicago, IL).

Results

During the study period, the AASTRE was carried out on 1323 patient-days. Medical patients were evaluated on 673 occasions (50.9 %), surgical patients on 343 occasions (25.9 %), trauma patients on 160 occasions (12.1 %) and neurocritical patients on 147 occasions (11.1 %). Table 2 shows the distribution of the types of patient, the SOFA score, the ratios of patients to professionals, length of stay at the time of the AASTRE and the comparison between core and non-core hospitals. The distribution of type of patients was similar at each of the participating hospitals. Severity of patient disease/condition was higher in the non-core hospitals, and the staff load was higher in the core hospitals.

Table 2 Distribution of the types and severity of patient disease/condition, staffing ratios and length of stay
Full size table

Feasibility

During the study period, the AASTRE was scheduled on 47 occasions in each hospital. Overall, 87.6 % were conducted. There were no significant differences between core hospitals and non-core hospitals in terms of feasibility (87.2 vs. 87.9 %, respectively). Midweek holidays prevented all of the scheduled safety audits from being carried out. In each audit, 100 % of the randomly selected patients were analysed. The average time spent on a safety audit, measured at Hospital 1, was 34.5 ± 29 min.

Utility

The overall IPR-AASTRE-G was 5.4 %, and it was significantly higher in core hospitals that in non-core hospitals (8.5 vs. 2.3 %, respectively; p < 0.0001).

Table 3 shows the number of eligible patients and the IPR-AASTRE in relation to the core and non-core hospitals for each safety measure. When all centres were analysed together, the IPR-AASTRE was >10 % for six measures. Globally, the tool has been useful to maintain improved control of plateau pressure to adjust monitor alarms to safer settings and to improve nutrition. Interestingly, in core hospitals, 16 of the 37 safety measures presented a significantly higher IPR-AASTRE compared to the non-core hospitals. Many of these are included in the good clinical practice guidelines, such as early removal of vascular catheters and semi-recumbent position assessment, among others.

Table 3 Distribution of eligible patients and IPR-AASTREa
Full size table

Table 4 shows the analysis of the measures grouped into blocks (IPR-AASTRE-B). In a joint analysis of the participating hospitals, this variable exceeded 20 % in the mechanical ventilation, nursing care and structure blocks. When comparing core and non-core hospitals, IPR-AASTRE-B was significantly higher in core hospitals in all blocks of measures.

Table 4 IPR-AASTRE-B: distribution and comparison between core and non-core hospitalsa
Full size table

Table 5 shows the impact of the independent variables selected in the IPR-AASTRE-B. Interestingly, the core hospital variable is independently associated to a higher IPR-AASTRE-B in all blocks of measures. The higher nursing workload was associated with a higher IPR-AASTRE in six blocks, while physician workload was only associated with a higher IPR-AASTRE in two blocks; the same was observed for the SOFA score.

Table 5 Variables related to the utility of the AASTRE (multivariate analysis)
Full size table

The “No” response (when the measure analysed could not be changed despite the safety audit) occurred in 2.52 % of cases, with significant differences between core and non-core hospitals (1.74 vs. 3.13 %, respectively, p < 0.05). The distribution of this response was homogeneous among all variables and was associated with the lack of local clinical protocols.

Discussion

This study is the first to prospectively analyse the feasibility and utility of a clinical interaction system applied to daily healthcare practices, namely, the AASTRE, which identifies unsafe healthcare situations in real time, allowing them to be rectified. We found that the utility of this system, which has been especially effective in improving adherence to clinical practice guidelines, was higher in the core hospitals participating in the study than in the non-core hospitals. Our proposed tool contributes additional information and methodology to other reactive initiatives aiming to reduce AEs [16, 17] and at the same time tackles the problem of lack of adherence in daily practice to clinical guidelines [18].

Clinical practice is based on experience and the habits acquired through it, and the latter are often difficult to change [19]. In our study the use of the AASTRE allowed clinical practice to be adapted on >5 % of occasions; for some measures, this proportion can exceed 20 %. However, two findings mark the utility of the AASTRE in our study: (1) The AASTRE led to greater improvement of the care process in core hospitals; (2) the AASTRE facilitated adherence to the good clinical practice guidelines.

The differences in behaviour between core and non-core hospitals may be found in the circumstances that condition the introduction of new approaches to work or changes in routine clinical practice. In this context, Kash et al. [20] described that any change initiative, if accepted as pertinent in the group dynamics (acceptance within the local culture and values), and if properly steered, will be more likely to be included in work habits. Indeed, in our study, the work performed by the core hospitals in designing and developing the tool may have resulted in a broader dissemination and increasing safety awareness of this tool among the healthcare professionals of these hospitals, thereby facilitating its incorporation in daily practice. As a result, there would be less resistance to change as the tool had been recommended as a strategy for improving clinical practice.

In terms of facilitating adherence good clinical practice guidelines, the failure to follow guidelines based on scientific evidence is widely described [21–23]. The AASTRE can facilitate adherence to clinical practice guidelines, thus avoiding errors of omission and facilitating the transfer of knowledge into clinical practice. Gurses et al. [24] recently described four categories of factors that can contribute to such a transfer: factors related to (1) the clinician (training, experience), (2) the guidelines themselves (clarity), (3) cultural inertias of work and (4) the dissemination of guidelines. The AASTRE is a transversal tool that affects each of these categories: (1) it interacts directly with the clinician in terms of daily decision-making; (2) it evaluates the key factors of clinical practice guidelines; (3) it questions cultural habits challenging the scientific evidence; (4) it ensures that the evaluation process itself will facilitate the transfer of best clinical practices gradually and systematically throughout the field of critical medicine. In this context, AASTRE may play a role in educating and facilitating team communication. This aspect also contributes to greater adherence to good clinical practice guidelines, as previously described [25].

The utility of the AASTRE may be conditioned by intrinsic aspects of the field of critical care patients, such as the distribution of human resources, the severity of patients and the length of hospital stay. In our study, a patient:nurse ratio of >2 was independently associated with a greater number of changes in six of the ten blocks. Other authors have reported a negative impact of this variable in monitoring protocols [26, 27], and it has also been linked to a lower quality of patient care, to a higher incidence of AEs [28] and to an increase in average length of hospital stay and mortality [29–31]. In the case of the attending physician, a ratio of >1 physician to 3 patients was independently associated with a greater utility of the AASTRE in just two of the blocks of measures. A care load is perceived by the intensive care physician as being a negative factor on the quality of healthcare [32]. However, the most suitable ratio is not known [33], and neither is its impact on mortality [34].

In our study the disease severity of the patient was independently associated with a greater utility of the AASTRE in terms of renal function and CRRT, treatment and structure blocks. These data are consistent with the observations of Ilan et al. [4] who reported that the most severely ill patients are often excluded from structured decision trees in good clinical practice guidelines. This does not mean that care is neglected; conversely, it does mean that a particular problem (perhaps resuscitation) absorbs all the attention of the healthcare providers, relegating less urgent but equally important measures to second place. In this scenario, the AASTRE has proved to be especially useful since, without any interruption to the work flow, aspects of healthcare for the severely ill patient are recalled and their definitive inclusion into treatment is left to the discretion of the attending physician, based on the indication:risk balance.

Regarding the methodology used for the randomization of patients and variables, the AASTRE has the potential to increase safety awareness of the clinical staff while providing prompt feedback on team performance in critical patient safety domains. However, the interaction between the healthcare professionals caring for the patient and the prompter (all of whom are members of the ICU team) is fundamental in providing empathy and recognizing the relevance of the clinical problems analysed via the AASTRE. This aspect has been previously described by Weiss et al. [35] who, in a prospective study, showed that checklists of safety measures guided by an observer improved mortality and average length of stay in an ICU compared to those carried out through self-verification.

There are a number of limitations to this study. First, the degree of safety awareness at the beginning and at the end of the study was not considered as a variable in the design of the study. Second, the ICUs included in the study are heterogeneous, which may have affected any comparison of results. Third, the length of the study time period (4 months) may have been too short and prevented our addressing other goals, such as the impact on mortality. Fourth, the time spent performing the safety audits was only analysed in one core hospital; however, it should be noted that this aspect is an objective element of improvement since very few other studies explain this [36]. Fifth, the lack of randomization of assessment days may have biased the results. Sixth, the care burden on the nursing staff could have been represented using a nursing workload score rather than the patient:nurse ratio. Finally, the impact of the tool on result indicators, such as incidence of AEs, was not measured.

To conclude, real time safety audits are a useful and feasible tool for the prevention of errors in adult ICU patients. In our study, this tool was most useful in the hospitals which participated in its design. The AASTRE was found to improve adherence to clinical practice guidelines, proving most useful in situations of high healthcare load and in the more severely ill patient. More extensive studies are needed to ascertain its impact on the evolution of care of critically ill patients.