Background

Globally, an estimate of 15 million people suffer from stroke annually. Of these, more than six million people die and five million develop a lifelong disability [1, 2]. Swallowing difficulties or dysphagia, is a common co-morbidity after acute stroke and affects 37 to 78% of all stroke-patients [3]. The incidence of dysphagia is highest early in the course of disease, decreasing from 51% at day zero to 27% at day seven [4]. There is a three times higher risk of developing pneumonia for stroke patients with dysphagia compared to patients without dysphagia [3]. As many as 22–52% of patients with dysphagia aspirate [3], and the risk of developing pneumonia is 11times higher for these patients compared to those who do not aspirate [3]. In addition, aspiration pneumonia is associated with a three times increased mortality risk compared to stroke patients without pneumonia [5]. The overall odds of malnutrition also increases with dysphagia [6]. Another challenge is that half of the stroke patients with dysphagia are unaware of their swallowing problems, which place them at high risk of aspiration and its associated consequences and such a lack of awareness correlates with health problems [7].

Despite the fact that clinical guidelines based on systematic reviews are available and clearly recommend screening for dysphagia before giving anything orally [8,9,10,11] several national audits have demonstrated a deficit in clinical practice [12,13,14]. To increase the adherence rate and the identification of dysphagia, studies have found that implementing a dysphagia screening protocol using a multifaceted implementation strategy can be effective [3, 15,16,17,18]. Consequently, early identification of dysphagia should be of high priority, to reduce the risks of co-morbidity, malnutrition and mortality [6, 19].

The aim of the project was to determine the level of adherence to an evidence-based recommendation on swallow assessment, in a small stroke unit at a Norwegian hospital, and to take actions to improve practice if necessary. We hypothesized that there was a gap between practice and recommendation, and that conducting a criteria-based clinical audit (CBCA) would improve practice.

Methods

Criteria-based clinical audit (CBCA) is a quality improvement cycle, where defined aspects of care is reviewed against evidence-based criteria, to evaluate the degree of adherence against these criteria and to implement necessary changes to practice [20] (Fig. 1). In CBCA, criteria reflects explicit recommendations from evidence-based guidelines and should not give room for interpretation. The stepwise model of Healthcare Quality Improvement Partnership HQIP [21] was used to conduct this quality improvement project. HQIP is an independent organisation promoting quality in healthcare.

Fig. 1
figure 1

Course of the CBCA, from preparation and planning to sustaining improvement

Full size image

Setting, criteria and standard

Our setting was a small stroke unit at a Norwegian hospital. The stroke unit consists of four beds within a medical ward with 34 beds. On average, 90 stroke patients are submitted to the stroke unit annually. The unit is staffed with registered nurses who receive theoretical and practical training in the assessment of dysphagia. In 2011, The Norwegian Board of Health Supervision conducted a national audit of the treatment of elderly stroke patients. For our hospital, this resulted in remarks to deficits, and several measures were instigated to improve the quality of care. One measure was to introduce the swallow screening instrument, recommended in the Norwegian guideline on stroke [9]. We sat the criterion: All stroke patients (100%) with ICD-10 stroke diagnoses should be screened for swallow deficiency with a standardized swallow screening instrument recommended by The Norwegian Directorate of Health.

Process criteria do not measure the result of an activity, but the activity itself [22]. Our process criterion reflected nurses’ adherence to evidence best practice on swallow screening. This criterion was founded on recommendations in several national guidelines for stroke care [8,9,10,11], which was later confirmed in updated resources for clinical decision support [23]. The AGREE II instrument was used to assess the quality of the guidelines [24]. The overall quality was scored 5/7 and 6/7, respectively, for the Norwegian and Australian guidelines, implying they are both guidelines recommended for use.

The swallow screening protocol

The swallow screening protocol recommended in the Norwegian guideline was based on consensus [9]. The screening instrument is a three-step test. The first step is to establish if the patient is eligible for the water swallow test, by assessing consciousness, the muscular control of trunk and head, as well as the patients’ ability to protect the airways. If this assessment is satisfactory, the second step is the water swallow test that can be performed with or without fluid thickener. In this step larynx elevation, and extended and multiple swallowing is observed while the patient swallows water from a teaspoon three times. Coughing, rattling voice and change in respiration is also observed at this step. If the patient pass the second step of the test, they continue to the third step where they are allowed to swallow 50 ml of water and a final assessment of swallow ability is made.

Data and data-collecting tool

None of the guidelines we employed provided an audit tool, a data-collecting tool, to go with it. A search for one in the literature was unsuccessful, so we developed our own [25] ( Table 1). Our audit tool is based on the Australian and English audit on stroke performance and the swallow screening tool recommended in the Norwegian guideline [8, 9, 26].

Table 1 Data Collecting tool. Description of data: Data Collecting tool for demographic factors and mapping the practice
Full size table

The tool was designed to assess adherence to recommendations on swallow screening, and to collect the demographic variables time of admission, gender, age, type of stroke and stroke severity. Stroke severity was classified using the The National Institutes of Health Stroke Scale (NIHSS). The NIHSS scale range from zero to 42, where zero indicates no signs of the disease. The categories of severity were mild 0–8, moderate 9–16 and severe > 18 [27].

An expert panel of four nurses assessed the validity of the audit tool in a three-step procedure. The nurses were considered experienced in assessing stroke patients’ deglutition ability. The first step was a face validity check [25], where the nurses were asked if questions and formulations in the audit form seemed comprehensible and sensible. The panel had a positive subjective impression of the tool. Secondly, content validity was addressed using the content validity index (CVI). Each individual item of the tool (I-CVI), as well as the tool in its entirety (S-CVI) [28] was judged for relevance to the topic investigated. The expert panel used a 4-point scale, ranging from not relevant to highly relevant. The I-CVI was calculated as the proportion of experts that graded the item as “quite relevant” or “highly relevant” (scores 3 or 4), and the S-CVI as the proportion of items that all experts graded with scores of 3 or 4. Content validity was excellent [29], both on item-level and for the overall scale (CVI = 1.0). Third, we pilot tested the audit tool, where two expert nurses individually collected data in eight randomly selected electronic medical records (EMR). To select the EMRs we used the randomization program on the website https://www.random.org/. The purpose was twofold: We wanted to ensure that the audit tool was understandable and relevant, and secondly to check if we would find answers to the questions asked in the audit tool [25, 28, 30, 31]. The data collected by the two nurses were compared and assessed for inter-rater reliability [28]. The proportion of agreement for the audit tool was 93%. A written manual was developed to ensure a uniform use of the tool [31]. One person collected data from the patient’s EMR and the same person checked each record twice to identify errors [30]. For a patient to score “screened for dysphagia” a complete swallow screening had to be documented.

Sample

To identify eligible stroke patients we searched the electronic patient system by the ICD-10 codes, I61-Non-traumatic intracerebral hemorrhage, I63-Cerebral infarction, I64-Stroke, not specified as haemorrhage or infarction and G45.9-Transient cerebral ischemic attack, unspecified (TIA). We excluded patients if they had a pre-existing swallowing problem prior to the acute incident. These patients were identified by having a feeding tube upon admission. A power calculation using IBM SPSS Sample Power 3 informed the number of included patients. As we anticipated a higher number of patients available for inclusion at baseline (retrospective audit data) than at re-audit, at least during a reasonable re-audit time frame, we allowed for unequal sample sizes (2:1). Thus, a total of 115 patients, 80 at baseline and 35 at re-audit were needed to detect an improvement from 10 to 35% as statistically significant (p < 0.05) with a power of 80%. We used a consecutive sampling plan. While baseline data (n = 90) was collected in retrospect comprising EMRs between December 2012 and January 2014, Re-audit data (n = 51) was collected prospectively from January 2014 to May 2014.

Planning and assessing the implementation

Our implementation strategy was to develop a multifaceted intervention, largely based on implementation science to overcome local barriers towards the recommended practice on swallow assessment. To determine the barriers we organised a brainstorming session among the nurses in the unit [32]. To determine what caused the barriers and to identify the main subject, we performed a root-cause analysis [33]. Our barriers were related to knowledge, skills and attitudes among the staff, the screening tool and lack of paper reminders [34,35,36]. A tailored intervention was planned to overcome the multiple identified barriers. This consisted of interactive-didactic workshops [16, 37, 38], the use of local opinion leaders [39,40,41], manual paper reminders [41] and feedback [42]. These multifaceted, tailored interventions are presented in Table 2. In addition we made some minor changes to the layout of the screening tool to make it easier to use [41].

Table 2 Implementation strategies
Full size table

Ethical considerations

The project was approved by the Data Protection Authority in Helse Bergen Haukeland University Hospital according to The Health Personnel Act § 26 [43]. The authors declare that there are no competing interests.

Statistical analysis

Our statistical calculations were performed using IBM SPSS Statistics (version 22.0). The significance level was set at 5%. We reported the demographic data as number and percentage, and as median, range and interquartile range. To assess homogeneity in patient characteristics, and to assess differences in swallow screening between baseline and re-audit, we used the Pearson Chi-Square test or exact test (if assumptions were not met) for categorical variables, and the Mann-Whitney U test for continuous variables. Observed differences were reported with 95% CI. We used the statistical software R version 2.15.0 (http://www.r-project.org/) to calculate 95% CI for the difference in proportions and median.

Results

We analysed 88 EMRs at baseline and 51 in the re-audit. Two EMRs were excluded from the baseline because the patients were transferred to another hospital within few hours after admission. There was no statistically significant difference in median age (p = 0.1), gender (p = 0.3), ICD-10 diagnoses (p = 0.3) or stroke severity categories (p = 0.1) when comparing the patients at baseline to the re-audit. Median stroke severity was however somewhat higher for the baseline group (p = 0.007) (Table 3).

Table 3 Patients characteristics for baseline and the re-audit
Full size table

I61-Non-traumatic intracerebral hemorrhage, I63-Cerebral infarction, I64-Stroke, not specified as haemorrhage or infarction and G45.9-Transient cerebral ischemic attack, unspecified (TIA).

The re-audit showed an improvement of 55%, (p < 0.001), in screening stroke patients for dysphagia compared to baseline (Table 4). Time from admission to swallow screening was documented in only 7% of the EMR at baseline but increased to 76% in re-audit (p < 0.001). The median time from admission to swallow screening was reduced by nearly four hours from baseline to re-audit (p = 0.02) (Table 4).

Table 4 Adherence to recommendation for swallow screening and documentation of time from admission to swallow screening
Full size table

Documentation on whether patients were screened for swallow deficit before they received food or drink was found in 17 (31%) of the EMR at baseline and in 19 (45%) in re-audit (p = 0.24). Among the patients who were registered not screened, 59% had an incomplete screening at baseline and 50% at re-audit (p = 0.6).

Discussion

This quality improvement project revealed a serious and large discrepancy in the care of acute stroke patients at baseline, when reviewed against evidence-based recommendations. Specifically, only 6% of patients admitted to the unit had their swallow ability examined. A systematic approach to develop a change strategy, tailored to local barriers and largely founded in implementation science, led to a significant improvement where this percentage increased to 61%. Other substantial improvements gained were the documentation of timing of screening, which increased from 7 to 76% and the time from admission to screening for dysphagia, which was reduced by nearly four hours.

Our project had several noteworthy strengths. First, we invested efforts in developing the audit tool, and later in testing it for validity. Also, we made sure that recommendations used to set the criteria, explicitly came from evidence based guidelines that we systematically searched for, which in turn were appraised for quality using the acknowledged AGREE II instrument. We believe that transparency in this process gave the project credibility among the staff in the unit, including the management. Later, we included them in the process of identifying the barriers and in the development of the multifaceted change strategy, in line with the success factors highlighted in the literature.

Our significant improvement is not unique if compared to other similar studies. In fact, most other studies aimed at improving screening for swallowing problems demonstrated a better care after intervening, with improvements between 35 and 58% [16, 17, 44]. Still, the post intervention result in our project showed a higher overall adherence than these other studies. This may be due to the difference in context in which we did our studies, such as in an emergency department [44], across a large hospital [17], or even in multi centres [16]. We anticipate that all of these settings are more complex to implement changes to practice, than ours. Unlike the aforementioned studies, our hospital is a small local hospital with only 90 stroke patients admitted annually. Our target group was the nurses conducting swallow screening, and the swallow screening tool had already been introduced prior to our first audit. Where the healthcare professionals targeted in referred studies were dealing with so much more than stroke patients on a daily basis, our target group consisted of nurses assessing stroke patients for swallowing problem. This may explain partly why sustainability was poor in several of the studies [16, 44]. Also adding complexity may be if the goal is not only to improve screening for dysphagia, but extends to other areas of stroke care too [16]. When Hinchey et al. discussed reasons for only gaining smaller amounts of improvement than hoped for, or the lack of sustainability, then reasons like lack of involvement of physicians or leaders, a short intervention time frame, and a lack of resources for the implementation were listed [45].

Quality improvement projects must always be assessed cautiously for generalizability, due to its local contextual factors. Still, we believe that many of our success factors may be of importance and relevance in other settings and projects. The overarching factor of success in our project was to contextualize the project by targeting local barriers and facilitators and to adapt implementation strategies from implementation science that fit the barriers profile of the setting. In this work, we involved the health care professionals and their leaders for who we expected to change their behaviour. There are some threats to the internal validity of our project. The most important one is the chance of a Hawthorne effect, where the health care professionals in our unit may have adjusted their behaviour as they knew that they were being observed for a certain time period. We believe that the risk of bias in the baseline and the re-audit measurements was small. This is due to the same trained person collecting data [28], the data was verified twice and because there was an instruction to the audit tool [31].

Since we did not have control over other factors than our intervention, a causative association between intervention and result cannot be used to draw definitive conclusions [28]. However, we are not aware of any other intervention or factors, besides those we implemented, which could have affected the outcome of this CBCA. It is therefore likely that our implementation strategy led to this increase in adherence.

Conclusions

This criteria-based clinical audit proved successful in improving adherence to evidence based recommendations on swallow screening in the care for acute stroke patients. Improving processes of care to identify dysphagia in the acute phase of a stroke, is the first and necessary step to reduce fatal consequences of aspiration pneumonia. We believe that our tailored implementation strategy, developed to fit the local context, has practical relevance for improving the quality and safety of stroke care in other contexts.