Background

Head injuries are a frequent presentation to emergency departments (EDs) worldwide. They are caused by external forces to the head (such as sport, falls, motor vehicle accidents, assaults or blast injuries) [1]. Country-based incident estimates range from 108 to 332 hospitalised new cases per 100,000 population per year [2], and the incidence is rising as a consequence of increased transport-related injuries in low- and middle-income countries [1]. Traditionally, head injuries are classified as mild, moderate or severe head injury, depending on the patient’s presenting level of consciousness as expressed by the Glasgow Coma Scale (GCS) score [3]. The scale ranges from a low of 3 (comatose) to 15 (awake and following commands). The vast majority of patients (80 to 90% depending on definitions) present with normal or near normal GCS scores (14-15/15) and are therefore classified as “mild”. In non-paediatric patients, the highest incidence of mild traumatic brain injury (mTBI) is seen in males between the ages of 15 and 24 years and in men and women 65 years of age and older [4].

In most cases, patients who experience mTBI will recover fully, typically within days to months. Up to 15% of patients with a normal GCS score of 15 have an acute lesion on head computed tomography (CT), but less than 1% of these patients have a lesion that requires neurosurgical intervention [4–6]. However, up to 15% of patients diagnosed with mTBI experience persistent disabling problems [7, 8] including reduced functional ability, heightened emotional distress, and delayed return to work or school [9, 10] 3 months or more after injury. As the ED is the main, and often only, point of medical contact for these patients, ED care may provide opportunities to impact on the outcomes of these patients.

The challenge for the emergency physician is to identify which patients with a head injury have an actual traumatic brain injury (TBI) requiring further management, and which patients can safely be sent home [4]. Patients with subtle symptoms or signs can still progress to suboptimal outcomes [4]. Despite the existence of several high quality evidence-based clinical practice guidelines (EBCPGs) internationally and in Australia [11], research shows inconsistent implementation of recommended practices. In earlier phases of our study, relevant EBCPGs were identified and assessed for their quality [11]. Subsequently, key clinical recommendations and the evidence underpinning these recommendations were studied, and two consensus meetings were organised to develop locally relevant recommendations to serve as the basis for developing our management outcome measures [12]. Table 1 outlines the evidence-based recommendations, their relevance to the management of this patient group, and evidence regarding gaps in practice.

Table 1 Key clinical recommendations in the management of mild traumatic brain injury in emergency departments
Full size table

We are not aware of trials which have evaluated implementation of this suite of recommendations in ED practice internationally or in Australia. A cluster randomised trial (CRT) performed by Stiell and colleagues [29] in Canada implemented the Canadian CT Head Rule (CCTHR) to reduce unnecessary CT scans. Their implementation intervention package included education (distribution of articles describing the development and validation of the rule, pocket cards and posters, and a 1-hour teaching session to review the evidence and clinical application of the rule), and a mandatory real-time reminder of the rule at the point of CT scan request that required checking off the rule before the scan would be performed by the imaging department [29]. The intervention was designed to be low-cost and simple and had been shown to be effective in reducing cervical spine imaging rates for neck trauma in a simultaneous study [44]. In the implementation of the CCTHR, the intervention failed to reduce the number of CT scans requested; in fact, CT imaging rates increased in both intervention and control group (62.8% before compared with 76.2% after, absolute difference +13.3%, and 67.5% to 74.1%, absolute difference +6.7%, respectively; a difference in change between groups of 6.6%). However, the baseline rates were much lower than expected and approaching the estimated ‘safe’ rate (62.4%), which may have reduced the likelihood of further reductions (that is, indication of a ceiling effect) [29].

Increasingly it is considered best practice to base interventions aiming to change clinical practice on a theoretical approach to identifying factors potentially influencing practice change [45]. Theory can offer a generalisable framework for considering effectiveness across different clinical conditions and settings [46], and, given the wide variety of factors that may influence practice change, basing interventions on different theoretical assumptions may prevent overlooking important factors [47] and will ultimately help to understand how and why change is achieved. Additionally, this approach offers the possibility of optimising interventions to increase effect sizes in subsequent studies.

Aims and objectives

The aim of this trial, which is part of the Neurotrauma Evidence Translation (NET) programme [48], is to test the effectiveness of a targeted, theory- and evidence-informed implementation intervention to increase the uptake of three key clinical EBCPG recommendations regarding the management of adult patients (18 years of age or older) who present to Australian EDs with mild head injuries, compared with passive dissemination of the EBCPG.

More specifically, the primary objective is to establish whether the intervention is effective in increasing the percentage of patients for which a prospective measure of post-traumatic amnesia (PTA) using a validated tool is performed in the ED until a perfect score is achieved or the patient is transferred or admitted.

Secondary objectives include establishing whether the intervention is effective in: increasing the percentage of patients for which two other assessment methods of PTA were performed (assessment without using tool, and whether the administration of the validated tool was completed at least once); increasing the percentage of patients for which CT scanning is appropriately performed; increasing the percentage of patients who receive patient information upon discharge home from the ED; changing factors that are thought to mediate the effect of the intervention (such as increasing the staff members’ knowledge, or decreasing negative beliefs about consequences); and improving selected patient clinical outcomes and quality of life.

In addition, our study aims to: quantify the trade-off between the hypothesised improvement in patient management and patient health outcomes, and the additional costs (savings) arising from delivery of the implementation intervention and from any subsequent changes in clinical practice and healthcare utilisation by conducting a cost-effectiveness analysis; and evaluate intervention fidelity, assess acceptability of the intervention, and assess perceptions around success of the implementation processes and influencing factors, by conducting a process evaluation.

Methods/design

The design of this study is a CRT, with the EDs being the clusters including the ED staff members involved in the treatment of patients with mTBI and the patients treated. A randomised design is the preferred one to evaluate the effectiveness of an intervention since it minimises bias in estimating intervention effects compared with other study designs [49, 50]. In this study, clusters (hospital EDs) have been chosen for two reasons: the intervention is targeted at the team of ED staff, and EDs represent patient populations in geographical areas, precluding the use of an individually randomised design [51, 52].The study has two levels of participation (Figure 1): NET and NET-Plus. NET will measure clinical practice outcomes, but not patient outcomes, whereas NET-Plus will include both. These two options will be offered to hospitals because we anticipate that some may prefer not to recruit patients.

Figure 1
figure 1

Trial design. EBCPG, evidence-based clinical practice guideline; ED, emergency department; NET, Neurotrauma Evidence Translation.

Full size image

Recruitment and eligibility

Recruitment of emergency departments and inclusion/exclusion criteria

The sampling frame for this study will be derived from the Australasian Society for Emergency Medicine ED Directory list of 24-hour Australian EDs (regional and metropolitan) [53]. EDs involved in the pilot and/or development of the intervention will not be invited to participate in the trial.

Hospitals will be included if we receive written informed consent from the ED Director to randomise the ED to control or intervention group. Hospitals will be excluded if they are either non-24-hour EDs or specialised hospitals (for example, women’s or children) and do not therefore routinely treat adult patients with TBI. In addition, EDs will be excluded if they do not have a CT scanner on site, or if there is significant risk of bias (for example, risk of contamination due to two EDs having the same ED Director, or senior influential clinicians working across sites).

We intend to recruit 34 EDs (see sample size section). Based on a previous Australian study (Knott J, Bosch M, personal communications) which observed a 50% ED participation rate, we plan to initially invite 68 randomly selected hospitals to participate in the trial. Our recruitment strategy will follow recommendations regarding the recruitment for CRTs in secondary care settings [52]. ED directors will be considered ‘gatekeepers’ (people who can grant formal permission for the organisation to be involved) of the clusters. As a first step they will be contacted by telephone to inform them about the study, and let them know that they will receive a recruitment pack (including an invitation letter from the research team, explanatory statement, and an expression of interest form). The ED Director will be encouraged to discuss the trial with the relevant stakeholders within their ED (in particular Directors of Trauma or staff members responsible for trauma and/or research, and senior clinicians). Upon receipt of the expression of interest, a recruitment meeting will be organised (via telephone, skype, video-link or face-to-face) with the study investigators and the relevant stakeholders in the ED. During this recruitment meeting, the details and logistics of the study will be discussed. Following this meeting, the ED Director will be asked to provide written informed consent to trial entry (either NET or NET-Plus), using a consent form sent prior to the meeting [51, 52, 54]. Non-responders will be followed up by email and telephone [55] at least once, before inviting the next random selection of hospitals.

The content of the recruitment information will be designed to target factors influencing the decision to participate in this trial [56] that were identified in semi-structured interviews in an earlier phase of this study (for example, by providing information around resources and including information around the benefits for the patients to create ‘tension for change’). In addition, professional bodies will be engaged to endorse the trial [57].

Recruitment of emergency department staff members for data collection purposes and inclusion/exclusion criteria

ED medical and nursing staff will be invited to complete two questionnaires (at baseline and endpoint). The sampling frame for staff participants will be based on the active list of ED medical and nursing personnel in each ED. Inclusion criteria for nursing staff are: 1) current ED appointment; 2) on active practice roster; 3) enrolled or registered nurses. Inclusion criteria for medical staff are: 1) current ED appointment; 2) on active practice roster; 3) registrars, Hospital Medical Officers or consultants. For both groups, exclusion criteria are: 1) students/interns; 2) clinicians not currently engaged in clinical practice (for example, academic staff); and 3) for nurses, bank or agency nurses.

We plan to randomly select 50 eligible ED clinicians per clinical group per hospital (that is, 100 in total) when available at baseline and endpoint. For hospitals with fewer than 50 clinicians per clinical group, we plan to invite all clinicians satisfying the inclusion criteria to participate for data collection. Staff members who were contacted at baseline will be asked to participate at endpoint, and additional staff will be randomly selected in instances where staff have been lost to follow-up. Staff members will be included if they fit our inclusion criteria and consent to participate.

In addition, some key staff members in eight intervention sites will be recruited to participate in a brief evaluation interview over the telephone. To identify staff, we will use a key-informant method [58].

Finally, all ED Directors or their delegate will be invited to complete two brief checklists over the telephone regarding ED characteristics at baseline and endpoint.

Identification of study patients and inclusion/exclusion criteria

In order to determine the effect of the intervention on clinical practice outcomes, a chart audit will be conducted by independent chart auditors. Patients are eligible if they are aged 18 years or older, present to the ED within 24 hours following acute blunt head trauma, and have a GCS score of 14 or 15 at presentation [14]. Exclusion criteria include penetrating injuries, and non-traumatic TBI such as caused by stroke.

A patient identification protocol will be developed and chart auditors will be trained with the aim of maximising consistency in identification of patients, and minimising selective recruitment. We anticipate that patients will be identified retrospectively from clinical records using medical codes (for example, relevant International Statistical Classification of Diseases and Related Health Problems Tenth Revision, Australian Modification (ICD-10AM) discharge codes). Instructions will be piloted and tailored to the hospital medical record system. As there may be some between-hospital variation in the approach taken, the exact process to identify patients will be carefully recorded in each hospital.

Recruitment of patients for follow-up – NET-Plus only

In hospitals which choose to participate in NET-Plus, patients included in the chart audit will be contacted by telephone by an ED staff member to invite them to participate in a follow-up telephone interview questionnaire by trained researchers. Informed consent to pass on their contact details to the NET research team will be sought (see ethical review and informed consent).

For this component, additional exclusion criteria are: 1) not consenting to participate in the follow-up study; and 2) not being able to participate in a telephone interview, because of an inability to speak English, being hearing-impaired, or having cognitive or intellectual impairment (history of dementia, other neurological disorder, intellectual disability drug or alcohol abuse or other major psychiatric disorder such as psychosis for which hospitalisation has been required).

These criteria will be confirmed by the ED staff asking for consent and again by the trained researcher at the start of the interview.

Randomisation and allocation concealment

EDs will be allocated to the intervention and control groups using the method of minimisation proposed by Pocock and Simon [59]. Minimisation has been shown through statistical simulation studies to outperform stratified allocation methods for maintaining baseline balance in prognostic factors [60]. In addition, the method allows for balancing across a greater number of prognostic factors. The minimisation factors will include: the existence of a protocol for mTBI (consistent with our recommendation for appropriate PTA assessment); size (annual presentation rate 2012), included as a proxy for other variables that are harder to measure such as staffing; rurality; and type of participation (NET or NET-Plus). Minimisation is essentially a deterministic method [60], and therefore use of this method can increase the risk of selection bias [61]. We plan to reduce the risk of selection bias through inclusion of a random component and allocation of batches of EDs by a statistician independent of the trial. The statistician will be provided with a file containing only ED codes and the minimisation factors (thus no identifying information). For each batch of EDs, the statistician will randomise the order of EDs before using the minimisation algorithm to allocate the EDs to intervention and control groups.

Blinding

Due to the nature of the intervention, it will not be possible to blind ED staff members to group allocation. To limit the possibility of selection and detection bias, chart auditors will be independent, where possible, of the hospital. In instances where the hospital does not approve an independent chart auditor, a solution will be discussed which aims to reduce the risk of selectively identifying patients (for example, using a staff member from another unit who will receive training from the research team). External chart auditors and the trial statistician will be blinded to group allocation.

Interventions

Intervention group

Intervention group EDs will receive a targeted, theory- and evidence-informed implementation intervention. The intervention builds upon the previous phases of this study. In phase one, relevant EBCPGs were identified, assessed for their quality, and the focus of the study was determined [11, 12]. In phase two, semi-structured interviews underpinned by theoretical perspectives on individual and organisational change were conducted with 42 ED staff members in 13 Victorian EDs. Thematic analysis was used to identify the factors perceived to be influencing current practice as well as change in the ED setting. In phase three, our intervention was developed to address the modifiable influencing factors to maximise the effectiveness of the intervention. Intervention components were piloted for feasibility and acceptability. Table 2 summarises the content of the intervention. Only broad components are described in this protocol to prevent contamination of the control group. The process of development of the intervention and a description of its content will be published in detail post-data collection.

Table 2 Planned delivery of the intervention
Full size table

Control group

Control EDs will receive a copy of the EBCPG [14] and all the materials/components needed for the outcome assessment. Post-data collection, control EDs will be invited to a repeat train-the-trainer 1-day interactive workshop where they will also receive the components for the local education and all other intervention materials.

Timing of recruitment, intervention delivery, and follow-up

Figure 1 shows the flow of an ED through the trial. We started recruitment and completion of ethics and research governance procedures in February 2013. Baseline data collection has commenced (October 2013). Intervention delivery and endpoint data collection will take place in 2014.

Study outcomes and outcome measurement

Tables 3 shows our clinical practice outcomes and data collection methods.

Table 3 Clinical practice and proxy clinical practice outcomes and data collection methods
Full size table

Primary outcome

Our primary outcome is ‘appropriate PTA screening’ (PTA; Table 3). This outcome measures whether a prospective assessment of PTA was appropriately undertaken, where appropriately undertaken is defined as using a validated tool, until a perfect score was achieved (indicating absence of acute cognitive impairment) before the patient was discharged home (or the patient was admitted or transferred). This measure has been chosen as the primary outcome for the following reasons. First, PTA has been shown to have better predictive ability with clinical outcomes compared with GCS [15–18]. Many patients are oriented by the time they are first assessed, and therefore achieve the top score on the GCS scale, which leads to the common misperception that a perfect score means a normal examination [62]. Ponsford and colleagues conceptualised the Revised Westmead PTA Scale for screening of PTA in patients with mTBI, identifying it as more sensitive to the presence of PTA than GCS [63]. Shores and colleagues [17] studied the diagnostic accuracy of the Revised Westmead PTA scale and showed that at the time of the second neurological observation 60% of participants were not able to lay down new memory; yet 87% of them had been assigned a GCS of 15. Another recent study [15] showed that whereas patients with GCS scores of 13 or 14 did not differ from those with a score of 15 with respect to neuroimaging abnormalities, patients who had experienced PTA for more than 30 minutes were more likely to have intracranial abnormalities on imaging. So, by using a validated tool as part of the neuro-observations until patients receive an optimal score, one reduces the risk of failing to classify mTBI patients and prevents patients from being discharged from hospital while they are suffering from acute cognitive impairment [19, 20].

Second, this outcome can be reliably measured (the score is an objective measure in the medical records). Finally, given the estimated low rate of appropriate PTA screening (see sample size section), there is considerable opportunity for improvement.

Secondary outcomes

Clinical practice outcomes

Two other measures for PTA were selected as secondary outcomes. ‘PTA screening-tool’ measures whether the administration of the validated tool was completed at least once. ‘Memory-clinical’ measures whether staff members have made an assessment of memory using questions in their clinical assessment. Other secondary measures assessing the effectiveness of the intervention in improving the ED management of mTBI are listed in Table 3. On the cohort of patients for whom risk criteria are recorded, ‘CT scan-clinical criteria’ (CT) measures whether a CT scan was provided in the presence of a risk factor that justifies the scan (see Additional file 1). This measure is therefore indicating whether a scan was appropriately received; however, we will not be able to measure whether a scan was ‘appropriately denied’ because of the inconsistency in recorded clinical criteria. ‘Provision of patient information’ (INFO) measures whether information was recorded as provided upon discharge home from the ED. Via the chart audit, we will not be able to measure what information was handed out (for example, the information used in our intervention [40] or other). However, this will be assessed in the patient follow-up interview in NET-Plus hospitals. Two measures of ‘safe discharge’ are included, the first being a composite score of PTA and INFO (for all patients), and the second provides a composite measure of whether the patient received appropriate care for all of the three clinical practices (PTA, CT, INFO; for the subgroup of people for whom risk criteria were recorded).

Clinical practice outcomes will be measured retrospectively through chart audit by an independent, trained chart auditor. Data will be collected over the 2 month period following the last intervention component.

Proxy measures of clinical practice

Secondary outcomes also include proxy measures of the clinical practices of interest. These proxy measures consist of self-report measures and clinical vignettes [64].

Predictors of clinical practices

Table 4 shows the factors we hypothesise to mediate the intervention effects, listed per clinical practice. Factors have been grouped into two categories: 1) collective level constructs – clinicians’ self-reported cognitions about their EDs team climate for innovation [65, 66] in relation to the clinical management of mTBI in general; and 2) individual level constructs – clinicians’ self-reported cognitions in relation to each clinical practice (such as the extent to which the clinicians feel confident in screening for PTA using a validated tool (beliefs about capabilities), whether a clinician believes these clinical practices will lead to favourable outcomes (beliefs about consequences) [67], and whether a clinician intends to perform the practices) [68]. For our primary outcome, individual level predictors along the causal pathway will be assessed (see Table 4). We have decided to only include these for the primary outcome, and not other outcomes for two reasons: 1) because the primary outcome is the best measure of clinical practice from the medical records which can be measured for all patients; and 2) to reduce the responder burden for clinicians.

Table 4 Predictors of clinical practices
Full size table

Proxy measures of clinical practice and predicting factors are measured using staff questionnaires. The NET team will prepare the questionnaire packs (including invitation letters, explanatory statements and baseline questionnaires) and send these to a local survey coordinator, who will be tasked with the distribution of the packs to their randomly selected staff members (for example, via mail or pigeon hole). Non-responders will receive reminders verbally or via email, and will be sent a reminder pack [69]. Questionnaires will be available upon request, and from the website (http://www.netprogram.org.au/).

Figure 2 depicts our conceptual framework, informed by Frambach and colleagues [70], Greenhalgh and colleagues [71] and Michie and colleagues [67]. It shows the specific factors our intervention will target, listed within categories (in bold).

Figure 2
figure 2

Conceptual framework. CT, CT scan-clinical criteria; HRQoL, health-related quality of life; INFO, provision of patient information; mTBI, mild traumatic brain injury; PTA, appropriate PTA screening.

Full size image
Patient and cost outcomes

Table 5 outlines the patient and cost outcomes that will be included in the study. We hypothesise that provision of patient information containing advice, education, and reassurance upon discharge from the ED reduces anxiety and number of self-reported symptoms [40]. In addition, we will investigate whether this leads to improved post-accident functioning (return to normal activities including work and health-related quality of life (HRQoL)) and fewer adverse events (for example, re-presentations).

Table 5 Patient and cost outcomes
Full size table

Patient outcomes in the NET-Plus hospitals will be collected via a computer-assisted telephone interview by a trained researcher, following consent received via a local ED staff member. Anxiety will be measured using the relevant questions in the Hospital Anxiety and Depression Scale [72, 73]. Post-concussive symptoms will be measured using the 13-item Rivermead [74]. HRQoL will be measured using a 12-item short form health survey (SF-12) and SF-12-based SF6D [75, 76]. The SF6D’s six domains - physical functioning, role participation, social functioning, bodily pain, mental health, and vitality - encompass variation in patient outcomes with respect to anxiety, interference with normal work and social function due to sleep problems, and physical functioning. Australian population norms for the SF-36-based SF6D are now available [77] and the SF6D has previously been used as a measure of HRQoL in TBI [78].

Data quality assurance

Chart auditors will be trained to identify eligible patients in the medical record system, and to log data into the secure web-based system in participating hospitals. Auditors will assess the same pilot records, and training will continue until sufficient consistency is achieved. Auditors will receive a data collection manual with instructions (such as definitions and data dictionary). The web-based database will be designed to minimise errors (for example, inability to leave particular fields open, or warning when answer is outside expected range). Double data entry will be used for all paper-based questionnaires and returned questionnaires will be checked for consistent errors or missing data.

Sample size

The primary outcome in the NET trial is appropriate PTA screening. In the calculation of sample size for this outcome, adjustment needs to be made for the clustered nature of the design. The variance inflation factor, or design effect, used to achieve this is a function of the average cluster size, variation in cluster size, and the intra-cluster correlation (ICC) [79].

The assumed ICC was calculated from two sources reporting ICCs from various datasets [80, 81]. We included only ICCs for process measures in secondary care settings since empirical research has demonstrated that ICCs tend to be larger for 1) process measures compared with patient outcomes (median 0.063 versus 0.030, respectively), and 2) secondary care compared with primary care (median 0.061 versus 0.045) [63]. Thirty three ICC estimates for process measures in secondary care were available, and the median ICC was 0.18. The variation in cluster size was incorporated into the sample size calculation through an estimate of the coefficient of variation, defined as the ratio of the standard deviation of the cluster sizes to the mean cluster size [79]. The estimated coefficient of variation of 0.47 was calculated from annual attendance at EDs across 170 hospitals in Australia in 2009.

There were limited studies providing estimates of rates of compliance for our primary outcome [82], and no studies undertaken in adult populations. We therefore undertook a retrospective audit of two Australian metropolitan EDs. The estimated rates of appropriate PTA screening for patients presenting in these hospitals (with an initial GCS of 14 or 15) were 0% (95% CI 0% to 14%; n = 24) in one hospital, and 31% (95% CI 24% to 39%; n = 164) for a second (which had an mTBI protocol in place) (Bosch M, McKenzie J, unpublished observations). We expected that hospitals without a protocol for mTBI would be more interested in participating in the trial, and therefore calculated our sample size based on a control group rate of 10% (that is, closer to the estimated rate in the hospital with no protocol in place). We wish to detect an absolute increase in the rate of appropriate PTA screening of at least 20%. Our rationale for selecting a difference of 20% is based on considering the size of effect required to justify a resource intensive intervention such as NET, and estimates of the effects of local opinion leaders on professional practice observed across a range of studies [83].

To detect an absolute increase of 20% in the rate of appropriate PTA screening (equivalent to an odds ratio of 3.9, log odds 1.3) (assuming a control group rate of 10%, an ICC of 0.18, coefficient of variation = 0.47, an average of 30 patient participants per ED, and a 5% significance level) with 80% power, we will require 15 EDs per intervention group. A total of 30 EDs will provide 900 patient participants for whom ED staff management will be assessed. For 900 patient participants, the width of the 95% confidence interval for the observed difference in appropriate PTA screening rates between groups will be approximately ±14% (equivalent to a width of ±0.51 on the log odds scale). Allowing for 10% attrition in EDs, we plan to initially recruit 17 EDs per intervention group.

Power is dependent on the selected parameters, but in cluster trials is primarily determined by the chosen ICC and control group rate. Assuming all other parameters are as defined above, a lower control group rate (for example, 5%) will result in greater power (87%), while a higher control group rate (for example, 30%) would result in less power (60%). A smaller ICC (for example, 0.10) would result in greater power (94%), while a larger ICC (for example, 0.30) would result in less power (59%).

Sample size calculations were undertaken using the module clustersampsi[84] implemented in the statistical package Stata (StataCorp LP, USA) [85].

Effectiveness analysis

The effectiveness of the intervention on management outcomes, predictors of management outcomes, and patient outcomes will be estimated with marginal modelling using generalised estimating equations. These models will appropriately account for correlation of responses of individuals within EDs. We plan to fit an exchangeable correlation structure where responses from the same ED are assumed to be equally correlated [86]. We will use robust variance estimation which yields valid standard errors even if the within-cluster correlation has been incorrectly specified [87, 88]. For binary outcomes, a logit link will be used.

Models will include adjustment for minimisation factors (size (annual presentation rate in 2012), existence of an mTBI protocol, NET or NET-Plus, and rurality) and pre-specified potential confounding variables (Figure 3). The potential confounding variables have been selected through discussion amongst the investigators and from published research. All pre-specified confounders will be included in the models even when no baseline imbalance exists. In addition, for continuous outcomes which are collected at both baseline and follow-up (for example, clinicians’ self-reported cognitions, social and environmental influences in relation to the key behaviours, cognitions about their organisation and team climate) we will include the baseline measure of the outcome in the model. Adjustment for the baseline measure of a continuous outcome yields unbiased estimates of intervention effect in circumstances where there is baseline imbalance, and has the benefit of providing the most powerful analysis [89]. Our primary effectiveness analysis will be the model (as described above) that estimates the intervention effect on the primary outcome, appropriate PTA screening.

Figure 3
figure 3

Confounders. mTBI, mild traumatic brain injury; NET, Neurotrauma Evidence Translation.

Full size image

Estimates of intervention effect from these models with binary outcomes will yield odds ratios. To aid interpretability, we plan to also provide estimates of risk differences [90]. Estimates of risk differences will be computed from marginal probabilities estimated from the fitted logistic models [91].

Missing data for the clinical practice outcomes are likely to be minimal since eligible patients will be identified and data extracted through chart audit. Missing data could arise if EDs choose to withdraw post-randomisation but prior to data being extracted from clinical records, resulting in empty clusters. Accounting for empty clusters requires strong assumptions to be made about patient characteristics and outcomes based on ED staff or ED characteristics. We will attempt to examine the potential impact of empty clusters on the intervention effects for the clinical practice outcomes using weights to allow for patterns of ‘missingness’ [92]. For continuous outcomes, generalised estimating equations yield unbiased estimates of intervention effect when 1) data are missing completely at random, or 2) when the correlation structure is correctly specified and known covariates that are associated with the missing data mechanism are included in the model (missing at random) [93, 94]. For the continuous proxy measures of clinical practice and hypothesised mediators, we plan to identify potential predictors of missing data through modelling (for example, [95]), and include these predictors in the primary analysis model. We will investigate methods to impute missing outcome data collected at baseline (for example, baseline behavioural constructs) [96].

All estimates of intervention effect will be reported with 95% confidence intervals. No adjustment will be made for multiple testing. All tests will be two-sided and carried out at the 5% level of significance. A full statistical analysis plan will be developed and written prior to undertaking any data analysis.

Potential confounders at patient level are collected via chart audit. Potential confounders at the level of staff members, as well as organisational history of change [97], organisational learning [98], leadership [99], staff culture [99] and team tenure [100] will be collected via self-report surveys. ED structural characteristics will be measured via a telephone checklist completed by the ED Director (or delegate).

Process evaluation

Our theory-based evaluation of factors along the proposed causal pathway (see Table 4 and Figure 2) will be complemented by other components that form part of our process evaluation [101, 102]; these are: 1) an assessment of intervention fidelity (whether the intervention was successfully and consistently delivered as planned (delivery), and whether it reached the target group (receipt)) [103, 104]; 2) perceptions of success of delivery and receipt [105], and factors that contributed to this [103]; and 3) perceptions of acceptability and feasibility of the roll-out of the intervention [106].

Table 6 summarises the main elements included in the evaluation, methods planned, and indicative measures. A more detailed description of the fidelity assessment will be provided in a future publication.

Table 6 Overview of mixed-methods process evaluation
Full size table

We plan to conduct semi-structured interviews with ED Directors, local opinion leaders and staff members in eight intervention hospitals. We will sample for maximum variation (for example, size and location of EDs). The interview guide and analyses will be guided by a theory-based framework that was used in phase two of this project (the interview phase). Analyses and interviewing will be done concurrently to inform emerging themes. If data saturation has not been reached after conducting interviews in eight hospitals, we will add sites and continue sampling until data saturation [107].

In addition, a brief checklist-guided structured interview will be conducted by telephone with ED Directors or delegates to assess sustainability of the changes in intervention hospitals (for example, plans to integrate a validated tool into charts).

Ethical review and informed consent

Pre-recruitment ethical approval of this trial protocol has been granted by Alfred Health Human Research Ethics Committee (approval Number 398/12). When participating hospitals have been identified, local ethics and research governance procedures will be completed. This will include a “letter of understanding”, which contains information around what the trial entails, what the expectations are from hospitals and researchers, and the process to authorise the independent chart auditor to undertake the data extraction from medical records. The document will be signed following recruitment, and prior to randomisation. It will still be possible for EDs to withdraw from the study at any time should they wish to do so.

Consent procedures will be in line with recent guidance regarding ethical issues in CRTs [54, 108], and include the following.

For all hospitals

ED Directors will be asked to consent to: 1) ED trial randomisation (NET or NET-Plus); 2) delivery of the intervention to their ED; 3) permission to approach staff members for data collection purposes; 4) extraction of medical record information by the chart auditor for eligible patients over a 2 month period; and 5) permission to use this data for the current study and potentially future linked studies, with the understanding that all identifiable information will be removed, unless participants stated otherwise.

For NET-Plus hospitals

For the hospitals that opt to participate in the NET-Plus component, ED Directors are requested to, in addition to the points in the preceding paragraph, also consent to patients being contacted by telephone by an ED staff member post discharge to: 1) ask consent to participate in the telephone follow-up interview that will be performed by a trained researcher with experience in follow-up in this patient group; and 2) to allow the NET researchers to receive their contact details.

Staff members, intervention delivery

In cluster trials, it is impractical/impossible for researchers to obtain informed consent from every individual cluster member to receive the educational intervention before randomisation; therefore, we applied for a waiver for intervention delivery to staff members. To ensure the decision is in the cluster’s interest, ED directors will be asked to consult their staff members prior to their decision to participate in this study [54].

Staff members, data collection

For staff members selected to receive written questionnaires, we will use the implied consent principle (that is when we receive a completed questionnaire this will constitute informed consent). This will be clearly explained in the introduction to the questionnaires. Return of a blank questionnaire will be considered as refusal to participate. Staff members invited to the process evaluation interview over the telephone will be identified by the local opinion leaders, receive an invitation letter and opt in by returning a consent form to the researchers.

Patients

NET component

The NET component of the trial does not recruit patients. Data extraction from patient files will be performed by a trained chart auditor with clinical experience, who is familiar with several medical records systems, and who will sign a confidentiality agreement. This person will be authorised by each hospital to do the chart audit, and hence can be considered part of the circle of care at that hospital. We have sought a waiver for consenting patients to this part of the study for the following reasons. First, the chart auditor will identify the patients retrospectively. At this time they will confirm eligibility, and, for feasibilities reasons, also collect the outcome data. If consent from patients had been sought following confirmation of eligibility, this would have required substantial additional resources. In addition, this process may have introduced selection bias, whereby those patients choosing to participate may have differed from patients who declined. Furthermore, only routinely collected information is extracted, and the central research team will only have access to non-identifiable information, which maintains the privacy of the patients. Finally, the intervention is not being delivered at the level of the patient; patients are only affected indirectly by the study intervention and there is limited risk of the intervention affecting their interests adversely [54, 109].

NET-Plus component

In NET-Plus hospitals, patients included in the chart audit who meet the initial inclusion criteria for telephone follow-up (see section: recruitment of patients for follow-up) are contacted by telephone by an ED staff member post discharge to: 1) ask consent to participate in the telephone follow-up interview that will be performed by a trained researcher with experience in follow-up in this patient group; and 2) to allow the NET researchers to receive their contact details. The NET team will then send explanatory letters to the patients who consented to the above, to provide them with the information regarding the follow-up interview. This letter explains participation in the interview is entirely voluntary. At the start of the actual interview, the patients will again be given the opportunity to opt out of the study, should they wish to do so.

Confidentiality of data

Confidentiality of data will be ensured via the following processes. All information collected will be entered via a web-interface into a secure server database. Each user will have a unique username and will be assigned a role that will provide them with access to only the information they need to see. The database administrator will have access to the full data, and will sign a privacy statement acknowledging responsibilities and restrictions on handling identifiable information according to ethics guidelines.

Patient data, NET component

Independent chart auditors who have been authorised by the hospitals to collect data will be able to see and log identifiable information (for example, patient numbers and names) in the database. For data quality purposes, patient data need to be re-identifiable. However, this information will not be visible to study researchers who will only be able to see “study participant numbers” (a unique number automatically generated by our database system) and non-identifiable information.

Patient data, NET-Plus component

Only patients who provide full verbal consent to the ED staff member will be identifiable from information in the database by the study researchers. Identification of patients for NET-Plus is necessary for completion of the patient interview.

Staff data

Survey packs (including a participant identification number) will be prepared by NET staff, and will be distributed by a local ED staff member (survey coordinator) in each hospital. For follow-up purposes, data need to be re-identifiable. Therefore, the survey coordinator will have access to names as well as study participant numbers. Completed surveys will be returned directly to the researchers and the data will be entered into the database. Therefore, the survey coordinator will not have access to the data. Researchers will have access to the data and study participant numbers, but not identifiable information.

Economic evaluation

A number of previous studies have estimated the costs and benefits associated with adherence to various diagnostic management strategies for mTBI [110–112]. Stein and colleagues [112] conducted a modelled cost-utility analysis comparing selective CT scanning based on the CCTHR, CT for all patients, skull radiography for all patients, prolonged ED observation, 24-hour hospital admission, and no treatment. Smits and colleagues [111] conducted a modelled cost-utility analysis based on data from the Computed Tomography in Patients with Minor Head Injury (CHIP) trial [26] (comparing selective CT as per the New Orleans Criteria, CCTHR, and CT in Head Injury Patients (CHIP) criteria) and CT for no patients. Holmes and colleagues [110] conducted a modelled cost-utility analysis comparing selective CT scanning (as per National Institute for Health and Care Excellence (NICE) guidelines, the CCTHR and various other guidelines), CT for all patients, and discharge for all patients without testing. Findings from each of these studies suggest that – at the mean – selective CT scanning is cost-effective in comparison to other strategies. However, in interpreting these findings, Holmes and colleagues [110] noted that “… the cost of CT scanning is very small compared to the estimated cost of caring for patients with brain injury worsened by delayed treatment” (page 1,423) and concluded that “… all hospitals receiving patients with minor head injury should have unrestricted access to CT scanning for use in conjunction with evidence based guidelines” (page 1,423). Comparisons between alternative criteria for selective CT suggested that they offer “broadly similar costs and quality-adjusted life years” [110] (page 1,428), though adherence to the medium risk CCTHR had the highest probability of being the most cost-effective strategy in the study conducted by Holmes and colleagues [110]. Results reported by Stein and colleagues [112] and Smits and colleagues [111] are broadly consistent with these findings.

The costs and benefits associated with adherence to various diagnostic management strategies are clearly relevant to the question: what is the cost-effectiveness of an implementation intervention to increase adherence to key recommendations regarding diagnostic management? However, addressing this question entails a number of complications that were not addressed in the studies by Stein and colleagues [112], Smits and colleagues [111] and Holmes and colleagues [110]. First, the potential for cost-effective implementation is predicated on the existence of an evidence-practice gap (see Table 1). Previously published studies by Stein and colleagues [112], Smits and colleagues [111] and Holmes and colleagues [110] set aside the difficulties associated with closing the gap between evidence-based recommendations and clinical practice. Statements of cost-effectiveness taken from these previous studies are therefore conditional upon the assumption of perfect adherence to each of the evaluated diagnostic management strategies [27]. Second, implementation typically entails an additional investment that may or may not be offset by any health gains and/or reductions in health service utilisation derived from increased adherence to evidence-based recommendations. The economic evaluation described here will be the first to consider the trade-off between the hypothesised increase in adherence due to implementation of key clinical EBCPG recommendations regarding the ED management of adult patients with mTBI, and the additional costs (savings) arising from the implementation intervention.

Specifically, economic evaluations alongside NET and NET-Plus will be conducted with the aim of quantifying additional costs (savings) and health gains arising from delivery of the NET implementation intervention in adult patients (18 years of age or older) with mTBI in Australia, compared with passive dissemination of the recommendations/EBCPG. Evaluation of costs and health gains arising from delivery of the intervention (ex post of development of the implementation intervention) will be informative to policy-makers and hospital administrators considering a wider roll-out of the NET implementation intervention. Secondary aims will be to determine whether the incremental treatment costs of the NET intervention are offset by reductions in health service expenditure (that is, whether implementation is cost-saving as compared with existing practice), and to determine whether the NET intervention dominates existing practice (that is, less costly but no less effective). The time horizons for inclusion of relevant costs and consequences for the trial-based evaluations described here coincide with the final scheduled follow-up of participants in NET (2 months post-intervention) and NET-Plus (3 to 5 months post-discharge for patients treated in the 2 months post-intervention). The economic evaluation alongside NET and NET-Plus will take a health sector perspective in identifying, measuring, and valuing costs and consequences within the time horizon for each component.

Additional methods for the economic evaluation including methods for the identification, measurement and valuation of outcomes and resource use are described in Additional file 2. Results from the economic evaluation alongside NET will be expressed as additional costs (savings) per patient appropriately screened for PTA, per patient who received patient information upon discharge home, and per patient safely discharged. Results from the economic evaluation alongside NET-Plus will be expressed as additional costs (savings) per point difference on anxiety questions of Hospital Anxiety and Depression Scale at 3 to 5 months post-discharge, additional costs (savings) per point difference on the Rivermead Post-concussive symptoms checklist, and additional costs (savings) per point difference in SF6D utility index scores.

Discussion

The cluster trial described in this protocol aims to evaluate the implementation of a targeted theory- and evidence-informed intervention to improve key evidence-based recommended practices for the management of mTBI in Australian EDs. To our knowledge this is the first trial to evaluate this suite of key recommendations. It addresses calls to use and test theory-driven models of change from a range of scientific disciplines to enhance knowledge translation efforts in ED settings [113]. More broadly, we hope this protocol may assist those who are undertaking quality improvement studies in emergency care settings.

Trial status

At the time of submission of this manuscript, recruitment of sites had been completed, and collection of baseline data had been started; however, data cleaning or analysis has not commenced. The trial was registered in the Australian New Zealand Clinical Trials Registry on 12 December 2012 (ACTRN12612001286831).

Publication policy

The results from the trial will be published regardless of the outcome. Reporting of this trial will adhere to the relevant, and most up-to-date, CONSORT (Consolidated Standards of Reporting Trials) statement [90] and its relevant extensions [114–116].