Background

Dissemination of best practice, usually via clinical guidelines, is rarely sufficient by itself to implement effective or de-implement ineffective or harmful treatments [1, 2]. Observed variations in adherence to clinical recommendations are often poorly explained by routinely available patient or practice variables and are likely to be attributable to differences in clinical or organisational behaviour [3, 4]. There is generally a need for active implementation strategies to translate evidence into routine care [5]. Implementation strategies have important but variable effects (absolute effect sizes range from 3 to 16% [1, 6,7,8,9,10,11]). Research resources are potentially wasted on randomised trials of implementation strategies that do not advance understanding of what makes such strategies likely to be more or less effective [12,13,14,15]. Attempts to improve this understanding are often hampered by insufficient description of intervention content [10, 11, 16, 17]. Accurate descriptions are needed to interpret heterogeneity of effects within systematic reviews, inform replication of promising features of implementation strategies and guide the exclusion of less-effective features.

Many implementation studies focus on one clinical behaviour or condition [18, 19]; it is uncertain whether an implementation strategy developed for one problem will work for another. This is particularly problematic for settings such as primary care where clinicians need to manage a wide variety of conditions. It is impracticable and inefficient to develop and evaluate an implementation strategy for every clinical guideline. Adaptable strategies are required, which can potentially be generalised across different quality indicators and sustainably integrated within existing resources.

Targeting implementation strategies according to determinants of adherence (also known as barriers, enablers, or facilitators that influence or affect behaviour) may improve their effectiveness [16, 20, 21]. The Theoretical Domains Framework (TDF) offers a structured approach for exploring the perceptions of those whom are targets for the intervention [12, 22,23,24], whilst the Behaviour Change Taxonomy outlines 16 categories of 93 specific, theoretically informed or evidence-based behaviour change techniques (BCTs) that are hypothesised to change behaviour [23, 25]. BCTs are observable, replicable and irreducible ‘active ingredients’ that offer a common language with which to describe intervention content. There is a lack of guidance on how best to operationalise, tailor content and combine BCTs to enhance effectiveness. For example, motivation may be increased by fear of negative consequences, but excessive fear may inhibit action [26].

We previously screened the National Institute for Health and Care Excellence (NICE) guidelines and associated quality standards to identify 2365 guideline recommendations considered relevant to UK primary care [27]. Following a cross-sectional analysis of patient data [4], we derived four ‘high-impact’ quality indicators based on criteria including as follows: the burden of illness, the potential for significant patient benefit from improved practice, the likelihood of cost savings without patient harm and the feasibility of measuring change using routinely collected data. The four quality indicators comprised the following: diabetes control (achieving all recommended target levels of haemoglobin A1c (HbA1c), blood pressure (BP) and cholesterol in patients with type 2 diabetes); risky prescribing, largely focusing on avoiding adverse gastrointestinal, renal and cardiac effects of non-steroidal anti-inflammatory drugs (NSAIDs) and anti-platelet drugs; BP control in patients at high cardiovascular disease risk; and anticoagulation for stroke prevention in patients with atrial fibrillation (AF) [28].

We aimed to systematically develop an implementation package and adapt it for the four quality indicators. Whilst intervention descriptions have previously been coded for the presence of BCTs [25, 29], and the delivery of intended BCTs [30], we have not yet encountered any published studies prospectively evaluating intervention content before evaluation. Prospective coding reduces the likelihood of post hoc rationalisation whereby intervention descriptions are influenced by knowledge of evaluation outcomes [31]. In this paper, we will address two research questions. First, were the BCTs we intended to operationalise identified by an external coder during a directed content analysis? Second, which verified BCTs were shared or unique to the implementation package adapted for four quality indicators?

Methods

We used an overlapping multi-staged approach adopting an interpretivist stance to design and verify the content of an implementation package adaptable for four quality indicators using the BCT taxonomy (Fig. 1 & Table 1). All research was undertaken in West Yorkshire, England, which covers a socioeconomically diverse population of 2.2 million residents [32]. Approximately 300 general practices are organised within 10 clinical commissioning groups (CCGs). Demographically, they are broadly typical of the national average, with the exception of higher deprivation levels [33].

  • Stage 1: selecting delivery mechanisms. We aimed to create an adaptable implementation package based upon resources typically available within primary care. Commonly used delivery mechanisms known to be effective [1] were selected by the intervention development team: audit and feedback [8], educational outreach [11] and computerised prompts and reminders [10]. We aimed to embed evidence-based features known to increase their impact, e.g. repeated feedback of audit data, the requirement for users to select a reason for over-riding a computerised prompt [8, 10].

  • Stage 2: identifying candidate BCTs. We aimed to enhance selected delivery mechanism effects by embedding BCTs (e.g. ‘feedback on behaviour’ or ‘action planning’ [23]). Team members with experience of applying behavioural theories to implementation strategies (LG, RL, RM and RF) independently mapped the 12 determinants from the TDF [34] to one or more of the 16 broad BCT categories and then to individual BCTs using an electronic spreadsheet. Results were collated and any BCT category nominated by three or more researchers was considered eligible. The team discussed discrepancies until consensus was agreed. We aimed to generate an inclusive list of ‘candidate’ change techniques. A matrix was produced to indicate BCTs with the potential to target one or more theoretical determinants.

  • Stage 3: identifying and prioritising relevant theoretical determinants of behaviour. We have earlier described the methods and findings of interviews with primary care clinicians and managers to explore determinants of adherence [35]. Given the timeframe required to analyse and compare four substantial sets of interview data, emerging interview findings (themes and illustrative quotes) based on interviewer field notes and exploration of frequency data were compiled for each indicator. We convened a series of multi-disciplinary stakeholder panel meetings, one for each quality indicator. The intervention development team used their previous knowledge to identify stakeholders involved in achieving each quality indicator. We invited five to ten stakeholders representing clinicians (general practitioners, practice nurses, pharmacists), practice managers, quality improvement specialists and service commissioners. All invited stakeholders were willing to participate in the consensus process. We presented them with emerging interview analyses [35] (frequency data and illustrative quotes for each determinant of achievement). After reviewing the range of determinants, stakeholders were asked to suggest additional professional or organisational determinants and contextualise our findings. Candidate BCTs (identified during the mapping exercise in stage 2) and messages which could be framed were reviewed for potential fit within the organisational context of primary care and feasibility of operationalisation within the different delivery mechanisms. Evidence-based intervention features were discussed to explore their acceptability prior to implementation. Field notes were used to record discussion points. We simultaneously convened a parallel group of nine patient and public representatives and followed similar methods. The research team communicated key messages from one panel to another. Suggestions from both groups were reviewed by the intervention development team, including social scientists and clinicians, to maximise acceptability and feasibility.

Fig. 1
figure 1

Multi-staged approach to develop and content analyse BCT content of implementation package. Multi-staged approach to develop and content analyse an implementation package with embedded BCTs adapted for four quality indicators

Full size image
Table 1 Standards for Reporting Qualitative Research (SRQR)
Full size table

Following the stakeholder panels, we further analysed interview findings using the TDF [34, 36] to identify the most prominent determinants and high-level themes. Determinants were grouped into four categories: core, prominent, less evident and not identified. Determinants considered core to all four quality indicators (i.e. consistently raised regardless of quality indicator) were ‘social and professional role’ and ‘environmental context and resources’; those considered prominent (i.e. determinants which varied in importance) were ‘beliefs about consequences’, ‘social influences’, ‘knowledge’ and ‘memory, attention and decision processes’; and those considered less evident were ‘skills’, ‘beliefs about capabilities’ and ‘motivation and goals’. Determinants ‘emotion’ and ‘behavioural regulation’ were not identified. These data informed the content of follow-up feedback reports.

  • Stage 4: designing intervention content. We created a prototype outline for each delivery mechanism (feedback report, educational outreach session and computerised prompts and reminders) including features known to enhance effectiveness [10, 16] and findings from stages 1 to 3. Computerised prompts were not developed for diabetes or blood pressure control because they were already widely used. Stakeholders also suggested patient-directed checklists to guide discussions around diabetes and blood pressure control respectively. We then embedded candidate BCTs (stage 2) that could target modifiable determinants of adherence (stage 3). Using contextual data from interviews, the prototype was adapted and tailored for each quality indicator. We used the vocabulary and experiences expressed in interviews with health care professionals and stakeholder panellists to tailor BCT content within delivery mechanisms. A graphic designer enhanced the final intervention template. All prototypes were reviewed by the intervention development group for feasibility and acceptability prior to piloting.

  • Stage 5: piloting intervention content and refinement. We piloted each delivery mechanism for all quality indicators with five consenting general practices involved in our earlier interview study [35]. Brief opportunistic semi-structured interviews were conducted by EI a social scientist researcher, previously unknown to the practice. She directly observed the delivery of each educational outreach session. Feedback reports, patient-directed checklists and protocols for computerised prompts were presented as written documents. She conducted brief, opportunistic semi-structured interviews with relevant practice staff (six GPs, two practice managers and three practice nurses). Participants commented on the acceptability and feasibility of prototype delivery mechanisms. Field notes were taken and reviewed by the intervention development team, leading to refinements of the first feedback report template. Participants were not asked to comment on the presence or absence of BCTs. The impact of BCTs on hypothesised determinants was not explored when piloting interventions.

  • Stage 6: verification of BCTs included within implementation packages. Three trained coders (LG, RL and KG-B) applied the BCT Taxonomy version 1 in two stages to identify intended and verified BCTs [23]. First, two members of the intervention development team (LG and RL) assessed which BCTs were included in a sample of delivery mechanisms (feedback reports, educational outreach visit plan for one quality indicator) to identify intended content. A directed content analysis exploring the consistency of identification was completed by a researcher external to the intervention development team (KG-B) [37]. All implementation package delivery mechanisms were coded to one or more BCTs independently by LG and KG-B. Discrepancies were noted and decisions made about sufficient evidence to verify inclusion. We then examined the extent to which BCT content was shared across (or unique to) the adapted implementation package and different delivery mechanisms.

Ethics

The ASPIRE programme was reviewed by the Leeds Central Research Ethics Committee in June 2012 (12/YH/0254) and National Health Service research governance permissions granted by the West Yorkshire Commissioning Support Unit.

Results

BCTs with the potential to target one or more theoretical determinants (Table 2) were identified in a matrix (Table 3). We identified 30 BCTs with potential to target determinants from our emerging interview study findings and stakeholder panellists (Additional file 1: Table S1 and Table 4). BCTs that could not be operationalised within our delivery mechanisms or fit within the context of primary care resources were considered ineligible (Table 5).

Table 2 Perceived determinants of adherence prioritised for intervention development
Full size table
Table 3 Candidate behaviour change techniques mapped to determinants of behaviour ordered by core, prominent and less-evident determinants identified during previous interview study [35]
Full size table
Table 4 Intervention description based on TIDIER [25]
Full size table
Table 5 Behaviour change techniques excluded from intervention development or intended but not subsequently identified during content analyses
Full size table

The implementation package adapted for four quality indicators

We tailored delivery mechanisms for each quality indicator: audit and feedback, educational outreach and computerised prompts and reminders. Full operational details of how each delivery mechanism was developed and intended to be delivered have been specified following TIDieR guidance [38] (Table 3). Audit and feedback comprised quarterly reports, computerised searches to identify patients and significant event audit templates to support root cause analyses (risky prescribing; anticoagulation for AF). Report content was designed to inform and prompt memory of clinical targets, highlight consequences of not changing behaviour, suggest potential strategies for change, give feedback on outcomes/behaviour, compare adherence with others, encourage goal setting and use of an action plan template and encourage reflection on progress towards goals. Educational outreach comprised an initial 30-min session to focus on action planning, increase motivation, discuss barriers to action, facilitate group reflection and increase confidence to act. Sessions were facilitated by pharmacists following 2 days training. A follow-up session was available to review goal achievement and create more challenging or attainable plans. Two days of pharmacist support were offered to support patient identification and review. Computerised prompts and reminders were created to reinforce clinical messages of who and what to target for each indicator. Nine computerised prompts for risky prescribing and two for anticoagulation for AF requiring a one-click justification (e.g. continue with risk, add medication, or stop medication) were developed. To avoid duplication with existing quality improvement systems, we did not develop computerised prompts for diabetes control or BP control [39]. Laminated reminders were created to convey key clinical information for BP control, anticoagulation for AF and risky prescribing. Patient-directed checklists to facilitate shared decision making were developed for distribution by practice staff in BP control and diabetes control practices.

Identification of BCTs included within adapted implementation packages

The directed content analysis identified the BCTs embedded within the adapted implementation packages for four quality indicators. Each implementation package included at least 27 out of 30 potentially applicable BCTs (Additional file 1: Table S1), representing 15 of 16 BCT categories across the range of intervention delivery mechanisms. Each package contained multiple unique instances of the different BCTs. Four BCTs that were intended for inclusion (‘identification of self as a role model’ and ‘verbal persuasion about capability’ in educational outreach, ‘discrepancy between current behaviour and goal’ in feedback reports, and ‘anticipated regret’ in feedback reports and educational outreach) could not be verified.

Extent of shared and unique BCT content across adapted implementation packages and delivery mechanisms

Adapted implementation packages for the different quality indicators shared common and unique BCT content. Twenty-three BCTs were shared across all four quality indicators (Additional file 1: Table S1). Twenty-seven BCTs were identified in strategies targeting risky prescribing and BP control, and 30 targeting anticoagulation for AF and diabetes control. Seven BCTs were unique to implementation packages largely focused on changing processes of care (risky prescribing and anticoagulation for AF contained BCTs relating to ‘goal setting for behaviour’ and ‘monitoring of behaviour’) and five BCTs were unique to packages targeting patient outcomes (BP control and diabetes control contained BCTs relating to ‘goal setting for outcomes’ and ‘monitoring for outcomes’). We did not operationalise ‘goal setting for behaviour’ or ‘monitoring of behaviours’ for the quality indicators relating to BP and diabetes control that focussed on outcomes of behaviour. This was only identified during the verification exercise once interventions had been delivered to participants.

It was possible to operationalise and identify intended BCTs within audit and feedback more frequently than for the other delivery mechanisms. Feedback reports for diabetes control and anticoagulation for AF contained 29 BCTs, compared to 28 for BP control and 26 for risky prescribing. Diabetes reports contained BCTs relating to ‘habit formation’, ‘reviewing outcome goals’ and suggestions for ‘restructuring the social environment’. Educational outreach for BP control and anticoagulation for AF contained 17 BCTs. Diabetes control and risky prescribing outreach contained 16 BCTs. ‘Goal setting outcome’ was not operationalised for risky prescribing nor anticoagulation for AF. ‘Information about social/environmental consequences’ was not identified for diabetes control. ‘Behavioural contract’ and ‘commitment’ were only designed to be included in significant event audit templates developed for risky prescribing and anticoagulation for AF and no other delivery mechanism. Computerised prompts were limited in the number of BCTs they could include because of their brevity but included additional BCTs not included in other mechanisms, (‘prompts/cues’, ‘pros/cons’, ‘adding objects to the environment’, ‘conserve mental resources’ and ‘credible source’). Paper-based reminders included ‘prompts and cues’ and ‘conserving mental resources’.

Our extended analysis of interview findings using the TDF identified ‘environmental context’ and ‘social influences’ as core determinants across all indicators. Antecedent BCTs to target these determinants were particularly challenging to include in our chosen delivery mechanisms (i.e. we prospectively excluded the BCTs ‘distraction’, ‘body changes’ and ‘avoidance’). ‘Adding objects to the environment’ was only operationalised in the computerised prompts developed for risky prescribing and anticoagulation for AF. Advice on ‘restructuring the physical environment’ and ‘restructuring the social environment’ were only included in BP control and diabetes control feedback reports respectively.

Discussion

We have provided a detailed description of the core BCTs that can be identified across an adaptable implementation package and those unique to a range of delivery mechanisms commonly available to the UK primary care setting. Whilst published intervention descriptions have previously been reliably coded for the presence of BCTs [25, 29] and the delivery of intended BCTs [30], we believe that intervention content has not previously been coded prospectively by someone external to the development team prior to evaluation.

In our prospective assessment of the extent to which BCTs were verified, we identified a large proportion of shared BCTs (at least 23 of 30 eligible BCTs) representing 15 of 16 BCT categories, suggesting that prioritised BCTs can be embedded and identified across delivery mechanisms adapted for different quality indicators. Whilst educational outreach appeared to contain fewer BCTs, it was designed to be co-delivered with feedback reports. It was possible to include a greater range of BCTs in educational outreach because of the face-to-face delivery of outreach (e.g. ‘graded tasks’) and the co-delivery of audit and feedback reports. We did not verify if BCTs targeted intended determinants of behaviour nor could we verify the presence of four BCTs, implying their operationalisation was unsuccessful or that these BCTS are less visible.

Previous research retrospectively identifying BCTs within a systematic review of interventions for diabetes care targeting healthcare professionals [40] identified seven BCTs. We identified a broader range of included BCTs, at least 27 in each of our four adapted implementation packages, incorporating many of those not identified in the systematic review of diabetes interventions. This could be because we used a prospective identification process, or because we attempted to target a wider range of behavioural determinants identified from our interview and stakeholder panel findings.

Limitations

We had to make trade-offs between what is theoretically desirable, clinically acceptable and operationally feasible within the context of delivery mechanisms and primary care resources. These trade-offs are partly reflected in our main study limitations.

Firstly, there were limitations in how we assessed and prioritised determinants of behaviour and subsequently linked them to BCTs. We acknowledge that perceptions of determinants are not necessarily predictors of behaviour [41]. We did not quantify the importance of each belief within determinants nor the relative importance of each determinant to the target population [42]. Therefore, it was not possible to identify individual psychological theories and use their corresponding evidence-based measures and instruments to develop each implementation package [43, 44]. We used emerging and extended interview findings to inform intervention development. It was not possible within our research timelines to use the extended findings to inform adaptation of educational outreach or initial feedback reports. We may not have adequately operationalised BCTs to target core and prominent determinants (‘social and professional role’ and ‘environmental context and resources’) within the following categories: ‘social support’, ‘antecedents’, ‘identity’ and ‘covert learning’ to target the determinants.

Secondly, BCTs from social cognition models and the TDF more generally focus on individual cognitions and may be insufficient to adequately target team, patient or organisational determinants.

Thirdly, although we aimed to develop an implementation package which could be adapted to target four (and potentially other) quality indicators, whether or not it is consistently effective is an empirical question. Whilst determinants of practice may only be relevant to countries with a comparable primary care organisation, methods to identify candidate BCTs and verify their presence are transferable.

Fourthly, despite the fact that trained and experienced coders conducted the directed content analyses, they may have omitted less-visible BCTs or over-coded BCT content. To minimise bias, a coder external to the intervention team conducted the directed content analysis. Whilst a single researcher performed the prospective content analysis, they had undertaken the BCT Taxonomy online training and a number of quality assurance steps (e.g. staged review of documents, meeting to discuss discrepancies). As coding was only conducted by two coders at each stage, it was considered inappropriate to statistically assess the reliability of this exercise, and more informative to identify discrepancies and discuss why these occurred and what could be done in subsequent research to improve the operationalisation of these techniques during intervention development. Coders will vary in the knowledge, skills and experience that they bring and may vary in their judgements of the presence/absence of BCTs.

Fifthly, the independent coder could not identify four BCTs that we intended to embed: ‘identification of self as role model’, ‘imaginary punishment’, ‘anticipated regret’ and ‘discrepancy between current behaviour and goal’). Coding discrepancies suggested that there is scope to either improve the operationalisation of some BCTs or the training provided prior to coding for BCTs.

Lastly, because of our research timelines, it was not feasible to draw out the high-level themes from the interview analyses or to modify interventions to address unsuccessful operationalisation as interventions were coded following dissemination of the fifth and final audit report. Adapting intervention content to include evidence-based findings in multiple domains (e.g. behavioural science and features known to enhance the effect size of implementation interventions) is challenging to deliver within research budgets and timescales. Future work could benefit from building in more time for iterative cycles to refine interventions and assessing the consistency of verification by coders with different skills and experience.

Implications for research

We suggest that the methods and transparency of developing complex interventions to promote the implementation of evidence-based practice would be advanced by reporting the following: candidate BCTs considered eligible following an analysis of key behavioural determinants; BCTs targeting key determinants that could not be included within chosen delivery mechanisms, within the constraints of project resources, or implementation context; and the degree of success in operationalising and verifying intended BCTs. It is important to identify and report potentially eligible techniques that could not be verified for the following reasons. Firstly, process evaluations could explore whether interventions do not work because BCTs intended to target the most important determinants could not be operationalised. Secondly, implementation researchers can consider how prioritised determinants and salient BCTs could be operationalised within alternative delivery mechanisms. Thirdly, there may be scope to add to the existing BCT taxonomy novel BCTs that can be operationalised within different delivery mechanisms.

During our research, Cane et al. mapped the 93 BCTs to theoretical determinants [45] building on the earlier work of Michie and colleagues [34]. We identified many of the same BCTs. In addition, Cane identified two categories of BCTs that we did not, feedback and monitoring, and antecedents were mapped to knowledge, and antecedents to skills. Whilst these were not included in our intervention development work, we did include shaping knowledge, natural consequences and comparison of behaviour and outcomes to target knowledge. Shaping knowledge, comparison of behaviour and repetition and substitution BCTs were included to target skills. Michie is currently leading work to advance this line of research, including the development of an ontology for linking BCTs to theoretical determinants [46].

More research is required to establish how to best operationalise antecedent BCTs to target the determinants ‘environmental context and resources’ and ‘social influences’ (both team and patient). Whilst BCTs to target organisational determinants are available, they were harder to operationalise because they involve addressing relatively complex systems and social processes. We were aware of the importance of patient factors but prioritised our effort and resources on changing professional behaviour. We also recognised that there is a much larger body of research on changing patient behaviour (e.g. for diabetes) that we could not address within the constraints of one research programme. We did partly try to address this shortcoming by developing patient-directed checklists but could not complete these to our satisfaction within our timelines.

We do not know if there is an additive, synergistic or negative effect of operationalising multiple BCTs within a BCT category. Whilst implementation packages contained one or more instances of the BCT, we did not quantify the ‘dose’ given. Further, we only assessed the presence of the BCT and not whether the BCT effectively targeted the most salient determinants of behaviour. Previous research identifying the BCT content of diabetes implementation strategies has suggested that there is strength in harnessing the potential of BCTs to change multiple behaviours [40].

Many BCTs originate from health psychology theories developed to understand why patients do not change their behaviour; changing or not changing may result in health consequences for those individuals. The applicability of this evidence base for professionals who do not personally experience health consequences needs to be further explored. Delivery mechanisms such as audit and feedback are potentially relatively efficient to deliver if drawing upon routinely collected data but they are unable to guarantee the delivery, uptake and engagement of BCTs that require more intensive and expensive delivery mechanisms such as educational outreach (e.g. ‘graded tasks’, ‘problem solving’, ‘action planning’ and ‘commitment’) which allows for tailoring and engagement at the practice level.

The effectiveness of our implementation packages is being rigorously evaluated in a pair of cluster randomised controlled trials [28]. A parallel process evaluation will determine what was actually delivered, received and acted upon. The analysis will draw upon both Normalisation Process Theory [47, 48] and the TDF [35, 45]. The TDF was applied to help compare planned versus actual intervention content. Normalisation Process Theory was adopted given its relevance to understanding implementation processes, particularly how individuals and groups conceive of, engage with, enact and reconfigure work in response to an intervention. Our ongoing trials and process evaluations [28] of these packages will not identify BCTs that act together or independently to enhance effectiveness and this should be explored in subsequent research. We invite others to build upon and improve our methods in reporting the BCT content of implementation interventions.

Conclusion

Implementation researchers need to identify effective and efficient means of selecting and adapting implementation strategies across a range of targeted quality indicators, rather than propose a new trial for every indicator (Table 5). We have demonstrated the specification and verification of BCT content for an adaptable implementation package targeting four quality indicators. We identified variable numbers of BCTs across the four adapted implementation packages and delivery mechanisms but would not claim that ‘more is better’; the ability to effectively target the most salient behavioural determinants is likely to be more important, although some delivery mechanisms may lend themselves better to adaptation than others. Whether or not an adapted implementation package is actually effective in targeting our four indicators, or others, is an empirical question.