Introduction

The problem of overuse of ineffective or harmful care which can lead to poor patient outcomes due to adverse events of treatments or unwarranted secondary tests and inefficient use of scarce healthcare resources threatening the sustainability of healthcare systems, has led to an increasing need to identify effective de-implementation interventions of low-value care [1,2,3]. For example, the BMJ’s Too Much Medicine campaign noted that identifying strategies to reduce unnecessary tests, diagnoses and treatments (i.e. de-implementation) will benefit patients by directly avoiding harm and helping create a more sustainable healthcare system [4, 5]. Projects such as Choosing Wisely have noted the importance of addressing de-implementation strategies to improve the care patients receive whilst eliminating wasteful spending [6,7,8,9]. Policy interest in de-implementation with programmes such as Choosing Wisely and Too Much Medicine campaigns advocate for more effective de-implementation interventions but remain unclear what the effective strategies should be.

Decreasing ineffective or harmful healthcare practices (de-implementation) may require different approaches than those used to promote uptake of new procedures (implementation) [10, 11]. However, there is little to no guidance on how to de-implement low-value or harmful healthcare practices or what types of interventions are better suited for de-implementation [2]. Investigation into the theoretical basis for designing implementation and de-implementation interventions differently has indicated that theories of behaviour change generally do not distinguish between implementation and de-implementation [12]. The one theory that does make the distinction (Operant Learning Theory [13]) may be difficult to apply in healthcare settings [12].

The term de-implementation is relatively new in the field of implementation research, having been discussed for only the last 8 years [10, 11, 14,15,16]. However, researchers have been designing implementation and de-implementation interventions for decades but rarely explicitly distinguished between them. It is unclear what approaches are being used and whether implementation and de-implementation interventions do require different strategies. It is thus important to describe the content of both implementation and de-implementation interventions and unpack their ‘active ingredients’ or behaviour change techniques to (1) determine what is actually included in both kinds of interventions and (2) provide a theoretical perspective to inform guidance for designing de-implementation interventions.

Behaviour change techniques (BCTs) are defined as ‘observable, replicable and irreducible components of an intervention designed to alter or redirect causal processes that regulate behaviour; that is, a technique is proposed to be an “active ingredient”’ [17]. Taxonomies of BCTs provide standard definitions and labels for intervention components that allow for systematic comparison of common components in intervention descriptions from a variety of clinical settings, behaviours and professional groups. The most comprehensive taxonomy is the BCT taxonomy (version 1), which consists of 93 techniques [17] which are hierarchically organised into 16 groupings. Each technique has a definition and an example to aid in designing interventions or coding of pre-existing intervention descriptions. The majority of the examples provided within the taxonomy are directed at changing health behaviours of patients and members of the public but can and have been applied to describe behaviour change interventions relating to clinical practice. Whilst there are a number of taxonomies that permit the identification of intervention components such as the Expert Recommendations of Implementation Strategies (ERIC) [18] or the Effective Practice and Organisation of Care (EPOC) [19] taxonomies, these taxonomies do not possess the granularity and specificity the BCT taxonomy contains. This granularity permits for an in-depth investigation of the potentially subtle differences in implementation and de-implementation that may be overlooked with other taxonomies.

Investigation of the BCTs used in implementation and de-implementation interventions to identify whether they are different will clarify the content of current interventions and may help to explain a theoretical base for designing de-implementation interventions. The objectives of the current study were to use the BCT taxonomy (v1) [17] to (1) determine what intervention components are included in both kinds of interventions and (2) whether these components differ.

Methods

Design and review selection

We conducted a secondary analysis of a subset of intervention studies included in three Cochrane systematic reviews. We sampled from systematic reviews completed under the auspices of the Cochrane Effective Practice and Organisation of Care group in the Cochrane Library. Titles, abstracts and plain language summaries of EPOC systematic reviews were screened for inclusion. The key eligibility criterion was that the reviews described the behaviour change as a change in frequency. We identified 142 potentially eligible EPOC systematic reviews and protocols on the date of extraction. Of these, 48 were protocols and 4 were withdrawn. Three reviews contained no studies. Of the 87 remaining reviews, 31 were excluded because the abstract or summary did not clearly indicate that health professionals were participants or report health professional behaviour as a study outcome. For example, in the review, ‘Effectiveness of intermediate care in nursing-led inpatients units’ the intervention focused on having nurses, rather than physicians, manage inpatient units; the behaviour was not a change in frequency (change in HCP role). Similarly, in the review titled ‘Dietary advice given by a dietician versus other health professionals or self-help to reduce blood cholesterol’, the outcomes were reported as patient blood cholesterol, body weight and high-density lipoprotein cholesterol levels, which were not measures of health professional behaviours.

Pragmatically, it would not have been possible to include studies from 56 reviews. Through a purposive selection process, reviews selected for inclusion in this study were Audit and feedback: effects on professional practice and healthcare outcomes [20], Interventions for improving antibiotic prescribing practices for hospital inpatients [21] and Interventions for improving the appropriate use of imaging in people with musculoskeletal conditions [22]. Criteria for purposive selection were (1) the reviews selected should include interventions that may target both implementation and de-implementation (i.e. increasing the frequency of high-value clinical behaviours and decreasing the frequency of low-value clinical behaviours) and (2) the reviews selected should not be limited to one professional group or setting but include various clinical settings and healthcare professions (e.g. primary care physicians, nurses, internists and other healthcare professionals in secondary and tertiary care facilities) to diversify the population of healthcare professional groups.

Screening and selection of articles

Titles and abstracts of the included studies from the systematic reviews were screened for eligibility. Studies were excluded if the intervention was not delivered directly to healthcare professionals, the behaviour change was not described as a change in frequency, or the desired change in frequency was unclear. In instances where eligibility was unclear, the articles were retained for full-text screening. The same eligibility criteria were applied in screening of full-text articles.

Inter-rater reliability

Two members of the research team (CSH, JJF) independently applied the inclusion criteria of 10% randomly selected articles initially screened by lead researcher (AMP). Agreement (Cohen’s kappa; κ) [23, 24] was calculated and any discrepancies were discussed with the lead researcher to clarify the coding of behaviour change frequency and to better inform the screening of remaining articles.

Data extraction

Data extracted from the articles included (1) desired change in behaviour frequency (implementation versus de-implementation); (2) health professional group (research participants); (3) sample size; (4) study design; (5) description of behaviour; (6) presence or absence of control group (y/n); and (7) intervention details as published.

Coding of intervention descriptions into behaviour change techniques

Using the BCT taxonomy (v1) definitions and examples from the coding manual [17], one researcher (AMP, who had completed formal training in BCT coding) coded the published intervention descriptions in all the included studies. Coding was conducted by assigning a BCT label from the taxonomy to passages of text of the intervention description from the article. All 93 techniques were considered for each intervention description. Content was coded for both the active intervention and the control condition. Two independent researchers with expertise in health psychology and training in the BCT taxonomy (v1) (HM and KM) coded the intervention descriptions of a subset of the included articles (20% each). Five coding assumptions were made to ensure consistency throughout the coding (see Table 1). Inter-rater reliability for identifying the same BCTs from intervention descriptions was assessed using percentage agreement; agreement above 80% was considered ‘satisfactory’ [25, 26]. Discrepancies between raters were resolved through discussion or consultation with a member of the BCT taxonomy research team (JJF).

Table 1 Coding assumptions for identifying behaviour change techniques [17] in intervention descriptions
Full size table

Data analysis

Identified BCTs for both implementation and de-implementation interventions were counted and ranked according to frequency. Chi-squared (χ2) tests (Yates’ continuity correction for cells less than 5; and Fisher’s exact tests for cells equal to 0) were used to compare frequencies of BCTs between implementation and de-implementation interventions [27, 28], with significance (p value) adjusted for multiple comparison.

Results

Selection of intervention articles

The three reviews included 255 articles describing interventions to change health professionals’ behaviour. Screening of titles, abstract and summaries resulted in the exclusion of 63 articles (see Supplemental file #1 for list of excluded articles). Articles were excluded because either the behaviour change was not described as a change in frequency or the desired direction of the change was unclear.

Inter-rater reliability

Inter-rater reliability of 26 double-screened articles was ‘substantial’ or ‘very good’ [25, 26] (κ = 0.839; 95% CI 0.626 to 1.000).

Full-text screening and data extraction

Full-text screening and data extraction of the remaining 192 articles resulted in the exclusion of seven articles. These articles reported interventions designed to change multiple behaviours and, from the published intervention descriptions, it was unclear which behaviour change techniques were targeting which behaviours. Figure 1 presents the flowchart for the identification of articles to be coded for BCTs. Supplemental File #2 contains characteristics of the included studies.

Fig. 1
figure 1

Flow diagram adapted from PRISMA to identify articles from three EPOC Systematic Reviews for BCT coding

Full size image

Of the 185 articles, 84 described implementation interventions and 101 described de-implementation interventions (see Table 2). The majority of studies for both implementation and de-implementation targeted a single healthcare professional group (n=56 and n=55, respectively; e.g. primary care physicians, internists, dentists). The remaining interventions targeted multiple professional groups (e.g. physicians and nurses in a practice, physicians in various hospital departments) or included the entire hospital population in the intervention. For the most part, study designs were either cluster-randomised controlled trials (implementation n = 59; de-implementation n = 31) or randomised control trials (implementation n = 16; de-implementation n = 16). However, interrupted time series (implementation n = 5; de-implementation n = 52), controlled before-and-after (implementation n = 2; de-implementation n = 1), and clustered controlled trials (implementation n = 1; de-implementation n = 1), were also reported. In study descriptions that identified a control group (n = 123), only 20 implementation interventions and eight de-implementation interventions described the control condition. The remaining studies with control groups reported that the control group ‘did not receive an intervention’ or ‘received usual care’.

Table 2 Descriptive characteristics for articles included for BCT coding
Full size table

Coding of intervention descriptions into behaviour change techniques

Agreement for BCT coding was 85% (AMP and HM) and 91% (AMP and KM). Sample BCT coding of an intervention description is provided in Supplemental file #3. Seven interventions descriptions could not be coded because there was not enough information to code [29,30,31,32,33,34,35].

In the remaining 178 articles, 25 of 93 possible BCTs (26%) were coded in de-implementation interventions and 29 (32%) were identified in implementation interventions. Table 3 provides a list of BCTs identified as well as definitions. Supplemental file #4 provides lists of BCTs coded in each implementation and de-implementation intervention descriptions.

Table 3 BCTs Identified in the study descriptions and definitions
Full size table

Frequency of BCTs in implementation versus de-implementation interventions

Comparisons were made between implementation and de-implementation interventions across all studies (Fig. 2) and within each review (Figs. 3, 4, and 5 present the frequencies of BCTs identified in the implementation and de-implementation interventions in Audit and feedback, Antibiotic prescribing, and Image ordering reviews, respectively).

Fig. 2
figure 2

BCTs identified in implementation and de-implementation interventions ranked by frequency; grey arrows indicate BCTs present in top 5 for both implementation and de-implementation; green arrows indicate BCTs present more frequently in de-implementation (dark green—top 5 de-implementation; light green—BCT not present in implementation); blue arrows indicate BCTs present more frequently in implementation (dark blue—top 5 de-implementation; light blue—BCT not present in de-implementation)

Full size image
Fig. 3
figure 3

BCTs identified in audit and feedback implementation and de-implementation interventions ranked by frequency; grey arrows indicate BCTs present in top 5 for both implementation and de-implementation; green arrows indicate BCTs present more frequently in de-implementation (dark green—top 5 de-implementation; light green—BCT not present in implementation); blue arrows indicate BCTs present more frequently in implementation (dark blue—top 5 de-implementation; light blue—BCT not present in de-implementation)

Full size image
Fig. 4
figure 4

BCTs identified in antibiotic prescribing implementation and de-implementation interventions ranked by frequency; grey arrows indicate BCTs present in top 5 for both implementation and de-implementation; green arrows indicate BCTs present more frequently in de-implementation (dark green—top 5 de-implementation; light green—BCT not present in implementation); blue arrows indicate BCTs present more frequently in implementation (dark blue—top 5 de-implementation; light blue—BCT not present in de-implementation)

Full size image
Fig. 5
figure 5

BCTs identified in image ordering implementation and de-implementation interventions ranked by frequency; grey arrows indicate BCTs present in top 5 for both implementation and de-implementation; green arrows indicate BCTs present more frequently in de-implementation (dark green—top 5 de-implementation; light green—BCT not present in implementation); blue arrows indicate BCTs present more frequently in implementation (dark blue—top 5 de-implementation; light blue—BCT not present in de-implementation)

Full size image

The most frequent BCTs in the implementation interventions were Feedback on behaviour (1st), Instruction on how to perform the behaviour (2nd), Social comparison (3rd), Credible source (4th), and Prompts/cues (5th). In the de-implementation interventions, the most frequent BCTs were Instruction on (how) to perform the behaviour (1st), Feedback on behaviour (2nd), Behaviour substitution (3rd), Monitoring of behaviour by others without feedback (4th) and Social comparison (5th). BCTs that were identified in implementation interventions and not in de-implementation were Goal setting (outcome) (n = 5), Material reward (behaviour) (n = 4), Behavioural contract (n =3), Action planning (n=2), Non-specific incentive (n = 2), Behavioural practice/rehearsal (n = 1) and Comparative imagining of future outcomes (n = 1). BCTs that were identified in de-implementation interventions and not in implementation were Restructuring the social environment (n = 13), Monitoring of outcomes of behaviour without feedback (n =3) and Demonstration of the behaviour (n=1).

Audit and feedback interventions

Twenty-seven BCTs were identified in implementation interventions and 19 were identified in de-implementation interventions (Fig. 3). The most frequent BCTs in the implementation interventions were Feedback on behaviour (1st), Instruction on (how) to perform the behaviour (2nd), Social comparison (3rd), Credible source (4th) and Goal setting (behaviour) (5th). In the de-implementation interventions, the most frequent BCTs were Feedback on behaviour (1st), Instruction on (how) to perform the behaviour (2nd), Social comparison (3rd), Feedback on outcomes of behaviour (4th) and Information about social and environmental consequences (5th).

Antibiotic prescribing interventions

Thirteen BCTs were identified in implementation interventions whilst 21 were identified in de-implementation interventions (Fig. 4). The most frequent BCTs in implementation interventions were Instruction on (how) to perform the behaviour (1st), Feedback on behaviour (2nd), Goal setting (behaviour) (2nd), Restructuring physical environment (3rd), Credible source (3rd) and Pros and cons (4th). In the de-implementation interventions, the most frequent BCTs were Instruction on (how) to perform the behaviour (1st), Behaviour substitution (2nd), Monitoring of behaviour by others without feedback (3rd), Restructuring physical environment (4th) and Feedback on behaviour (5th) and Restructuring physical environment (5th).

Image ordering interventions

Fifteen BCTs were identified in implementation interventions and 11 BCTs were identified in de-implementation interventions (Fig. 5). The most frequent BCTs identified in implementation interventions were Instruction on (how) to perform the behaviour (1st), Prompts and cues (2nd), Information about health consequences (3rd), and Credible source (4th). In the de-implementation interventions, the most frequent BCTs identified were Instruction on (how) to perform the behaviour (1st), Feedback on behaviour (2nd), Social support (unspecified) (3rd) and Credible source (4th).

Comparison BCTs of implementation and de-implementation interventions at the level of taxonomy hierarchical grouping showed no grouping of BCTs was consistently reported more frequently in implementation or de-implementation interventions (see Fig. 6). No intervention descriptions contained BCTs within the groupings: Regulation, Identity, Scheduled consequences, Self-belief and Covert learning.

Fig. 6
figure 6

Comparison of BCTs in implementation and de-implementation interventions at the BCT grouping level for (a) Antibiotic prescribing, (b) Audit and feedback and (c) Image ordering systematic reviews

Full size image

Data analysis

Table 4 reports the significant associations between direction of change (implementation versus de-implementation) and the BCTs reported in intervention description (p <.0015 adjusted for 32 comparisons; applying the Yates’ continuity correction for cells less than 5 and Fisher’s exact test for cells equal to 0). Specifically, Feedback on behaviour was identified more frequently in implementation than de-implementation (Χ2(2, n=178) = 15.693, p = .000057). Three BCTs were identified more frequently in de-implementation than implementation; Behaviour substitution (Χ2(2, n=178) = 14.561, p = .0001; Yates’ continuity correction); Monitoring of behaviour by others without feedback (Χ2(2, n=178) = 16.187, p = .000057; Yates’ continuity correction); and Restructuring social environment (p = .000273; Fisher’s 2-sided exact test).

Table 4 Association between desired change in behaviour (implementation (n=81) and de-implementation (n=97)) and BCT present
Full size table

Discussion

This study investigated whether implementation and de-implementation interventions as described in three purposively sampled Cochrane reviews contain different BCTs. Three of the BCTs reported more frequently in de-implementation interventions (Behaviour substitution, Restructuring social environment and Monitoring of behaviour by others without feedback) if associated with effective de-implementation interventions may help refine methods (and theory) for the design of de-implementation interventions.

Behaviour substitution is a technique that may be helpful in some de-implementation intervention designs. When designing theory-informed interventions, substitute behaviours were often introduced and the theories applied were used to increase the frequency of the substitute behaviour in order to de-implement the undesired or incompatible behaviour [12, 13]. When investigating determinants of behaviours in which the behaviour under investigation was a behaviour the HCP should avoid doing, researchers tended to frame the behaviour as ‘managing [clinical patient] without doing “x”’ [36,37,38]. Whilst permitting the respondents to reflect on the management of the patient rather than the specific behaviour under investigation, the framing of the question implies that the respondents could be doing something else.

One of the main benefits of using Behaviour substitution to de-implement behaviour is that it permits the healthcare professional (HCP) to focus on doing something, rather than doing nothing for the care of their patients. However, there are also potential challenges to using Behaviour substitution. Currently, we do not have methods for selecting appropriate substitute behaviours and the rationale for selection of substitute behaviour is rarely reported [12]. Researchers may have resorted to intuitive or pragmatic ideas within each context, resulting in no cumulative learning on how best to identify the behaviour to promote in Behaviour substitution. There is also the challenge of what to do in clinical contexts where there is no sensible substitute behaviour. This is probably less common in healthcare because, in the absence of performing the undesired behaviour, HCPs will likely decide to monitor the patient more closely or use strategies to address patient concerns. Example substitute behaviours in these cases are (a) for reducing unnecessary blood transfusions, the clinician continues monitoring the patient and orders additional tests [37]; (b) for deciding not to order an X-ray, the physician discusses their reasoning with the patient who has acute low back pain [39, 40]; (c) for deciding not to prescribe antibiotics in respiratory tract infections, the clinician could provide a viral prescription to the patients with symptom management strategies [41]. Further investigation is needed to clarify when Behaviour substitution would be effective (for what clinical behaviours and in what contexts).

Restructuring social environment may also be a candidate BCT for refining methods for de-implementation. For example, in many of the interventions for de-implementing antibiotic prescribing, HCPs were required to get approval from a senior or secondary clinician, or directly from the pharmacy, when requesting an antibiotic to be de-implemented. Seeking another clinician’s approval may be just enough of an inconvenience to deter the HCP from requesting the low-value care and the ease of order/prescribing is no longer there due to the added actions required. However, there may be unintended consequences of using this technique. Clinicians may perceive that their autonomy and clinical expertise is being challenged and there may be resistance. Additionally, applying this BCT would require additional labour resources that may not be available. Personnel would have to be willing to take on the added role and be available to review the requests. Identifying available resources to support application of this technique as well as addressing potential unintended consequences may be key factors its effectiveness and requires further study.

Monitoring of behaviour by others without feedback was reported more frequently in de-implementation than implementation. It is unclear why this is the case. One possibility is that de-implementation interventions historically have focused on quality improvement or cost saving measures and this BCT may have been applied as an organisational measure to document reduction in wasteful practice and cost as well as restrict clinicians’ practice behaviours. In addition, this BCT may work in the same manner as Restructuring social environment because clinicians would require other individuals to record and perhaps evaluate their practice behaviour. However, the unintended consequences of HCPs perceiving their autonomy and clinical expertise being challenged resulting in resistance, may be a concern when using this technique.

Strengths and limitations

This study was the first to apply the BCT taxonomy (v1) to compare implementation and de-implementation interventions that support HCP behaviour change. The taxonomy provided a relatively precise, ‘shared language’ [17] to describe a diverse range of implementation and de-implementation interventions. However, there were limitations to the current study.

It is important to note that our findings do not imply the effectiveness of the BCTs for implementation or de-implementation, rather simply that they were more frequently used and suggest that researchers who designed the interventions may have ‘implicit theories’ [42, 43] about different BCTs required for de-implementation and implementation. Whilst we would have liked to include effectiveness, we were unable to undertake any formal meta-analysis because studies across the three reviews were not sufficiently similar in terms of interventions evaluated and outcomes measured for the findings to be meaningful. Additionally, effectiveness reported in the Cochrane reviews included in this study, could not be attributed to a single BCT (whether used of de-implementation or implementation) but to the cluster of BCTs identified in the intervention. Therefore, we believe it would be premature to include reported effectiveness of these interventions without being able to attribute effectiveness to the specific BCT, as well as before fully understanding the intervention components and the fidelity of intervention delivery.

We found that 32 of the 93 possible BCTs were used in the interventions included in this study. Although it is possible that not all 93 BCTs would be appropriate in the settings investigated, this does suggest that there may be additional potentially effective techniques that researchers have not yet explored. Additionally, as we have indicated in the introduction, the BCT taxonomy is one of a number of taxonomies that could have been used to identify intervention strategies. We chose to use the BCT taxonomy for its granularity and specificity of techniques but recognise that, whilst a comprehensive list, it is not a complete list of all possible techniques and may omit techniques relevant to identify difference between implementation and de-implementation. The BCT taxonomy is currently in its first version and opportunities exist to add to or refine the taxonomy as new evidence and research emerges.

Whilst the number of studies in both implementation (n=81) and de-implementation (n=97) were similar, the number of studies in the reviews were skewed to either implementation or de-implementation. For example, the Audit and Feedback review included twice as many implementation studies as de-implementation studies (n=59, n=26 respectively). In the Antibiotic Prescribing review, the number of implementation studies was considerably smaller than de-implementation studies (n=8, n=60 respectively). BCTs in implementation interventions were found in a large number of audit and feedback studies and BCTs in de-implementation interventions were found in the majority of antibiotic prescribing studies. Whilst we have identified differences in the BCTs used for implementation and de-implementation interventions, these findings are limited to three Cochrane reviews and require replication for a wider range of clinical behaviours.

In addition, multiple statistical comparisons reduce power and therefore require that the significance criterion be adjusted to control for type 1 error. For this study, we applied a conservative approach and adjusted significance criterion to p<.00015 for the 32 comparisons. It could be argued that the significant criterion may have been too conservative and a number of comparisons that were not statistically significant (.05<p>.00015) would be significant without the adjusted significant criterion, including Goal setting (behaviour), Goal setting (outcome), Social comparison, Prompts and cues and Material reward identified more frequently in implementation and Restructuring physical environment identified more frequently in de-implementation. However, our conservative approach strengthens the confidence with which inferences can be made about the BCTs that distinguish between increasing and decreasing behaviour frequency, indicating a robust level of evidence that interventions contain some different techniques for implementation and de-implementation.

Implications for future work

As previously mentioned, we did not investigate effectiveness of identified BCTs. This is clearly a next logical step in understanding the differences between the processes of implementation and de-implementation. A recent study has proposed links from groups of commonly used BCTs in interventions to behaviour change theories, suggesting that there is a possible common theorising about how BCTs may work together which is grounded in behaviour change theories [44]. Our findings suggest that common theorising (even if implicit) may also exist in selecting different BCTs for de-implementation than implementation. Future work should include linking the BCTs used for de-implementation and their effectiveness to behaviour theories, which could guide further theoretical understanding about de-implementation.

Additionally, there is considerable opportunity to utilise the BCT Taxonomy (v1) [17] in other systematic reviews to determine if our findings are consistent with other comparisons of implementation and de-implementation interventions. Applying the same methods to intervention descriptions from other reviews, may identify whether these underreported BCTs were used more frequently or if they are consistently underused regardless of nature of the clinical behaviour or the clinical context. Because of the limited range of BCTs reported in the intervention descriptions of 187 published articles, there is opportunity to develop novel interventions that contain BCTs for the groups not reported and measure the effectiveness of those interventions to de-implement low-value care. It is also possible that the BCTs reported may not have been appropriate for the interventions because the majority of BCTs originated in context of clinical psychology, in which one-on-one interventions are delivered to people who have already acknowledged a need to change. Delivering the same BCTs in HCP behaviour change interventions may not be appropriate because of the different contexts and populations. Future research could develop novel interventions that contain BCTs for the underused BCT groupings. Investigating the application and effectiveness of less utilised BCTs may prove informative in interventions for de-implementation.

Conclusion

Policy interest in de-implementation has raised the question of whether the approaches to implement and de-implement are similar or different [4,5,6, 9, 11]. Implementation research has not yet addressed this question. This study has found that BCTs Behaviour substitution, Restructuring social environment and Monitoring of behaviour by others without feedback are frequent techniques for de-implementing low-value behaviours, and Feedback on behaviour was identified more frequently for implementing high-value care. Whilst there were some significant associations between BCTs reported in interventions and the direction of desired behaviour change (i.e. more or less, or implementation and de-implementation, respectively), there was also variation in BCTs included in these two kinds of interventions. These findings require replication for a wider range of clinical behaviours. The continued accumulation of knowledge and evidence into differences between the processes of implementation and de-implementation will serve to better inform researchers and, subsequently, improve methods for designing de-implementation interventions.