Background

The use of antibiotics is outpacing the discovery of new drugs, leading to antimicrobial resistance (AMR) as bacteria evolve to become ‘superbugs’, which are resistant to the drugs we use to treat them. In the UK alone in 2015, there were an estimated 52,971 cases of infection with antibiotic-resistant bacteria, resulting in 2172 deaths, which is similar to the burden of influenza, tuberculosis, and HIV combined (Cassini et al. 2019). A review of the problems posed by antimicrobial resistance commissioned by the UK Prime Minister in 2014 estimates a continued rise in resistance by 2050, leading to the deaths of up to 10 million people worldwide each year (O'Neill 2016). Increased illness also has an economic cost, and the review estimates that the cost of AMR could be $100 trillion worldwide by 2050.

A substantial driver of antibiotic resistance is the over-prescription of antibiotics in primary care (Her Majesty’s Government 2014). General practitioners (GPs) are often the first port-of-call for patients seeking medical help, and approximately 80% of antibiotics are prescribed in primary care, making this a suitable target for an intervention (Her Majesty’s Government 2014). Statistical modelling suggests that between 8.8% and 23.1% of all antibiotic prescriptions could be classed as ‘inappropriate’, meaning that the antibiotic is likely to have zero or marginal benefit, which is likely to be outweighed by the potential risks of prescribing to the patient (Smieszek et al. 2018).

Researchers at Public Health England (PHE) conducted a comprehensive review and analysis of the behaviours that support antibiotic stewardship, and which of the drivers of these behaviours are amenable to change (Pinder et al. 2015). Based on this analysis, they conducted a national randomised controlled trial in 2014/15, testing a social-norms-feedback letter, sent from England’s Chief Medical Officer (CMO) to 791 GP practices whose prescribing rate for antibiotics was in the top 20% in their local area (Hallsworth et al. 2016). The letter said that the practice was in the top 20% of prescribers in their area and listed three simple steps that prescribers could take to reduce inappropriate prescribing. The letter led to a 3.3% reduction in prescribing amongst GP practices that received it, with an estimated 73,406 fewer antibiotic items dispensed. Based on this evidence, similar letters were sent out to all practices whose prescribing was in the top 20% nationally in 2015/16 and 2016/17 (the NHS Local Area Teams, which were used in the original letter, had been disbanded). An evaluation of the 2016/17 letter using a regression discontinuity design showed that the letter continued to have an effect, causing a 3.69% reduction in prescribing, with an estimated 124,952 fewer antibiotic items dispensed (Ratajczak et al. 2019). The intervention has also been successfully implemented in Ireland and Australia (Behavioural Economics Team of the Australian Government 2018; HSC Public Health Agency 2017).

However, evidence suggests that there is overprescribing in all practices, not just those in the top 20% of prescribers (Smieszek et al. 2018). Given the success of the social-norms-feedback letter, we wanted to devise another social-norms message that we could send to prescribers outside of the top 20%. Practices are tasked with taking action to ensure that their prescribing behaviour is appropriate and to monitor judicious use of antimicrobials (NICE 2015). Antibiotic prescribing in primary care settings, measured in terms of antibiotic items being prescribed, has fallen by 16.7% from 2014 to 2018 (Public Health England 2018), which suggests that this message is having an effect and many practices are decreasing their prescribing. In the period of October 2016 to September 2017 (the most recent 12-month period of data available before the study’s launch), only 20% of practices increased their prescribing frequency by more than 4%, an increase that we judged to be greater than what might be expected due to natural variation alone.

At the time of our trial, there was no evidence on the effectiveness of providing GPs with social-norm feedback telling them that their practice is among a minority whose antibiotic prescribing is increasing. To address this knowledge gap, we conducted a randomized controlled trial in March 2018, of social-norms-feedback letters delivered to practices that were outside the top 20% of prescribers, but whose prescribing had increased by more than 4% between the financial year 2015/16 and financial year 2016/17. The intervention letter included a message telling GPs that they were in the minority of practices whose prescribing was increasing. We excluded the lowest prescribers from the trial (those whose prescribing was below the 95th percentile) because, after consulting with public health professionals, we decided it was unsuitable to send the lowest prescribers letters about overprescribing.

The aim of this trial was to determine whether a social-norms-feedback letter reduces prescribing rates in GP practices prescribing between 20th and 95th percentiles whose prescribing had increased by more than 4% in 12 months before the intervention letter was sent, compared to a control group of the same characteristics who were not sent a letter.

Methods

Trial design

We conducted a two-armed randomised controlled trial. In February 2018, a letter and a ‘Treating your infection’ leaflet were sent to half of the practices between the 20th and 95th percentiles of practices whose antibiotic prescribing (adjusted for practice demographics) was increasing, informing them that they were in the minority of practices whose prescribing was increasing (see the Interventions section for more details, and Appendix 1 for examples of the letter and the leaflet). The control group did not receive a letter, which was standard practice.

Participants

There were 930 eligible GP practices in the trial. The prescribing indicator was the rate of dispensed antibiotic items per 1000 population for the practice, after applying the specific therapeutic group age-sex related prescribing units (STAR-PU) controls for age and sex. Practices were eligible if their prescribing fell between the 20th and 95th percentiles of prescribers for October 2016 to September 2017 (the most recent 12-month period of data available before the study’s launch) and their antibiotic prescribing had increased by more than 4% in the financial year 2016/17 compared to the previous financial year, 2015/16. Practices above the 20th percentile were excluded because they were sent the standard practice social-norms letter, and practices between the 95th and 100th percentiles were excluded as outliers. The prescribing data are collected for each GP practice (individual prescribers’ data is not available) on a monthly basis by the NHS Business Services Authority. Public Health England makes these data available on Fingertips.Footnote 1 We used BNF code 5.1 and we divided the number of items prescribed by STAR-PU, in order to get the number of items dispensed per 1000 patients in the practice population when adjusted for age and sex.

Interventions

Letters from the CMO were sent to GPs at the end of February 2018, timed to land on March 1st, 2018, in an envelope that said on the outside that it was ‘From the Chief Medical Officer, Professor Dame Sally C Davies’, with a Department of Health and Social Care logo. The header stated that ‘Antibiotic prescribing rates in your practice have increased.’ Inside the text of the letter, there was a banner that informed GPs that ‘The great majority (80%) of practices in England reduced or stabilised their antibiotic prescribing rates in 2016/17. However, your practice is in the minority that have increased their prescribing by more than 4%.’

Like the letter sent to the top 20% in previous years, the letter listed three simple actions that the recipient could take to reduce unnecessary prescriptions of antibiotics: giving patients advice on self-care, offering a delayed prescription, and talking about the issue with other prescribers in the practice. All letters were accompanied by a copy of the patient-focused “Treating your infection” leaflet developed for the TARGET programme. A sample letter and the leaflet are reproduced in full in Appendix 1.

Outcomes and sample size

The primary outcome for the trial is the STAR-PU-adjusted rate of antibiotic prescribing in the 7 months following the intervention, March 2018 to September 2018. The trials were powered to detect a 2% change in antibiotic prescribing rate between arms of the trial, for each of March, April individually, and May to August combined (since prescribing is lower in those months), controlling for baseline prescribing behaviour — these were pre-registered outcomes, but by the time we analysed the trial, data was available for September 2018 as well, so we included that in our analyses; it does not affect the outcome. A 2% effect size was chosen as it was similar to the effects observed in comparable interventions carried out previously, and as smaller effect sizes would not represent a meaningful effect.

Randomisation and masking

A study investigator (MR) randomly assigned GP practices to intervention or control group, grouped by sampling a hypothetical binary distribution of control (0) and treatment (1) groups, where the probability of treatment group membership was 0.5. This was done using R (version 3.4.3) (Team 2017). GP names and addresses were matched to practices using GPs practice codes.Footnote 2

Participants in intervention groups are likely to have been aware of the interventions they were assigned to, but may have been unaware that they were involved in a trial. However, information was published on the PrescQIPP AMS Hub,Footnote 3 which supports Clinical Commissioning Group antimicrobial stewardship activity, including which GP practices had received letters, so we cannot be sure they were unaware. Since the prescribing dataset had to be matched with the dataset of GP names using GP practice codes, it was not practical to blind the study team to group assignment.

Ethical approval

Ethical approval was granted by the NHS Health Research Authority, REC reference: 14/LO/1544.

Statistical analysis

Monthly prescribing rates by different GP practices were considered to be independent, while those from the same GP practice are potentially correlated. Therefore, we used a statistical modelling approach that allows the monthly prescribing from the same GP practice to be correlated, by allowing the model residuals to be correlated. An autoregressive and moving average of first-order ARMA(1,1) correlation structure was adopted here for the correlated residuals. The model included a trend in addition to the intervention. The data analysis was done in SAS using [SAS/STAT] software, Version 9.4 of the SAS 64 BIT WIN (SAS 2016).

Results

There were 930 practices randomly assigned in the trial. Feedback letters were sent to 451 (48.49%) practices and 479 (51.51%) were not sent a letter. The letter landed on March 1st, 2018 and prescribing data from March to September 2018 inclusive was included in the analyses. By March 2018 there were 920 GP practices with prescribing data (data ceases when a practice merges or closes), leaving 448 (48.17%) practices who received the intervention and 472 (50.75%) in the control. See the trial flowchart in Fig. 1. A further 22 GP practices had their last month of prescribing between March and September 2018 and therefore had some missing data for the outcome measure. These practices were included in the analysis. The number of practices included in the analysis each month is shown in Table 1, along with the 7 months of prescribing data from March 2018 to September 2018 inclusive. The monthly mean for each arm is graphed in Fig. 2.

Fig. 1
figure 1

Trial profile

Full size image
Table 1 Antibiotic prescribing rates per STAR-PU for the overall sample, intervention, and control groups from March until September 2018 inclusive
Full size table
Fig. 2
figure 2

Monthly trend of prescribing mean over 8 months

Full size image

In the model, as expected, the monthly measurements (items per STAR-PU prescribed) from the same GP practice were highly correlated, with estimated autocorrelation ρ = 0.970, SE < 0.01 and moving average parameter γ = 0.77, SE < 0.01 (for model coefficients see Table 2). The declining item rates trend in Fig. 2 was strongly supported by the results from the model, which showed a statistically significant negative trend in prescribing (β < −0.01, z = −36.36, p < 0.001).

Table 2 Parameter estimates (Est) and their associated standard error (SE), z-value and p value
Full size table

There was no evidence of an effect of the intervention: the correlated adjusted estimate of the intervention effect was negative but it was not statistically significant (β < −0.01, z = −0.50, p = 0.565) — see Table 2. Model diagnostics (not reported here) did not indicate any serious model failure.

Harms and unintended effects

The CMO’s address was included in the letter, and she received correspondence about the letter, but no harms or unintended effects were reported.

Discussion

The social-norms-feedback letter to practices outside of the top 20% whose prescribing was increasing did not have a statistically significant effect (β < −0.01, p = 0.565). Prescribing was decreasing in both trial arms over the outcome period from March 2018 to September 2018 (β < −0.01, p < 0.001). The lack of effect of the intervention was surprising, given the evidence of effectiveness of previous social-norms-feedback letters to GPs in the UK (Hallsworth et al. 2016; Ratajczak et al. 2019) and internationally (Behavioural Economics Team of the Australian Government 2018; HSC Public Health Agency 2017).

There are various possible explanations why this letter was not effective.

Previous letters, which have been effective, targeted the highest 20% of prescribers (Behavioural Economics Team of the Australian Government 2018; HSC Public Health Agency 2017; Ratajczak et al. 2019), whereas these letters targeted practices whose prescribing was between the 20th and 95th percentiles. There may be more scope for reducing inappropriate prescribing amongst higher prescribers. However, statistical modelling shows that all practices have scope for decreasing inappropriate prescribing (Smieszek et al. 2018). Nevertheless, the GPs who received the letters in this trial may have known that they were not in the highest prescribing practices and therefore not have been very concerned about their antibiotic prescribing rates. In a discussion at a workshop about giving prescribing feedback to primary care, participants said that GPs care about being outliers on prescribing indicators (i.e. very high or very low) and are more comfortable being in the middle (Steels et al. 2021).

Because the target group was different, the message was different from previous letters: instead of telling practices that they were in a minority of practices whose overall prescribing was high, we told them that they were in a minority of practices whose prescribing was increasing. The active ingredient of the message was intended to be the social norms part, about being in a minority: social norms have been effective in a variety of areas (Sheeran et al. 2016), including tax evasion (Hallsworth et al. 2017), energy usage (Ayres et al. 2013), and charitable giving (Agerström et al. 2016), and there is nothing in theory to suggest that the exact content of the message or what group they were the minority of would have an effect (Bicchieri 2016; Cialdini 2007). Possibly the social-norms message was less salient than in our previous letters, since although the red font in the body of the letter told GPs that their practice was in a minority, the large font header at the top of the letter said ‘Antibiotic prescribing rates in your practice have increased’. In previous years, the letter header reinforced the social-norms message, saying ‘Your practice is amongst the 20% highest prescribers of antibiotics’ (Hallsworth et al. 2016; Ratajczak et al. 2019). Antibiotic prescribing rates in your practice have increased, but most other practices reduced or stabilised their prescribing’ or ‘Compared to other practices, your prescribing rates have increased’.

We found that prescribing was decreasing over the trial period, with a negative trend across both arms. Possibly the letter has ceased to be effective as prescribing has come down. However, evidence shows that there is still plenty of scope for practices to reduce prescribing (Smieszek et al. 2018). There were many other national interventions happening at the same time, such as the Quality Premium, Commissioning for Quality and Innovation (CQUINs) quality improvement goals, national guidelines launched by the National Institute for Health and Care Excellence (NICE), and the extension of the TARGET (Treat Antibiotics Responsibly, Guidance, Education, Tools) toolkit resources via a number of workshops (Public Health England 2018). Many of these interventions were also happening in previous years, when the letter to the highest prescribers was shown to be effective (Hallsworth et al. 2016; Ratajczak et al. 2019). However, our letter listed three actions that GPs could take to reduce inappropriate prescribing and, if these actions were already widely known and implemented, maybe as a result of other campaigns, then the letter would simply have reinforced actions to reduce prescribing that were already happening.

As well as running the trial, we sent the standard letter to the highest 20% of prescribers, which landed on the same date as the intervention letter. An evaluation of its effectiveness, for instance using a regression discontinuity design, could help discriminate between some of the explanations for the lack of effectiveness of the letter in the present study. For instance, if the top 20% letter was effective, then that is consistent with the explanations that the message that prescribing was ‘increasing’ caused the lack of effect or that targeting prescribers outside of the top 20% is less effective. However, it is inconsistent with the explanation that the national campaigns have already led GPs to implement all the actions listed in our letters, as the same actions were included in the standard letter to the highest 20% of prescribers. Alternatively, if the top 20% letter was not effective, then that would support the explanations that reductions in prescribing have left less room for improvement, or that the actions listed are already widely implemented, and it would not support explanations that refer to differences in message content.

The other hypothesised active ingredient of these letters is the messenger — the CMO at the time Dame Professor Sally Davies. The fact that the present letter was not effective suggests it cannot solely be the messenger effect which has made previous letters effective.

Although social-norms interventions have been widely found to be effective (Sheeran et al. 2016), this is not the first social-norms study to have returned a null result. Social-norms messages have failed to work in other healthcare interventions, including immunization (Leight and Safran 2019) and alcohol misuse (Foxcroft et al. 2015; Moreira et al. 2009), and there have been other null results in areas where they have previously been used successfully, such as energy usage (Harries et al. 2013). But it is hard to know why some interventions have been unsuccessful. Social-norms interventions are heterogeneous, and it may be that some types of social-norm interventions are not effective. Simply adding a photograph to a social-norms message aimed at increasing organ donation registration proved to be worse than an identical message with no image or an identical message with a logo, and less effective than the control arm using no persuasive message at all (Sallis et al. 2018). Message content and mode of delivery may be important (Dempsey et al. 2018). In this study, the mode of delivery and a very similar message have both been successful in the past (Behavioural Economics Team of the Australian Government 2018; Hallsworth et al. 2016; HSC Public Health Agency 2017; Ratajczak et al. 2019). Future research might investigate the effectiveness of different types of content of social-norms messages.

Conclusion

A social-norms-feedback letter to practices whose prescribing was increasing did not decrease prescribing compared to control practices who did not receive a letter. Prescribing decreased in both trial arms over the outcome period. There are various possible explanations why this letter was not effective in this instance. Further research could investigate them.