Background

Inappropriate use and overuse of antibiotics is a serious concern, especially in developing countries such as China [1]. It is perhaps the most prominent manifestation of irrational drug prescribing and contributes to bacterial resistance, which is an emergent threat to the global population [2–4]. Nosocomial or hospital-acquired infection of antibiotic-resistant bacteria has the capacity to increase medical costs, length of hospital stay, and ultimately patient mortality [5, 6].

Upper respiratory tract infections (URTIs) are one of the most common health conditions seen by primary care workers, and the associated overuse of antibiotics is widespread [7]. In the UK, for example, general practitioners prescribe antibiotics to 67% of patients presenting with respiratory infections, including 47% of those with URTIs [8]. It was estimated that 60% of respiratory infections prescribed with antibiotics in the UK are self-limiting conditions [9]. A large cohort study of antibiotic usage associated with URTI in UK primary care settings concluded that prophylactic antibiotic prescriptions are unjustified in reducing the risk of serious complications in URTIs [10].

In China, approximately 75% of patients with seasonal influenza were estimated to be prescribed with antibiotics, more than doubled of the maximum rate (30%) recommended by the World Health Organization (WHO) [1]. Several systematic reviews have shown that antibiotics are of limited effectiveness in the treatment of URTIs [11] and suggested no benefit in colds [12].

Growing public health concerns regarding the implications of inappropriate use and overuse of antibiotics have resulted in global action [13]; China is no exception. The recent health reform (2009–2011) in China articulated specific governance and incentive structures for public health organisations to address the issues of poor quality in prescribing, such as the enforcement of an essential medicine list policy and decoupling the nexus between medicine sales to health worker remuneration and bonus payments in primary care organisations [14]. Despite this, empirical evidence demonstrates that the effectiveness of these policy interventions has been short of expectations [15].

In recent decades, public reporting (PR) has started to attract attention as a vehicle for improving the performance of health organisations [16–20]. The United States has led the PR movement, along with the UK [21]. The rationale behind this approach draws experience from other industries with theoretical support from the domains of psychology, economics, and organizational behaviour. Public release of performance data is considered a bottom-up intervention compared with government regulation [22], and is intended to improve quality of care through imposing a higher level of transparency and accountability [23]. Frølich et al. believe that PR can provide both financial and reputational incentives for health providers to improve quality of care, although contextual factors (at the levels of the markets and provider organisation) and characteristics of providers and patients may enhance or mitigate their responses to those incentives [24]. Reputational incentives could also have financial implications.

This study aims to explore the potential use of PR for improving the quality of prescribing in primary care organisations using a social experiment. The study has important implications, primarily to reduce unnecessary antibiotic prescription, but also as an analysis of whether greater transparency leads to improved healthcare outcomes. Provider-specific comparative performance reporting is currently being debated and implemented in a subset of developed countries, but only in a small group of developing countries [25]. Scant information is available regarding the effectiveness of PR in primary care settings, particularly in relation to prescribing behaviours. The concept of PR is almost completely novel for health workers in China.

Methods

Setting

This study was undertaken in QJ city of Hubei province, involving 20 primary care organisations. Hubei has a population of 58 million and a GDP of 2,225 billion (Yuan in 2012), ranking in the middle range of all Chinese provinces. QJ is located in central Hubei, with a land area of 2,004 km2 and a population of 1 million. The annual GDP of QJ reached 49.3 billion (Yuan) in 2012, just above the average level of all cities in Hubei. The majority of inhabitants in QJ are of Han ethnicity, with about 8,000 from Hui, Tujia, and other ethnic groups.

We selected QJ due to ready access to prescription data and implementation of intervention measures. Primary care organisations in QJ have the benefit of an electronic health information system (HIS). The QJ government has been very supportive of this project.

Written informed consent was obtained from all participants. This study was approved by the Ethical Review Committee of Tongji Medical College, Huazhong University of Science and Technology (No. IORG0003571).

Experimental design

This research adopted a matched-pair randomized trial design. The 20 participating primary care organisations were paired according to their size, geographic location and economic status, and population serviced. We considered all the organisational characteristics presented in Table 1 in the pair-matching process. The Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) method was adopted to calculate a summed score from the matching variables (standardised) for each participating organisation. TOPSIS is a multi-criteria decision analysis method that estimates the geometric distance between each alternative and the ideal alternative to a given solution [26]. The participating organisations were ranked in order according to their distances to the positive and the negative ideal alternatives. Adjacent organisations were paired and assigned randomly to the intervention and control groups.

Table 1 Characteristics of participating institutions and patients
Full size table

Participants in the intervention group and the control group were exposed to the same political, financial, and administrative environments: the only exception being the intervention measures. Identical training on appropriate prescription practice was offered to both intervention and control groups.

Intervention measures

The intervention measures were designed after considering the following assumptions [21, 27]:

  • Patient choice of healthcare providers: patients or potential consumers would make an informed choice of their preferred provider(s) based on reported prescription indicators, exerting consumer pressure on healthcare providers.

  • Use of reported data by regulators and managers: medical insurance authorities, local health authorities, and health organisation managers may use the reported data for decisions in payment (or reimbursement), health planning, and performance management, exerting financial and administrative pressure on healthcare providers (individual or institutional level).

  • Provider efforts to improve quality of care: physicians and healthcare organisations have intrinsic motivation to improve quality of care, especially if their quality of care falls below average.

The PR intervention started in October 2013. Three elements were included in the PR of this study: prescribing indicators were calculated using data extracted from the HIS on a monthly basis; healthcare providers (at both individual and institution levels) were ranked using the prescribing indicators and the reports were displayed in a public space in the primary care organisations in the intervention group; the monthly performance reports and league tables were submitted to local health authorities and organisational managers in the intervention group (see Public reporting package below). The reports also contained a brief explanation about the purpose of the PR intervention (i.e., to curb overuse of antibiotics). Administrative measures (e.g., meetings and newsletters) were taken to ensure that all prescribers in the intervention group were aware of the reports. However, there was no action taken to individually alert patients to the reports, although these reports (including poster displays and brochures) were readily available every day in a designated public space.

Public reporting package

  1. 1)

    Timeline: Monthly reporting in the intervention group started in October 2013, which is still continuing at the time of writing.

  2. 2)

    Reported indicators: Prescription indicators were calculated at both individual (physician) and institutional levels, including:

    • Percentage of prescriptions requiring antibiotics (%) = Number of prescriptions requiring antibiotics/total number of prescriptions by a physician (or institution) in one month × 100%.

    • Percentage of prescriptions requiring IV injections (%) = Number of prescriptions requiring IV injections/total number of prescriptions by a physician (or institution) in one month × 100%.

    • Expenditure per prescription (Yuan) = Total expenditure of prescriptions/total number of prescriptions by a physician (or institution) in one month.

  1. 3)

    Dissemination: Prescribing physicians and hospitals were ranked in each of the above indicators. The league tables were publicly available to consumers and health workers alike, displayed on a bulletin board in the lobby of outpatient departments. Hard copies of the reports were also submitted to local health authorities and presidents of the hospitals in the intervention group.

  2. 4)

    Update of reports: The reports were updated monthly and made available on the first week of each month.

To ensure compliance with the intervention, we sent trained investigators to the intervention sites at random to monitor the implementation of intervention measures on a monthly basis.

Data collection

Data pertaining to the characteristics of the participating organisations were extracted through their respective administrative systems prior to commencement of the PR intervention, covering the entire year of 2012. These included population serviced, number of outpatient visits, number of hospital admissions, bed numbers, number of doctors, gross revenue, revenue from drug sales, and revenue from government subsidies. The research team validated these data in interviews with managers and reconciled discrepancies as necessary.

Prescription data were drawn from the electronic HIS. For the purpose of this study, we collected unidentified prescription data from the participating organisations for a period of four months prior to the intervention (1st March to 31st May, 2013) and four months after start of the intervention (1st March to 31st May, 2014). All prescriptions associated with URTI were identified and collated for data analyses. The prescription data also contained demographic information (gender, age, insurance status) of the patients, tests and examinations ordered by the physicians, and expenditure in relation to the prescriptions.

Statistical analysis

The prescribing indicators used in this study were adapted from the WHO/International Network for the Rational Use of Drugs indicators, which have been used widely [2, 15, 28]. We examined the extent to which the PR intervention changed the prescribing patterns in relation to those indicators, such as:

  1. 1)

    Percentage of prescriptions requiring antibiotics;

  2. 2)

    Percentage of prescriptions requiring two or more antibiotics;

  3. 3)

    Percentage of prescriptions requiring intravenous (IV) injection (peripheral IV administration using a syringe driver [29]);

  4. 4)

    Percentage of prescriptions requiring infusion (medicines are introduced into a vein or between tissues by using an infusion pump or bag [29]);

  5. 5)

    Expenditure per patient visit (including consultations, diagnostic tests, and prescriptions).

A difference-in-difference (DID) estimation of the above indicators was used in our study to assess the impact of the PR intervention on prescribing patterns. The DID method allows us to control confounding influences of independent variables, as well as imbalance between groups in baseline (dependent) variables (due to chances of imperfect randomisation).

We performed regression modelling for estimation of the effect size of PR. We reported both unadjusted estimates of effect size and those adjusted for characteristics of patients (age, sex and enrolment with the New Cooperative Medical Scheme (NCMS)) and hospitals (number of beds, number of doctors, population serviced, outpatient visits a year, inpatient admissions per annum, drug sales revenue per annum, variety of drugs in stock, percentage of drug sales in total revenue, and average revenue per doctor).

For binary dependent variables (in relation to indicator 1, 2, 3, and 4), we used logistic regression models. For a given patient i serviced within organisation j, the probability p ij of the occurrence of a dichotomous outcome y ij (i.e., value 1 for a patient receiving antibiotics, 0 otherwise) modelled as a logistic model can be expressed as:

In P i j 1 - P i j = a 00 + β t T i j + β g G i j + β e EFFEC T i j + ∑ o β o X o i j + ∑ p β p X p i j

and report marginal effects, where

  • i =1 . . . i patient, j =1. . . j primary care organisation;

  • T ij indicates ith patient receiving a prescription within jth primary care organisation before or after intervention (0 before or 1 after);

  • G ij indicates ith patient receiving a prescription within jth primary care organisation that had or had not implemented PR intervention (0 control or 1 intervention);

  • EFFECT ij  = T ij * G ij , which measures invention effect and coefficient β e indicates the effect size;

  • o =1. . . o patient visit covariate, p =1. . .p primary care organisation covariate.

For the continuous dependent variable (in relation to indicator 5, log transformed), we used a least squares regression model, which can be expressed as:

y i j = a 00 + β t T i j + β g G i j + β e EFFEC T i j + ∑ o β o X o i j + ∑ p β p X p i j

X oij and X pij are the control (independent) variables. In addition, a dummy variable was generated to measure the cluster-pair fixed effect. Robust standard errors were clustered at the organisational level.

Coefficient β e in the regression model 1 and 2 is actually an estimation of the interaction between intervention and time, measuring the DID. For all of the outcome indicators, we conducted subgroup analyses by gender.

All statistical analyses were performed using Stata (version 10).

Results

Characteristics of participating organisations and patients

Similar characteristics of participating organisations were shown between intervention and control groups (Table 1). Each organisation served on average a population of 40,000 and provided 50,000 outpatient consultations per annum. A total of 34,815 prescriptions for URTI were deemed eligible for inclusion in the study. Those patients prescribed with antibiotics had a mean age of 23.6 years (Standard deviation =23.5); about half (49.5%) of them were male and over 85.9% were enrolled with the NCMS.

Prescriptions requiring antibiotics

High rates of antibiotic prescription were noted in both intervention and control groups (Table 2). Prior to the intervention, more than 90% of prescriptions for URTIs contained antibiotics.

Table 2 Percentage (%) of prescriptions requiring antibiotics for upper respiratory tract infections
Full size table

The PR intervention led to only a slight reduction in the use of antibiotics, in particular for oral administration of antibiotics: a 9 percentage point reduction in oral administration of antibiotics was shown (P <0.05).

Paradoxically, there was an increase in the percentage of prescriptions requiring two or more antibiotics in both intervention and control groups; however, this increase was 7 percentage points less in the intervention group compared with that in the control group (P <0.05). This mainly occurred for male URTI patients (P =0.03), whereas the effect was statistically insignificant for female patients.

Prescriptions requiring IV injection or infusion

No significant differences were found between the intervention and control groups in the changes in percentage of prescriptions requiring IV injection or infusion (Table 3). Overall, the percentage of prescriptions requiring IV injection or infusion remained high during the period of study, with 77% to 90% of prescriptions for URTI requiring IV injection or infusion.

Table 3 Percentage (%) of prescriptions requiring IV injections and infusions and average expenditure per prescription for upper respiratory tract infection
Full size table

Average expenditure per visit

No significant differences were found between the intervention and control groups in average expenditure per visit (Table 3). Overall, the average expenditure per visit for URTI patients was low, ranging from 27 to 41 Yuan (roughly USD$5–7). The average expenditure per visit measured in the post-intervention period increased slightly compared to that in the pre-intervention period in both intervention and control groups. Such increase was 10% to 15% less in the intervention group than in the control group. However, the adjusted estimate of treatment effect was close to zero after controlling for independent variables.

Discussion

PR can have an impact on prescription patterns in primary care settings. This study demonstrated that PR interventions reduced the overall number of prescriptions associated with oral administrated antibiotics and retarded an anticipated potential rise in combined use of two or more antibiotics for URTIs. Although clinical outcomes of such a change were not explored in this study, it is reasonable to expect a positive outcome. Overuse of antibiotics is a widespread and serious problem in China [15, 28, 30], despite clinical guidelines for URTIs that advise against routine use of antibiotics [31]. Systematic reviews with meta-analyses of randomised controlled trials have shown minimal benefit associated with antibiotic prescriptions for URTIs [12, 32, 33]. Meanwhile, there are growing concerns about the possibility of antimicrobial resistance induced by irrational use of antibiotics. It was estimated that abuse of antibiotics for URTIs in low- and middle-income countries contributes to an additional 36% of medical costs [34]. Reducing irrational use of antibiotics can preserve these resources for more cost-effective interventions.

It is important to note that the overall use of antibiotics for URTIs in the participating primary care organisations remained at a high level despite the PR interventions. In addition, the PR interventions failed to show a significant impact on prescriptions using parenteral administration (injection and infusion). This study found that over 87% of prescriptions for URTIs contained antibiotics, much higher than the WHO recommendations [1]. The percentage of prescriptions requiring administration via IV injection and infusion for URTI reached a staggering level – over 77%. Arguably, IV injection and infusion are more likely to impose health risks for the public, leading to iatrogenic and nosocomial problems such as transmission of blood-borne viruses and infection of infusion sites [3].

The persistent high prevalence of antibiotic misuse probably reflects both inertia in physician practice and continued demand from patients. Nevertheless, many Chinese patients regard parenteral administration as integral to high-quality care [3]. The national zero-mark-up policy, introduced during 2009–2011, prohibits primary care organisations from generating excessive profits from retailing drugs, with the intention of making drugs more financially accessible. Health providers are wily and resourceful: fees can be charged in relation to injection or infusion services [35]. This may explain why the apparent consumer demand for antibiotics continues to be high, and why prescribing changes were mainly limited to oral antibiotics.

PR interventions did not have a significant effect on the average expenditure per visit. One plausible explanation could be that the expenditure had already been low due to a range of policies. The participating primary care organisations in both intervention and control groups had been exposed to the same policy environments over the past decade. Primary care organisations are only allowed to stock and provide drugs listed in the essential medicines list; these drugs can only be purchased through a regional centralized tendering and purchasing system at an agreed price; patients (and insurers) are charged with exactly the same purchasing price for their prescribed drugs. Empirical evidence showed that those policies have led to a significant reduction in drug expenditure [15]. Understandably, the capacity for further reduction of drug expenditure in primary care organisations is therefore limited.

There are policy implications arising from this study. Evidence from randomised control trials is regarded as the most robust and reliable. It is rare to find a study that uses a randomised control trial design to test the effect of PR interventions on physician prescribing practices within a complex health system and policy environment [16, 17, 21].

Internationally, healthcare systems are moving towards greater transparency and accountability [22]. Performance reporting in healthcare has been shown to improve the health systems of developed countries [16, 17, 21]. Despite this, reporting systems must be properly developed, considering a system-wide stakeholder participation, a change of organisational culture, and the accuracy and relevancy of reported data. It is also important to be mindful of minimising unintended consequences [36].

PR of prescription performance may impose “pressures” on prescribers arising from consumers, payers, managers, and colleagues. The magnitude of the pressure depends on the level of PR awareness and how the population understands it [24, 27, 37, 38].

We fear that consumer pressure to reduce irrational antibiotic and injectable prescription is limited in China, as the disclosed prescription indicators are difficult for consumers to understand due to limited health literacy. It is unlikely that the general public are either able or inclined to use the reported information to identify and choose healthcare providers. Previous studies show that consumer health literacy is low in China [39]; overuse of antibiotics is common; and physicians may be under pressure to prescribe more, not fewer, antibiotics [1]. The effect of PR on patients may also be jeopardised by limited exposure of patients to PR.

Pressure from payers is usually linked with financial incentives (or disincentives) [24, 40]. In this study, we found that the PR interventions are more likely to have an impact on the use of oral administered antibiotics compared with those administered via IV injection or infusion. This is perhaps because there are no financial incentives directly linked to reduction in irrational use of antibiotics, and fewer prescriptions of oral antibiotics (with zero-profit attached) are less likely to have a negative consequence on organisational revenues than reduced prescriptions of injected antibiotics (where a service fee can be charged). Clearly, these are demonstrable examples of perverse incentives.

Pressures from managers and colleagues may be a major driving force for the impact of PR interventions on prescribers. The working environment of physicians in China is characterised by strong administrative interventions from the government and organisations [41, 42]. Physician buy-in of PR is also essential for success; after all, they make the final prescription decision. Our experience, together with studies conducted elsewhere [43], show that physicians may challenge the validity of the PR and express concerns about performance reporting at individual levels for potential misinterpretations from patients. If a physician believes that the reported data are flawed (e.g., lack of adequate risk adjustment), he or she may be less likely to be influenced by the data. To make PR more acceptable, simplified data presentation, continued risk-adjustment refinement, and internal review before PR are necessary [43].

The development of an adequate PR system should be tailored to the macro- and micro-environment in which the healthcare organisations are operating [40]. Consumers and health care providers should be involved in the design and specification of information requirements, reporting approach, and communication strategies [23]. An ideal PR system should be able to assist patients or potential users to make better informed decisions [44]. Such a system should also address concerns from physicians in order to gain their acceptance [43]. A high quality information system is also critical for easy access and quality assurance of data. Translating good practice from one program to others is a practical approach for building high quality information systems [45]. Nevertheless, none of these measures are likely to take place if the income for services remains as a fee for service, and opportunities remain that maximise income through unnecessary drug administration practices.

Limitations

This study was undertaken during an early stage of PR interventions, which may underestimate the true impact of the interventions. PR does not have a direct impact on prescriber behaviours. It may take time for healthcare providers to understand the rationale underlying the PR interventions and subsequently to change management and clinical practices. The mechanisms and long term effect of PR on prescribing behaviours warrant further studies.

The positive but limited effect of PR on prescription outcomes might also be indicative of shortfalls in project design and implementations. For example, this project included a mixed reporting system at both organisational and individual levels, which prevents us from exploring further mechanisms of the effect. Nevertheless, the study has provided important evidence for further improving the reporting system.

Conclusions

This study demonstrated that PR interventions reduced the incidence of oral antibiotic prescription and slowed down the increase of combined use of antibiotics for URTIs. However, this effect is limited. Overall, the use of antibiotics remained at high levels, and the PR interventions failed to make an influence on the frequent use of injections and infusions as an administration route. The impact of PR interventions is dependent upon pressures from consumers, payers, managers, and colleagues. Patient empowerment mechanisms, financial incentives, reliable information systems, consumer education, and prescriber buy-in are required to maximise the impact of PR interventions.