Background

Pay-for-performance systems exist in an increasing number of countries [1–6] as a means of improving the quality of healthcare [7–9]. Where incentivised chronic disease management (CDM) has been implemented, small improvements in quality have been reported [10, 11]. However, physicians, incentivised to reach CDM targets, run the risk of providing inappropriate care to achieve performance targets [12], or of excepting an unlimited number of patients within a system in which external inspections are not comprehensive and of unknown effectiveness [2] or widening inequalities through deselection of patients who are more difficult to reach, or for whom incentivised targets are more difficult to achieve [11, 13, 14]. Inequalities in health outcomes, service experiences and variations in care led UK primary care services to introduce the pay-for-performance contract (Quality and Outcomes Framework (QOF)) in 2004 [15, 16]. The CDM contract ended in Scotland in April 2016 but continues in the rest of the UK. It financially incentivises CDM in general practices in the UK, through promoting creation of chronic disease registers and implementation of evidence-based care (as measured by a range of quality indicators) for CDM [17]. Unlike other incentivised CDM schemes [18–20], UK physicians can remove (except) ‘unsuitable’ patients from the denominator for any of the reasons given in Box 1, but because these reasons are seen as being beyond their reasonable control, excepted patients are not included in the calculation of a practice’s achievement against that indicator. Practice funding is reduced pro rata for exception reporting, and maximum payment is calculated through a system of achievement thresholds corresponding to points and payments for each indicator [21, 22]. Exception reporting was included in the UK contract as part of a high-trust system to ensure that practices providing a quality service do not lose out on payments through factors outside their control (Box 1) [22]. Overall exceptions are the sum total of the exceptions given in Box 1.

Exception categories 1–3 act as a safeguard against over or inappropriate intervention, or prevent practices being penalised for factors outwith their control. Category 4, ‘informed dissent’ (ID), differs because practices assign it to patients who could benefit from receiving appropriate care, but for whom the practice and patient have not managed to arrange a consultation. The incentive scheme is UK wide, but there is a difference between England and Scotland in the way ID is reported: the Scottish reporting system captures this reason under ID (Box 1) but English reporting systems include exceptions due to non-attendance after three invitations under the category ‘unsuitable’ [23]. The most recent publication describing practice-level ID rates in England was based on a subset of 37 indicators in 2008/9 [23]. The rate was 0.44%, accounting for 30.1% of exceptions overall. The criteria constituting ID in Scotland enables an examination of rates and predictors of non-engagement among all patients who are eligible to receive incentivised activities.

Exception rates quantify failed opportunities for practices and patients to engage in preventative CDM (an ‘engagement gap’) and for patients to receive appropriate care. Exception reporting is under practice control and may serve practices’ financial self-interests [24, 36].

After removing excepted patients from the denominator, practices can receive the maximum payment by achieving targets for a proportion of the remaining patients (‘reported achievement’) [17]. This highlights the scheme’s potential for a practice to achieve maximum points without covering the practice’s total population, thereby risking low uptake of preventative evidence-based interventions among hard to reach groups who may be most in need [25]. By omitting excepted patients, the reported achievement may also mask wider inequalities [26], and examining CDM performance reporting data without taking account of exceptions runs the risk of underestimating the extent of inequalities [27]. ‘Population achievement’ (Box 2) is a preferred measure of practice-level achievement because numbers achieving the indicator are expressed as a proportion of all patients, including those who are excepted [24, 28].

Exception reporting has been described as a quality marker because high rates can suggest that patient preferences are being considered [29]; however, qualitative [30, 31] and quantitative [17, 23] reports indicate that some practices use exception reporting at the end of the contract year, to help reach payment targets.

Patient-level exception reporting data have suggested that non-attendance accounts for one third of ID, and expressed dissent for two thirds of ID [32]. ID and overall exception rates are increasing over time, with predictors including increasing age and extent of deprivation [33].

Systematic reviews provide weak evidence of financial incentives reducing socioeconomic inequalities in CDM [32, 35] and health inequalities are believed to have diminished over time after the introduction of the pay-for-performance scheme [11, 34, 36]. During the initial years of the scheme, analyses suggested that the inverse equity hypothesis (which suggests that policy initiatives for health interventions initially benefit people of higher socioeconomic status and only later benefit people of lower socioeconomic status) [37] did apply [11, 38, 39]; however, we are not aware of any study examining practice-level exception data from pay-for-performance in the UK after 2009. Exception rates and population achievement over time are recommended as more appropriate measures to evaluate the impact of the programme on health equity [26, 40]. The absence of longitudinal analyses of practice-level ID, overall exception reporting rates and population achievement in the same practices using the same indicators has limited our understanding of the equity impact of the incentivised UK CDM programme.

In this study, we used longitudinal practice-level data to investigate rates, predictors and trends over time of ID exception reporting, overall exception reporting and population achievement in order to determine whether the inverse equity hypothesis holds true for the incentivised UK CDM programme.

Methods

Data

In Scottish practices, data from the incentivised CDM system (QOF) are automatically extracted from practices’ clinical computing systems and collated by the Information Services Division (ISD; part of National Health Services (NHS) National Services Scotland). Data were obtained in December 2014 after any adjustments agreed between practices and ISD, made to ensure patients who were exception reported had not subsequently been included as having met targets. ID, overall exceptions and population achievement were calculated as shown in Box 2. Patients with multiple morbidities are eligible for multiple indicators, and if they are excepted for one indicator, it is possible that they will be excepted for other indicators. For example, the indicators cholesterol and blood pressure control are relevant across many disease domains and, hence, exception reporting on these measures for a multi-morbid patient will likely be reflected across all of his/her relevant conditions. Another example relevant to numerous conditions was influenza immunisation, with patients most likely excepted due to refusal [33]. If two or more exception codes are associated with a patient, the first entered code is taken, in accordance with business rules for QOF reporting [41]. Some practices may have more patients of this type than others, so the analysis included random effects for practices to allow for clustering of exception reporting at the practice level. ID [23] and overall [17] exception reporting rates vary between practices and clinical indicators, so we included the same indicators from the same practices. To improve comparability of data across 3 years of study, we restricted our analysis to those indicators retaining the same definition for all 3 years (Table 1).

Table 1 Included indicators
Full size table

We included practices returning data for all these indicators for each year. This enabled an analysis of predictors independent of interference introduced by the indicator, practice or yearly variation.

Statistical analysis

We conducted a retrospective analysis of ID, overall exception reporting and population achievement from 1 April 2010 to 31 March 2013.

The dataset for analysis included separate observations for each of 29 indicators measured in 793 practices in three separate years (2011, 2012 and 2013), resulting in a total of 68,991 observations (793 practices × 3 years × 29 indicators). This represented a total of 15.0 million opportunities for exception reporting.

Data on the rates of ID exceptions, total exceptions and population achievement were in the form of the number of patients excepted from or achieving the indicator, and the number of patients eligible for the indicator. We fitted a mixed effects logistic regression model for the proportion of patients who met each outcome across all indicators, practices and years to determine significant practice-level predictors. All models included random effects for practices and indicators. Predictors were modelled as categorical variables and, where appropriate, as continuous variables; the results were broadly similar and the categorical results are presented. We looked at all hypothesised predictors individually and together in a best-fitting model found by backwards selection. All statistical analyses were performed with R software (version 3.0.1) using the lme4 package [42]. Results are presented as odds ratios (OR) compared to the reference category with 95% confidence intervals (CI) and an overall P value for the differences between the categories. Any individual patient could contribute to several indicators; however, the unit of analysis was the practice-indicator-year combination so a single patient contributed only once to each unit of analysis. Data summaries show the number of exceptions rather than the number of patients excepted. Our analysis combined indicators and therefore we report exceptions rather than individual patients.

We obtained both practice-level and indicator-level predictors to investigate factors associated with ID exception, overall exception and population achievement rates. These factors included the percentage of the practice population living in an area designated as Scotland’s top 15% most socioeconomically deprived; the mean age of the practice population; practice list size; patients per general practitioner (GP); number of GPs; mid-points of GP age bands; degree of rurality of the practice population; the percentage of patients who had not achieved each indicator in the contract year; and whether the achievement threshold for each indicator had increased during the 3-year period (April 2010 to March 2013). For each continuous variable, cut-offs were chosen to give sufficient numbers for analysis in each subgroup, ideally with approximately equal numbers in each subgroup. The characteristics of the practices varied from year to year; Appendix 1 shows the distribution of the predictor variables in 2010/2011. We describe the unweighted proportion of exceptions for each indicator separately and for all clinical domains combined [11, 17, 23]. The mixed effects logistical regression model used the number of exceptions as the numerator and the number of eligible patients as the denominator, so the analysis is appropriately weighted according to the number of patients for each indicator in each practice. We provide the median, which does not take account of weighting. However, fully weighted percentages are available from the corresponding author on request. Results of dependent variables are expressed as a combination of all 29 included indicators.

Results

We included data from 793 practices that returned exception reporting data for all 29 QOF indicators retaining the same definition throughout the 3 years (April 2010 to March 2013) of the study.

Three-year median rates of informed dissent, total exception reporting and population achievement

Table 2 shows the clinical indicators categorised by disease, with the median percentage ID rates, overall exception rates and population achievement from 2010 to 2013.

Table 2 Informed dissent exceptions, overall exceptions and population achievement rates (median %) for 2010–2013
Full size table

Eight clinical domains included more than one indicator: blood pressure (BP), chronic obstructive pulmonary disease (COPD), diabetes mellitus (DM), stroke, coronary heart disease (CHD), primary prevention of heart disease 01 (PP 01), epilepsy and chronic kidney disease (CKD).

Over the 3-year period, median rates of ID increased for 18 indicators, remained unchanged for seven (for which all values were 0 across all 3 years), and decreased for four (CHD indicators and depression (DEP), which was dependent on the CHD indicator). ID exception reporting rates trended in the same direction over time for each indicator within six domains: BP, COPD, DM, stroke, epilepsy and CHD. The highest rates (>1%) applied to indicators within the BP, COPD, DM, stroke and CHD domains. Grouped together by disease, the highest median ID rate was noted for patients with COPD followed by DM. Increases in the reported rates of ID between 2010 and 2013 were modest, for example, a 20% increase for the indicator DM 10. The median ID rate across all practice-indicator-year combinations was 0.86% (interquartile range (IQR) 0.0–3.23%).

Overall exception reporting rates increased for 18 of the 29 indicators over the 3 years, with decreases for the remaining 11. The median overall exception reporting rate was 6.98% (IQR 2.63–13.30%) across all indicators, practices and years.

The population achievement rate for the composite of indicators remained stable in each of the 3 years of analysis. The rate was 83.5% (IQR 75.1–90.7%) across all the practice-indicator-year combinations. For those indicators appearing more difficult to achieve, for example, epilepsy and COPD, corresponding rates of ID were either low (epilepsy 06, 0% for all years) or raised (COPD 10, median rate 4.42% in 2012/2013), suggesting that differences in the level of difficulty achieving the indicator were caused by the nature of the indicator or the ability of patients to attend their practice to receive the indicator. Higher rates of median achievement appeared to correspond with higher rates of ID exception reporting.

Predictors of informed dissent exception reporting

Table 3 illustrates the associations between hypothesised practice-level predictors and ID. The table shows ORs from models for the combination of all 29 indicators, across 793 practices, over 3 years as univariate analyses and also after controlling for confounding variables.

Table 3 Independent predictors of 3-year Quality and Outcomes Framework indicators for informed dissent
Full size table

Univariate analyses show ID increased over the 3-year period, and followed a socioeconomic gradient with higher median rates in the most deprived practices. ID was higher in practices with younger populations than in practices with older populations. Median rates increased as practice list size increased; practices with more patients per GP had higher median levels. Large urban practices reported higher rates of ID compared with rural practices.

A model-selection procedure identified a subset of five independent predictors associated with ID in the best-fit model: year of reporting, practice-level deprivation, list size, urban/rural practice location, and extent of indicator non-achievement. The odds of ID exception reporting for patients with at least one of the specified 11 diseases in 2012/2013 was greater than in 2010/2011 after controlling for other predictors (OR 1.16, 95% CI 1.15–1.17, P < 0.001). The most deprived practices were more likely to report ID compared with least deprived practices (OR 2.22, 95% CI 1.91–2.58, P < 0.001). Practices with the largest list sizes were also more likely to report ID (OR 1.89, 95% CI 1.43–2.51, P < 0.001). Significant differences were also seen with practice location (odds of ID reporting were 31% lower in rural areas) and when there were higher rates of QOF non-achievement in the same year: the odds of ID reporting by practices were over 20% lower in lower-achieving practices (practices with higher levels of non-achievement). Practices with higher achievement had higher rates of exception reporting.

Predictors of overall exception reporting

A model-selection procedure identified a subset of independent predictors of overall exceptions (Table 4) that were similar to those found with ID exceptions. These included further evidence that practices achieving higher points had higher levels of exceptions and that more deprived practices, urban practices, higher list sizes and the progress of time were also associated with higher exceptions.

Table 4 Independent predictors of 3-year Quality and Outcomes Framework indicators for overall exceptions
Full size table

Predictors of population achievement

Following a model-selection procedure, Table 5 shows that practice-level deprivation was independently associated with low population achievement: the lowest odds of achievement were associated with the most deprived practices (OR 0.84, 95% CI 0.80–0.87, P < 0.001).

Table 5 Analysis of 3-year Quality and Outcomes Framework indicators for population achievement
Full size table

Discussion

Our longitudinal analysis of practice-level data from 793 Scottish general practices serving 4.4 million patients showed that more deprived practices were at least twice as likely to report ID compared with the least deprived practices (OR 2.22, 95% CI 1.91–2.58, P < 0.001) and 32% more likely to report overall exceptions. Of all the predictor variables included in each of the regression analyses, deprivation was the strongest predictor (based on ORs) in every regression model. This result suggests that there is a lower uptake of appropriate clinical care for CDM that aims to prevent further disability and mortality for those most in need of preventative and ongoing health service intervention.

Rates of ID and overall exception reporting increased over time. High achievement (contract points and therefore remuneration) was associated with higher rates of ID and overall exception reporting, re-opening the possibility of gaming, that is, those who use the exception reporting rules more often achieve higher QOF achievement, or suggesting that higher-achieving practices are more adept at identifying and reporting exceptions.

In three recent CDM contract years, the year of reporting, increased deprivation, an urban location and large practice size were predictive of ID, overall exception reporting rates and population achievement. Contrary to previous findings based on the early years of the contract [11], our findings suggest that the inverse equity hypothesis does not hold, because practices in more deprived areas are more likely to show higher rates of exceptions, implying that more patients are missing out on CDM opportunities. Our study raises the possibility that the incentivised CDM programme, recently curtailed in Scotland but continuing to operate in the rest of the UK, may be widening the health equity gap by not directly incentivising practices to use methods of engagement other than letters, for example, house visits or assertive outreach. Given the central importance of primary care, the finding of a lower uptake among patients with at least one long-term condition who are living in more deprived backgrounds suggests the UK contract may not lead to an increase in health status, or not act in such a way as to decrease the higher use of accident and emergency facilities, emergency admissions and out-of-hours clinic use seen in patients from more deprived areas [43]. Practice-level CDM contract payment mechanisms do not take into account patient-level multi-morbidity or an individual’s other circumstances, for example, their ability to travel to their practice for multiple appointments [24, 44, 45]. Our data raise the possibility that the persisting ill health [46] and excess mortality [47] found in patients in Scotland with incentivised chronic diseases from more deprived communities may in part be explained by the lower uptake of evidence-based interventions, particularly in conditions associated with multi-morbidity and limitation of daily activity, for example, COPD. Other reasons for consistently high rates of non-attendance among indicators in patients with COPD or diabetes are not currently known. Some possible reasons may include patient unwillingness to attend appointments due to the number of repeated requests for attendance, or the duration of individual appointments. Transport difficulties, such as the cost of bus fares, or an inability to secure paid time off work to attend could also be contributory. Overall exceptions appeared higher in COPD, diabetes and stroke, which may be explained in part by the contribution made by higher ID rates (COPD and diabetes). Overall exceptions are calculated by the addition of the four categories described in Box 1. Therefore, high levels of 'Unsuitable', 'Registration date and diagnosis date' and 'Other' may account for higher overall exceptions.

Together, these data provide additional evidence [48, 49] that the inverse care law continues to operate in Scotland, despite over 10 years of the incentivised UK CDM contract designed to address health disparities.

Implications for clinicians

Experience from other pay-for-performance systems suggests that patients from some minority groups have difficulty attending appointments and re-appointments due to problems with transport [50, 51, 53], poor understanding of recommendations [54] or costs [55]. Our findings underscore that tailored engagement approaches (e.g. phone calls or text reminders, or through house visits for housebound patients) may be merited for patients who are less likely to attend the practice [52]. Tailoring invitations, for example by offering flexible appointments or using methods other than re-sending invitation letters three times as stipulated in the CDM contract, is known to increase the uptake of general practice-based preventative care in ‘hard to reach’ groups in more deprived areas [52–56]. While individualised forms of engagement may require additional administrative time, effort and funding [45], practices are free to adopt new methods of engagement because the CDM contract does not constrain practices’ engagement approaches. More research is needed to identify methods to optimise engagement in deprived areas, given the growing evidence of the unintended consequence of severely ill patients being excluded from care [33, 57, 58].

Implications for policymakers

UK primary care is seen as having a key role in improving clinical outcomes, particularly in deprived areas [16]. The incentivised CDM programme was seen as one mechanism for decreasing variation in the standards of healthcare and for potentially lessening inequalities in care provision [15, 16, 59, 60]. Our findings suggest there are socioeconomic inequalities in the uptake of best practice in clinical care for CDM and that the inverse equity hypothesis does not hold, contrary to previous findings [11, 34, 61, 62]. This reinforces calls to revise pay-for-performance programmes for CDM in a way that directly rewards reductions in inequalities [34, 63, 64], for example, by incentivising practices to reach patients who have more to gain but are less likely to attend their practice [76]. A revision of incentives based on need [47] or appointment systems, and increasing the skill mix in general practice to share the responsibility of managing chronic diseases [65], for example, by introducing practice-based, independent prescribing pharmacists who could visit patients in their homes, may help address the increased workload contributing to the underlying shortage of GPs.

As a result of the higher uptake of preventative healthcare in more affluent populations, international health improvement policies have recommended the use of targeted, preventative primary care [66–68], for example, using outreach workers to engage ‘hard to reach’ subgroups [69, 70]. Given the likely variation in invitation methods and associated successes or failures, contract revisions could include encouragement or incentivisation for practices to assess the acceptability and effectiveness of different engagement methods and use the method most suited to their patients. A comparison between UK and Californian programmes suggested that increasing financial rewards did not address health inequities: a revision of design and implementation was seen as more appropriate [18].

Strengths and weaknesses of the study

We present findings from the majority of Scottish practices and describe, for the first time, practice-level time trends and factors predicting non-attendance due to ID among patients eligible to receive the UK’s incentivised CDM programme in Scottish primary care. The recording of non-attendance and refusal as ID for all 29 indicators over three recent contract years enabled us to evaluate these factors independent of changes to indicator definitions and to avoid selection bias. We accounted for changes to thresholds in our analysis, because of the finding that threshold changes influence exception rates [71].

The study has several weaknesses. Reporting systems are unable to differentiate ID non-attendance due to expressed refusal or when no reason is given. This limited our ability to make inferences about the extent to which patients chose not to attend rather than lacked the capacity or capability to attend, or did not receive an invitation, for example due to having moved house. Absolute rates of exceptions may be conditional on the indicators included because exception rates vary by indicator. However, this would not be expected to affect observed differences over time with the same indicators. In addition, the UK CDM contract rules only record the first exception reporting reason encountered in the information returned by practices, which may underestimate the true rate of ID. Participation in the pay-for-performance contract is voluntary, and there are differences in, for example, practice size and rurality, between practices participating and not participating in the UK pay-for-performance contract [77, 78]. Therefore, our conclusions, as with other practice-level analyses (Appendix 2) based on 83% of participating practices, may be limited to those participating in the UK pay-for-performance contract.

Our conclusions may not apply to other settings, such as those with market-based healthcare systems or low-income countries, and we did not consider dimensions of inequality such as ethnicity or gender, due to a lack of patient-level data.

Comparison with existing literature

Median rates of ID and overall practice-level exception reporting were higher than those found in previous studies using data from English practices (Appendix 2). However, our ID results are not directly comparable because CDM programme reporting systems in England do not routinely differentiate ID from other exception reporting reasons, and include cases of ‘non-attendance after three invitations’ as ‘clinical – unsuitable’. Previous studies have not examined the same indicators over time, have included data from only one contract year, have not considered population achievement or have only examined data prior to 2010 [11, 17, 23, 33, 36], thereby limiting understanding of the inverse equity hypothesis in recent years of the UK CDM system.

Prior to the 2005 longitudinal study by Campbell et al. [72] and following the introduction of the incentivised CDM programme [11], analysis of practice-level data showed that the socioeconomic gradient in care quality narrowed, as anticipated by the inverse equity hypothesis [37]. However, studies used practice-level reported achievement without considering time trends in ID, overall exceptions or population achievement [10, 11, 38, 72]. Therefore, previous results may have masked underlying inequities [34].

Our practice-level findings are supported by patient-level data from other studies, which have suggested worsening of health disparities under the CDM contract through the exclusion of already disadvantaged groups, such as ethnic minorities with diabetes [73], patients with co-morbidities [32], and those at higher risk of diabetes complications [26, 74].

By using longitudinal data containing practice-level ID and overall exceptions together with population achievement, our study addresses calls from a recent systematic review [34] and adds further weight to emerging evidence that the incentivised system for CDM in the UK has not diminished inequalities [33, 34, 75]. More research is needed to explore the typology and engagement patterns of patients who are serially not engaging in incentivised CDM programmes so that approaches to engagement can be individualised.

Conclusions

Overall and ID exception reporting rates have increased in Scotland. Rates of exception reporting including ID are higher in practices receiving higher remuneration, which may reflect practices being more adept at detecting and recording exceptions to avoid missing out on payments. Overall and ID exception reporting is higher in practices with greater numbers of patients from socioeconomically deprived areas. If, as would seem likely, ID rates represent non-engagement with CDM appointments, this would imply that ‘hard to reach’ patients are receiving suboptimal care. These results suggest there are widening socioeconomic and other inequalities and confound the inverse equity hypothesis. The CDM scheme is unlikely to be an example of a truly equitable public health intervention, suggesting a need for policymakers who are currently revising the scheme, or others contemplating such schemes in other countries, to consider including incentives for engaging with the 'hard to reach' who are most at risk of adverse outcomes if reducing health inequalities is deemed important.