Background

Wide variation in the provision of evidence-based care is recognized as a fundamental issue in all health care systems worldwide. The consequences of such variation often impact negatively on patients in terms of health care quality, safety, experiences and increased financial costs (including additional treatments and litigation) associated with sub-optimal clinical practices [1, 2].

It is widely accepted that there is a need to minimize unnecessary variation to improve the reliability of best practice care provision and the associated financial costs [2–4]. Practicing evidence-based medicine and implementing clinical care guidelines are promoted to assist clinical decision making and optimal management of patients, but does not necessarily ensure that patients who should receive all appropriate care actually do so [5–8].

Around 50% of hospital patients may receive the full recommended care and treatments which their clinical condition merits. The difference between the highest and lowest performing health care systems suggests that there is ‘an enormous gap’ in evidence-based (or recommended) care provision [1, 9]. In primary care settings, evidence of wide variation has also been found between individual health care providers [10]. For example, in general medical practice in the United Kingdom (UK) substantial variation in patient care has been described in anxiolytic, hypnotic, antidepressant and antibiotic prescribing [11–13].

The Quality & Outcomes Framework (QOF) is a pay-for-performance (P4P) scheme that was introduced to UK general practice in April 2004 to help address longstanding variation in the quality of primary care provision [14]. Clinical conditions are suitable for QOF inclusion and therefore financial incentivisation if they are common, associated with significant morbidity (and to a lesser extent mortality), and are diagnostically unambiguous. Indicators should be evidence based, achievable by every primary care team, clearly defined and consistently extractable from different computerized information systems [14].

Since 2009, QOF indicators have been developed through a rigorous National Institute for Clinical Excellence (NICE) led process which includes input from an expert panel and extensive piloting [14]. The Framework consists of a number of incentivized ‘point-in-time’ indicators arranged into four main groups: additional services, patient experience, organizational and clinical sections [15]. Practices ‘earn’ points according to their level of achievement for each indicator, with payment currently starting at a minimum threshold (usually 40%) rising to a maximum (usually 90%).

The average achievement of available QOF points for the period April 2009 to March 2010 was 93.7% in general practices in England and 97.2% in Scotland [16, 17]. The implication is that the quality of care delivered by practices to patients with incentivized disease conditions is very high. However, there is some concern that maximum payment thresholds for QOF indicators are actually too low and that the high performances achieved by most practices may give the inaccurate impression that care quality does not necessarily need to be improved further [18, 19].

There is growing interest in the use of composite – as well as individual - measures of care delivery as an alternative method of describing the quality of clinical care processes and outcomes [10]. The main benefit of the composite (‘all or nothing’) approach is that it may highlight opportunities for further improvement in care provision even when individual measures already indicate that care quality is high.

The care bundle concept is one such composite measure that is promoted as a systematic method of monitoring and improving the reliability and quality of health care [20–22]. A care bundle is simply a number of health care interventions grouped together and which normally have a synergistic relationship that impacts on clinical outcome [23]. Bundles usually contain three to six components which may include clinical interventions such as care processes, procedures, or diagnostic tests, but are not deemed suitable to act as comprehensive lists of all possible care. Selection of appropriate bundle components is based on best evidence, local considerations and may change with time and experiences [24, 25].

Specific care bundles have been implemented in a range of secondary care settings such as paediatric and adult ICU, medical and surgical wards and Accident and Emergency departments in North America and the UK [26, 27]. Related clinical outcomes have included significant reductions in health care acquired infections, condition-specific and all-cause mortality, and reduced re-admission rates of elderly patients, length of ICU stay and number of ventilation days [20, 28–30]. Although higher compliance rates with bundles are associated with improved outcomes [31], these are difficult to sustain because of a combination of system and human factors which often results in rates below 50% [32–34].

Measuring compliance with bundled interventions on a composite ‘all-or-nothing’ basis may provide the healthcare team with a more accurate indicator of care quality and evidence-based care provision [35]. In essence this means every relevant care component should routinely be delivered (or considered) for every single patient on time and every time. Embracing this rationale may act as a greater prompt to improve patient care than the current method of monitoring data with individual bundle elements, which can give a misleading impression of overall performance.

Evidence of the potential value of composite measures of care quality in general practice – specifically QOF data - is limited [36, 37]. Given the benefits of the bundle approach in acute hospital settings, we aimed to develop care bundles based on those QOF disease areas and indicators in general practice that were judged to be most suitable. We further aimed to measure individual and composite compliance with developed care bundles to highlight the extent of any potential care ‘gaps’ which may point to opportunities for further improvement.

Methods

Design of care bundles as proxies for composite measures

The condition-specific care bundles and associated individual components were selected from the ‘Quality and outcomes framework guidance for GMS contract 2009/10’ by the authors [15]. Studying QOF data was a pragmatic choice, as it provided a measure of current evidence-based clinical practices. We agreed selection criteria to help identify potentially suitable clinical conditions and indicators according to previous published guidance [24] and local practical considerations. All QOF sections and indicators were initially considered. The selection criteria (Table 1) were applied in a step-wise manner as illustrated in Figure 1 to exclude unsuitable indicators and conditions.

Table 1 Care bundle selection criteria
Full size table
Figure 1
figure 1

Step-by-step process to select suitable clinical conditions (care bundles) and QOF indicators (bundle components).

Full size image

Measuring care bundle compliance

Setting and sample size

We emailed a convenience sample of 14 West of Scotland general practitioners (GPs) who are also senior medical educators based in the west region of NHS Education for Scotland (NES) with details of the proposed study during February 2011. We invited them to discuss the invitation with their practice teams and indicate to us a preparedness to participate.

Data collection

On the 31st March 2011 the practice manager in each participating practice identified and downloaded the relevant QOF data requested (using guidance provided) from local information systems to an encrypted USB mobile data device. This device was then transported by the GP educator to the regional office for analysis by the research team. The 31st March is the date on which recorded performance by the practice is formally measured against QOF criteria and associated annual earnings calculated. Practice teams work towards this date all year to record and achieve the maximum patient care as all indicators are automatically ‘reset’ on the following day - 1st April.

Practice managers were asked to indicate the total number of patients registered on their practice list and on each disease register. We decided to code ‘exception reported’ indicators as if they had been met - in other words that the specified care was ‘delivered’ to patients. We based this premise on the fact that exception reporting has historically accounted for ≤ 1.6% of all QOF costs [38] and we assumed that the suitability of the care described by an indicator would have been considered (if not delivered) for each patient during the exemption process.

Statistical analysis

Data were uploaded to and coded in Microsoft Office Excel 2007. All patient and practice identifiers were removed immediately, with practices being coded to enable feedback to be provided. Descriptive statistics were performed using SPSS v.17.0 to calculate each practice’s compliance as a percentage for the selected QOF indicators (bundle component) and the composite compliance for the selected clinical conditions (bundles). Overall compliance with care bundles (composite measures) and their components were calculated after adding the patients on each disease register of all the participating practices together.

Results

Care bundles (composite measures) and indicators

Five QOF clinical conditions fulfilled the pre-specified inclusion criteria: Stroke/TIA, secondary prevention of coronary heart disease (CHD), diabetes mellitus (DM), chronic kidney disease (CKD) and chronic obstructive pulmonary disease (COPD). The 24 retained indicators are shown in Table 2.

Table 2 Retained clinical conditions and QOF indicators (care bundles and components)
Full size table

Response rate and demographics

A total of 12/14 practices agreed to participate. Nine practices were able to provide data in the format requested to allow meaningful analysis (64.3%). The combined patient population on the 31st March 2010 was 56,948. The prevalence of the selected clinical conditions in all practices was: CKD 3.4% (range 2.5% to 4.9%), stroke/TIA 2.2% (range 1.7% to 2.7%), diabetes mellitus 4.2% (range 3.6% to 5.5%), CVD 4.6% (range 4.2% to 6.2%) and COPD 2.3% (range 0.8% to 3.8%). Further demographic and disease register details of participating practices are shown in Table 3.

Table 3 Individual indicator and composite measures of patient care for selected clinical conditions by general practice for the period 01/04/2010-31/03/2011
Full size table

Composite measures and individual indicator compliance

The minimum and maximum payment thresholds for the selected indicators ranged from 40-50% and 60-90% respectively. All practices achieved the minimum threshold for all indicators and the vast majority also achieved the maximum threshold for most indicators.

The composite measures of the selected conditions (care bundles) varied for all practices and were as follows: CKD 69.0% (range 64.0-81.4%), stroke/TIA 74.1% (range 51.6% to 82.8%), Diabetes mellitus 58.4% (range 50.3% to 65.2%), CVD 64.0% (range 47.8% to 71.9%) and COPD 82.0 (range 47.9% to 95.8%). Care bundles with more components such as DM and CVD generally had lower compliance rates.

The diabetes mellitus care delivered by all practices as specified by individual indicators ranged from 82.9% (DM24) to 98.5% (DM15), yet overall care bundle compliance was substantially lower at 58.4%. In other words, 58.4% of all eligible patients received all the care specified by the bundle and 41.6% of patients did not receive one or more care components. For CVD care, individual indicator achievements ranged from 82.6% (CVD10) to 99.0% (CVD11) with 64.0% of patients receiving all specified care. The COPD care delivery was the most reliable, with 82.0% of patients receiving all components. However, overall the results suggest that the composite bundle approach highlights a greater level of variation within and between practices in evidence based care delivery and arguably therefore provides a more informed and discriminatory measure of care quality than the current, largely homogenous approach. The variation in the individual indicator and composite measures for each clinical condition and practice can be observed in Table 3.

Discussion

Main findings

This small study is the first known attempt to measure composite care delivery for specific clinical conditions in UK primary care using a care bundle approach. We identified five suitable clinical conditions for care bundles: COPD, CKD, DM, TIA/Stroke and CHD. Three to eight components were chosen and adapted from QOF indicators for each bundle. Compliance with individual bundle components was generally high, but substantially lower overall for the ‘all or nothing’ bundles. We would suggest that this ‘gap’ in performance may be a more valid reflection of the variation in expected care delivery at the patient level than current individual measures presently indicate.

The ‘gap‘ between component and care bundle compliance

There are at least five possible reasons for the observed ‘gap’ between individual component and overall bundle compliance. Firstly, the Framework financially incentivizes an individual indicator approach so this becomes the practice focus. Secondly, some components are easier to deliver than others. Thirdly, care bundle compliance (the composite measure) will always have a value equal to or lower than that of the indicator with the lowest performance - consequently there will always be a gap, although it is unknown whether or when the size of the ‘gap’ becomes of clinical significance. Fourthly, the ‘gap’ will usually increase exponentially with the rise in the number of bundle components. Finally, natural variation is inherent in all health care systems and will be affected by the effectiveness and efficiencies of local processes and team efforts.

Variation in care is introduced ‘naturally’ through different and often complex patient presentations and ‘artificially’ through individual clinician differences and priorities, extenuating circumstances and local systems. ‘Patient characteristics’ and ‘deprivation’ have previously been identified as the two factors which are strongly associated with variation (‘the gap’) in primary health care settings in Scotland [12], England [11, 13, 39], the Netherlands [40], Australia [41] and the USA [42].

The care ‘gap’ between components and care bundles in our study was observed for all clinical conditions and participating practices. This ‘gap’ (variation) may be at least partially accounted for by hidden patient and deprivation factors. The implication may be that the care bundle method may measure the gap but could not necessarily help to eliminate it.

In UK secondary care settings, reported compliance with a variety of clinical care bundles ranges from 19-52% [32, 33]. Low compliance rates have important patient safety implications, as a positive and significant association has been found between compliance rates and clinical outcomes such as mortality [27, 28, 32]. A similar association has not yet been shown for primary care. Although the participating practices’ compliance with all the selected bundles was ≥ 58% it is not known whether compliance can be compared meaningfully across settings and for different care bundles. It would also be desirable to determine to what degree different compliance levels may impact on clinical outcomes in primary care settings.

The QOF and care bundles

We adapted care bundles from the QOF, currently the most ambitious, comprehensive and largest P4P scheme and quality measure in international healthcare and a ‘natural experiment in progress’ [43]. Framework research and experience may therefore be applicable to our study findings. For example: the Framework can cause a ‘street lamp effect’, i.e. neglect of health care activity and clinical conditions which do not attract financial incentives; it can create a de-personalizing ‘box ticking culture’; it is vulnerable to data distortion and potential gaming; it is accelerating a transition to nurse-led primary care; there is tension between the different QOF roles as a quality improvement method, regulatory framework or remuneration mechanism; the Framework promotes simplicity over complexity and measurability over meaningfulness. These valid concerns could similarly be raised about the care bundle method, with no clear answers at present [19, 43–46].

The impact of QOF on clinical outcomes continues to be debated. There is no evidence of any ‘discernable effect’ on hypertension-related outcomes [47], while new depression indicators failed to improve disease detection or treatment [46]. In addition, associations between QOF measures and all-cause mortality and emergency admissions have been found to be ‘small and inconsistent’ [39]. On the other hand, evidence of improved care quality for patients with specific chronic diseases and a ‘trend towards improvement’ in process and outcome at least partly attributable to QOF have recently been reported [44, 48–50]. It is unclear what impact (if any) surrogate measures such as care bundles would have on clinical outcomes, how this could be feasibly measured and what period of time would be required.

‘Time-interval’ indicators have been proposed as an alternative and more reliable measure to ‘point-in-time’ indicators to assess care quality [51]. Performance is evaluated over a specified period rather than at a single point in time. Mabotuwana et al applied this approach to hypertension management and found it feasible but with lower measures for ‘interval indicators’ compared with point-in-time indicators. The differences, similarities and potential benefits of any one method, whether care bundles, time-interval or QQF indicators are unclear.

Care bundle implementation

The care bundle method may be a useful new care quality measure and help to reduce healthcare variation. It may also help to better differentiate the care quality of practices, given that their current QOF scores are now broadly comparable. To implement the bundle a number of challenges would have to be overcome, for example: resistance from practices who are financially disadvantaged by ‘lower’ measures than the current thresholds; accounting for ‘natural’ variation so that practices are not unfairly penalized; existing information technology systems would have to be redesigned; a full evaluation to identify both impact and unintended consequences (e.g. a proportion of patients may end up receiving ‘no care’) would be necessary. If this new measure is considered desirable, it would have to be promoted and incentivized.

Reducing variation in quality can decrease costs if the care ‘gap’ is large, but costs increase as the gap narrows until there is a net expense [52]. Evaluation of care bundle implementation in some secondary care settings has found them to be cost-effective [53]. However, implementing interventions require initial financial and resource investment. McNeill et al found that only 20/265 (12%) of acute medicine units in the UK had the minimum facilities to comply with the ‘surviving sepsis care bundle’ [34]. However, the applicability of care bundles in acute settings cannot be assumed to translate to QOF-bundled components because care packages over 9-15 month periods.

The financial implications of quality-reporting programmes vary greatly. The QOF ‘probably’ represents value for money [54]. Adapting the Framework’s reporting system to allow composite quality measures would require substantial investment and feasibility and acceptability concerns would have to be addressed. Known incentives that facilitate increased practice participation include: financial payments, staff training and providing technical support [54, 55].

Implementing care bundles and increasing compliance with them relies not only on individual health care professionals but even more on the availability of adequate resources, support systems and leadership. Aligning care bundles with larger quality improvement initiatives and providing related training also appears to be an important factor influencing success and impact [28]. These issues need to be considered if the care bundle approach is to be successfully implemented in primary care settings.

Exception reporting and threshold ceilings

The threshold ‘ceiling’ for maximum financial reward is set at ≤ 90% achievement. Practices could hypothetically ‘stop’ delivering care once they reach these thresholds without financial penalty. Our findings indicate that many indicators with relatively low achievement ceilings were still performed for the vast majority of patients which seems to suggest that thresholds are an unlikely reason for the observed ‘gap’.

Exception reporting has previously been found to be consistent across practices and regions and accounts for a tiny minority of cases [38]. Concerns that practices may be ‘gaming’ results through excessive or inappropriate exception reporting have not been substantiated. However, exception reporting combined with less than 100% maximum threshold targets introduce an incentive ceiling [18].

Strengths and limitations

Our findings are based on a small, convenience sample of practices. We did not take into account patient list size, geographical distribution (degree of remoteness), socio-economic class and level of deprivation or prevalence of disease. Our findings may still have wider application, as different degrees of remoteness, access and practice size do not significantly contribute to variation in care quality as measured by QOF results [56, 57]. A recent review found that inequalities in chronic disease management have largely persisted after introduction of QOF (even though quality of care may have increased) [58]. While deprivation is a major contributing cause to overall care, it has not been shown to affect QOF scores. For example, Hamilton et al found that the impact of QOF on diabetes mellitus care was comparable in affluent and deprived areas [59].

We decided to code exception reported indicators as if the described care had been delivered. The consequence was a ‘smaller’ compliance ‘gap’, but one that may have more acceptability to frontline staff. Exclusion systems have an important role to help militate against the potential impact of socio-economic, patient and other factors out-with the practice team’s control. Simple care processes are mitigated more than complex ones and the additional work required in deprived areas is not always being rewarded [60]. We therefore purposefully selected ‘simple’ care bundle components in preference to ‘complex’ ones where possible. We accept that there are many other potential care bundle topics for general practice. However we chose a pragmatic approach to give an initial indication on the feasibility of the concept as applied to QOF data. Others may have chosen alternative approaches and clinical topics for bundles.

Conclusions

Based on our small study, we believe that the care bundle approach to QOF data has the potential to provide a more insightful measure of the quality of evidence based care provision than the current approach which focuses on compliance with individual indicators rather than individual patients. However, issues around the feasibility and acceptability of the implementation of this method as part of routine general practices need to be explored.

Authors‘ contributions

CdW led on the study design and data collection, undertook statistical analysis and interpretation, and drafted the initial manuscript. JM contributed to study design, data collection and critical review of the manuscript. PB conceived the study idea, acquired funding and contributed to the study design and the drafting and critical review of the manuscript. All authors read and approved the final manuscript.