FormalPara Key Points for Decision Makers

The evidence to assess efficiency and productivity of general practices is currently weak because prevalent approaches to measure output do not consider the wide range of general practice services and their value in terms of health improvements.

We propose a multi-dimensional framework and indicators of valued output from the healthcare decision-maker’s perspective.

The framework provides a tool to measure the valued output of primary care and better understand the value for money of resource allocation decisions.

1 Background

Healthcare services worldwide are under severe pressure due to the increasing demands from a more complex patient population set against constrained and, in many cases, diminishing resources. The strain facing healthcare systems is particularly acute in English primary care [1]. Even before the COVID-19 pandemic, general practices were struggling to meet demand for primary care services due to a growing population characterised by increasingly old and multi-morbid patients [2]. In parallel, general practices in England have been experiencing a workforce crisis with the number of general practitioners (GPs) per capita declining since 2009 [3] due to challenges in retention and recruitment.

In 2016, the National Health Service (NHS) England’s General Practice Forward View made several commitments including increasing investment in general practice and recruiting at least 5000 additional doctors by 2020/21 [4]. Since 2019, additional funding has been allocated to Primary Care Networks (PCNs) as groups of neighbouring general practices to diversify their skill mix through additional staffing roles including clinical pharmacists, social prescribing workers, physiotherapists and paramedics [5].

Expansions and reforms of the primary care workforce are crucial because general practices rely heavily on their human resources to deliver effective healthcare. Understanding the impact of these changes on efficiency and productivity is key to ensure that general practices can meet the needs of a growing population with increasingly demanding and costly healthcare needs while easing the current pressure.

Efficiency and productivity are interrelated concepts describing how well healthcare systems can convert their limited resources into healthcare activities and, ultimately, health. Cylus et al. refer to this process as the production of healthcare outputs from inputs [6]. In this process, financial, capital and labour resources are examples of the necessary inputs to deliver healthcare activities, i.e., the ‘physical’ output. The patients’ outcomes resulting from these activities are considered the main ‘valued’ output of the production process, as health improvement is regarded as one of the main socially valuable outcomes of healthcare [7]. More in general, the notion of ‘valued’ output may encompass a wider set of outcomes reflecting the achievement of the healthcare system’s goals that are valued by society, such as improved equity in health and healthcare access.

A suitable measurement of the ‘valued’ output is key to identify best practice in primary care efficiency and to assess the impact of recent reforms on productivity. In fact, healthcare episodes are an intermediate product of healthcare, which, in isolation, may provide an inconclusive picture of the value produced by primary care. For example, the impact on patients’ health outcomes will vary according to the types of healthcare services performed; or the same service will generate varying levels of health improvement depending on the quality of its delivery.

However, a recent systematic review of efficiency studies in the economic literature found that the measurement of primary care output is often limited to the volume of consultations and other diagnostic or medical procedures performed (i.e., the physical output) [8]. Quality adjustments are undertaken by means of process-related variables, such as indicators of care continuity, patient access, or provider’s experience. Few studies have considered the ‘valued’ output of primary care in terms of impact on patients’ health. Further, the differentiation across output dimensions has been crude, failing to represent the diversity of primary care services, illnesses and conditions addressed. These limitations in measuring primary care output are partly due to the holistic nature of primary care as a service providing long-term, continuing and generalist support to patients, and the related challenges in defining the boundaries of primary care output [9].

More systematic attempts to articulate the process that underpins the achievement of healthcare quality and improved health outcomes have been undertaken in the health services literature through frameworks of performance and effectiveness. These frameworks are typically based on the Donabedian’s ‘structure-process-outcomes’ model [10], which stresses the sequential relationship between the three domains to guarantee high healthcare quality and positive health outcomes. For each domain, these frameworks provide a comprehensive array of dimensions to assess performance and effectiveness. These frameworks could support a more systematic and meaningful measurement of primary care output when their dimensions are consistent with the chosen definition of valued healthcare outcomes.

This paper presents the results of a mixed-method research aiming to address the limitations of the economic literature on efficiency in defining primary care output. To this end, we built on existing frameworks of primary care performance identified in the health services literature, and on established clinical practice standards in England, to develop a multi-dimensional framework and indicators of primary care ‘valued’ output. Our framework defined ‘valued’ output from the perspective of the healthcare decision-maker in England, whose aim is maximising the health of the population, given a constrained set of resources. We focused on primary care delivered by general practices, which are the bodies delivering the majority of primary care services in England.

2 Methods

2.1 Development of Valued Output Framework

The development of the framework of primary care ‘valued’ output consisted of three steps: (1) identifying and reviewing existing primary care performance frameworks; (2) synthesising the eligible dimensions of valued output from the perspective of the healthcare decision-maker in England; and (3) developing indicators of primary care valued output.

The search and identification of existing primary care performance frameworks was completed in the second quarter of 2020. To identify relevant performance frameworks, we conducted a targeted search within a subset of the literature identified by the authors as part of a systematic literature review on primary care efficiency [8]. The original search was performed on databases of the economic and health service literature and relied on broad search terms around efficiency and performance measurement. To narrow down the pool of relevant results, we used multiple combinations of terms including ‘framework’, ‘assessment’, ‘measure’, ‘performance’ and ‘effectiveness’. Studies in the literature were deemed eligible if they were published in the period 2015–2020, to ensure maximum relevance to current primary care institutional arrangements (i.e., the introduction of Primary Care Networks, announced in 2014 and implemented in 2019 [11, 12]); included a multi-dimensional framework of primary care performance; were relevant to the whole or a considerable subset of primary care; and were relevant to England based on similarities across primary care functions (e.g., gatekeeping), financing (universal or almost-universal health coverage) and settings (e.g., high-income).

The extant frameworks in the selected studies were then assessed to synthesise the eligible dimensions of valued output. Eligibility was determined by consistency with the definition of valued output from the perspective of the healthcare decision-maker in England, whose objective is maximising the level of aggregate health in the population (generally understood as quantity and quality of life), given a constrained set of resources [13, 14]. Therefore, we focussed our selection on the frameworks’ domains and dimensions that were closely linked to the impact on patients’ health outcomes. Adopting the English healthcare decision maker’s perspective separates the dimensions of our framework from those responding to the objectives of other healthcare system’s stakeholders. For example, a patient’s perspective may also value process outcomes that have a direct impact on a patient’s utility, such as satisfaction with healthcare, ease of access, or ability to book appointments; a healthcare provider’s perspective may also value outcomes like the quality of practice management or satisfaction with the working environment.

To maximise the framework’s relevance to the remit of general practice activity in England, we excluded dimensions deemed more relevant to other primary care services (e.g., midwifery or community services), or referring to healthcare conditions with low incidence and burden on general practice activity.

For each selected dimension of the output framework, we developed indicators of valued output. With those indicators we aimed to measure as closely as possible the impact of a healthcare episode on health. To this aim, each indicator combines the measurement of healthcare episodes (i.e., the ‘physical’ output) and the resulting impact on health outcomes (i.e., the ‘valued’ output). Patient-reported outcome measures would provide the ideal input to assess the impact of healthcare episodes on patients’ health. However, patient-reported outcome measures are not routinely collected in primary care settings in England [15]. Therefore, the indicators include: (intermediate or final) health outcomes that are amenable to the healthcare delivered by general practices; or clinical quality standards to proxy the impact on health, when the health outcomes achieved depend on the care provided at other levels of the healthcare system (e.g., when a condition diagnosed in primary care requires referrals to secondary care services).

To ensure consistency with clinical standards and practice guidelines in England, we conducted a review of the National Institute for Health and Care Excellence’s (NICE’s) Clinical Knowledge Summaries [16] and the Quality and Outcomes Framework (QOF) indicators [17]. The review allowed us to identify relevant healthcare episodes for each dimension of the framework, the related health outcomes, and clinical quality indicators.

2.2 Review and Process Validation

A first review exercise aimed to test the comprehensiveness of the framework’s dimensions and the face-validity of the indicators from a clinical perspective. Three clinical experts identified based on primary care clinical background and/or practice, and expertise in primary care data in England, were invited to provide feedback on the completeness of the frameworks’ dimensions in capturing the main areas of primary care activity; and on the suitability of the clinical outcomes and quality standards included in the indicators.

Representatives of the main primary care stakeholder groups were included in a proof-of-concept exercise of the framework through seven interviews with members and/or representatives of: patient participation groups (n = 2), general practice staffing groups (including general practitioners, nurses, practice managers and paramedics) (n = 4), and NHS England (n = 1). Interviewees were selected based on current affiliation with a relevant professional stakeholder association (patients, staffing groups) or commissioning bodies (NHS England). The objective of the interviews was to test the external acceptability of the framework’s dimensions and indicators and to validate their development process. To this end, the interviews described the thought process guiding the synthesis of the framework’s dimensions, and indicators and interviewees were invited to discuss the conceptual appropriateness of the measurement tool from their stakeholder group’s perspective.

An overview of the questions asked during the review and process validation stages is provided in Appendix 1 of the Online Supplementary Material (OSM). The framework’s dimensions and indicators were iteratively revised to incorporate feedback from the clinical expert reviews and stakeholder interviews.

3 Results

3.1 Performance Frameworks Review

Three studies [18,19,20] met our inclusion criteria for consideration in the synthesis of the multi-dimensional output framework.Footnote 1 The studies by Kringos et al. [18] and Barbazza et al. [19] provide frameworks and indicators to measure primary care performance in the countries of the European Union [18] or those of the World Health Organisation (WHO) European region [19]. The third study by Dawson et al. [20] provides a tool tailored to England for the measurement of primary care productivity.

Table 1 provides an overview of the three frameworks’ structure. Frameworks by both Kringos et al. [18] and Barbazza et al. [19] follow the Donabedian’s logic model [10]. Kringos et al.’s framework is composed of ten domains classified under the Donabedian’s model ‘structure-process-outcomes’. Barbazza et al.'s framework includes six domains classified as the Donabedian’s model components that are relabelled into ‘capacity-performance-impact of primary care’. The framework by Dawson et al. [20], instead, includes 11 primary care objectives grouped under four performance areas: external focus, practice management, patient focus and clinical care.

Table 1 Domains of selected frameworks
Full size table

3.2 Syntheses of Multi-Dimensional Output Framework

In the two frameworks inspired to the Donabedian structure [18, 19], the ‘outcomes’ and ‘impact’ domains were considered consistent with the definition of ‘valued’ output from the perspective of a healthcare decision-maker in England. Among the four objectives of the Dawson et al. [20] framework, clinical care was considered consistent with the idea of healthcare impact on patients’ health outcomes.

The resulting combined multi-dimensional output framework is shown in Table 2 (see Appendix 2 of the OSM for a detailed mapping of the dimensions across the frameworks). The 13 dimensions and associated 34 sub-dimensions aim to capture the most prevalent conditions, the main patient populations, and the priority areas of general practices in England.

Table 2 Multi-dimensional output framework
Full size table

The output framework in Table 2 incorporates feedback from the expert reviews and the stakeholder interviews. Clinical experts and stakeholders commented that certain dimensions in the frameworks from the literature may refer to specialties beyond the remit of general practices (e.g., family planning, sexual health advice or ante-natal care) or to conditions with very low incidence in England (e.g., tuberculosis). To address this comment, non-relevant dimensions were excluded from the framework (see Appendix 2 of the OSM for more detail), which ensured that the indicators under the included dimensions would measure the outcomes achieved under general practice-specific activities (e.g., in the case of antenatal care, general practices have a greater role than midwifery community services in following high-risk pregnancies). A few additions to the dimensions identified in the literature frameworks were also suggested: acute minor illnesses, which is an area of significant primary care activity; and cancer types that are either the most prevalent in England (breast, prostate, lung, bowel) [21] or for which a screening programme is delivered by general practices (cervical).

The experts judged the frameworks’ dimensions to be resilient to the impact of the COVID-19 pandemic on general practice services. They did not foresee the emergence of new primary care services not captured by the output framework, except for a potential seasonal COVID-19 immunisation programme. However, they expected a shift of general practice activity towards conditions that have been neglected or were exacerbated during the pandemic (e.g., mental health services).

3.3 Valued Output Indicators

Table 3 shows the valued output indicators for a subset of the output framework’s dimensions. The full indicators list (N = 51) is available in Appendix 3 of the OSM.

Table 3 Valued output indicators for selected dimensions of the framework
Full size table

Multiple indicators per dimension are possible if the healthcare delivered by general practices fulfils different functions. As a benchmark, we used the functions listed in the NICE Clinical Knowledge Summaries Goals and Quality Standards of current evidence base and guidance on best practice for primary care practitioners [16]. Based on those, functions were grouped into: (i) diagnosis and referral to other parts of the healthcare system (e.g., secondary care) for appropriate follow-up of symptoms and/or treatment of conditions; (ii) screening of high-risk population groups, aimed to an early diagnosis of conditions; (iii) primary prevention of new conditions; (iv) secondary prevention of adverse outcomes due to an existing condition; and (v) management of acute or long-term conditions to resolve or to minimise the impact on the normal functioning of a patient.

The valued output indicators combine the measurement of healthcare episodes in a certain observation period (e.g., year or semester quarter) with the related outcomes. In Table 3, the indicators of Smoking Cessation (primary prevention), Diabetes Type II (secondary prevention) and Breast Cancer (management of long-term conditions) focus on outcomes achieved within a certain timeline. For example, smoking cessation is measured within 12 months of the general practice consultation because general practice smoking records are updated annually, according to NHS guidance [22]. Concerns around emotional or social wellbeing, finances, tiredness, and pain following breast cancer care reviews are measured from 18 months of diagnosis when completing the Cancer Quality of Life Survey [23].

Indicators for Diabetes Type II (diagnosis and referral), Breast Cancer (diagnosis and referral) and Influenza immunisations (primary prevention) use measures of adherence to clinical quality standards that ensure effective care. For example, the outcome of influenza immunisations is based on the number of vaccines given before the start of the flu season, when their effectiveness in preventing an illness is assumed to be higher [24]. The timing of the primary care episode is also the defining factor for the effectiveness of diabetes type II diagnosis and referral. In this case, the 9-month timeline from diagnosis is derived by the correspondent QOF indicator [22]. Effective diagnosis and referral of breast cancer cases is closely dependent on the quality of the referral letter to enable specialists to make informed clinical decisions without delay [27].

The indicators presented in Table 3 and in Appendix 3 (OSM) incorporate the feedback from the clinical experts and primary care stakeholders. The experts found the structure of the output indicators consistent with the definition of valued output used in this paper. They commented on the need to evaluate the indicators alongside suitable environmental factors (e.g., socio-economic and demographic characteristics), which may affect general practices’ output. We examine this aspect in more detail in the Discussion section. Further, interviewees stressed that, where possible, the indicators’ wording on the outcomes and clinical processes should align as closely as possible to routinely collected metrics (e.g., the QOF indicators) to avoid duplication of effort in data collection.

4 Discussion

This study proposes a synthetic, multi-dimensional framework with indicators that define the valued output of general practices in England. This instrument represents a first step towards addressing the limitations of the economic literature on efficiency measurement, which has historically failed to use appropriate definitions of primary care output, considering the wide range of general practice services and their value in terms of health improvements [8].

Our framework is built around a set of dimensions encompassing the main areas of primary care activity and conditions. The multi-dimensional aspect of the framework allows us to appropriately differentiate across primary care services, in contrast with traditional output definitions considering aggregate activity-based measures (e.g., total consultations performed). The dimensions are based on existing performance frameworks that previously articulated the complex and diverse landscape of the outcome areas associated with primary care services. Importantly, our framework does not intend to replace the existing performance and effectiveness frameworks in the health service literature, or those routinely implemented in the real world (e.g., the QOF). In fact, these frameworks have a broader remit than the assessment of efficiency, which is only one dimension of the overall performance. Furthermore, our framework does not provide a final picture of efficiency without a simultaneous consideration of the resources (such as workforce, pharmaceuticals, or medical equipment) and other contextual factors involved in the healthcare production process. We further discuss the correct use of the multidimensional framework in the context of economic efficiency analyses in the paragraphs below.

The second limitation of the economic literature, commonly failing to measure the value produced by healthcare episodes, is addressed through the valued output indicators attached to the framework’s dimensions. Based on target outcomes and clinical standards in England, these indicators attempt to move the measurement of primary care output closer to the impact on health, which is the main outcome of healthcare from the perspective of the healthcare decision maker in England [14]. Arguably, health is not the only valuable outcome of healthcare [13]. Other socially valuable outcomes of healthcare may include the achievement of more equitable outcomes [25, 26], or improvements in process outcomes (such as waiting times) [27]. Recognising a potential limitation in the scope of the relevant healthcare outcomes included in the definition of valued output, in this study we focused on health improvements as the main primary care outcome. The reason for this choice is the current lack of measurement tools that include societal valuations for healthcare outcomes other than health and that would be necessary to inform potential trade-offs across different healthcare outcomes.

In the future, the multi-dimensional framework and indicators of valued output could support an understanding of primary care efficiency that better contributes to the healthcare decision maker’s pursuit of value for money. However, more work is necessary to operationalise the measurement of efficiency through the multi-dimensional framework and indicators.

First, further research should evaluate the current measurability of the proposed indicators through existing data sources. We choose ‘ideal’ health outcomes and clinical quality standards for the indicators based on our review of the NICE’s Clinical Knowledge Summaries. For example, the indicator of optimal management of breast cancer patients measures the presence of ‘significant concerns around emotional or social wellbeing, finances, tiredness, and pain’ during the cancer care reviews. This is a relevant outcome of the cancer care reviews, whose importance is underscored by recent efforts to improve long-term care of cancer survivors [23]. However, this is a difficult outcome to track unless patient-reported outcomes are measured systematically. Similarly, data on information included in referral letters from primary to secondary care, which we use as an indicator of the quality of diagnoses and referrals, may be difficult to access [28]. A similar assessment on the indicators’ measurability may benefit from further stakeholders and expert engagement, which, in this study, was undertaken to refine the scope of the framework’s dimensions and to test the external acceptability of the measurement tool, but it was limited in numbers.

A second area for additional work concerns the use of analytical methods to operationalise the valued output framework in efficiency and productivity measurement. Efficiency and productivity can be analysed using linear programming (e.g., data envelopment analysis, DEA) or statistical methods (e.g., stochastic frontier analysis, SFA), based on data on outputs produced, inputs employed, and other exogenous factors impacting the healthcare production process (e.g., demographics and socioeconomic status of the patient population, provider’s characteristics and location). These methods allow us to compare efficiency levels between healthcare production units, such as general practices, that belong to the same efficiency frontier (DEA) or distance function (SFA). DEA can consider multi-output definitions based on a weighted sum of the outputs. However, the weights are obtained through linear programming, which may not reflect the relative contribution of each output dimension to health improvements. SFA requires a unidimensional output definition, and a set of weights for each valued output indicator is unlikely to be readily available. Future research should consider suitable approaches for implementing the output definition provided by our framework, potentially considering a synthetic index combining multiple indicators.

Finally, we developed the framework and indicators with reference to the English general practice context and clinical standards. Therefore, they may not be directly transferable to other countries. However, their use outside of the English context could still support a more systematic understanding of valued output in primary care, particularly in settings where health maximisation is the principle guiding the allocation of constrained resources, and primary care shares similar gatekeeping and prevention functions as in England.

5 Conclusions

Improving efficiency and productivity are key objectives to ensure that general practices in England can meet the needs of a growing population with increasingly demanding and costly healthcare needs. Nonetheless, the evidence to assess efficiency and productivity of general practices is currently weak, partly because prevalent approaches to measure output are uninformative about the value for money obtained through limited resources.

This paper makes a first contribution to the improvement of primary care efficiency measurement by means of a synthetic multi-dimensional framework of valued output. Our proposed approach aims to bridge the gap between the simplified output measurement used in the economics literature with the health services research and policy literature, which attempts to capture the complex, multi-faceted and multi-dimensional work and outcomes of primary care providers. Adopting this approach for the economic analysis of primary care efficiency in future research will enable a more realistic understanding of the relative efficiency of different primary care providers, the trade-offs they face in the production of health, and the impact of factors such as team structure and technology on output and efficiency. At this stage, future research is required to address questions around the measurability of the indicators through available data sources and the implementation of the framework through analytical approaches for efficiency measurement.