Applied Health Economics and Health Policy

, Volume 12, Issue 5, pp 497–510

What is the Quality of Economic Evaluations of Non-Drug Therapies? A Systematic Review and Critical Appraisal of Economic Evaluations of Radiotherapy for Cancer

Authors

    • Centre for Health Economics (CHE)University of York
  • H. L. A. Weatherly
    • Centre for Health Economics (CHE)University of York
  • R. Ara
    • The School of Health and related Research (ScHARR), University of Sheffield
  • H. Basarir
    • The School of Health and related Research (ScHARR), University of Sheffield
  • M. Sculpher
    • Centre for Health Economics (CHE)University of York
  • R. Adams
    • Velindre Cancer Centre
  • H. Ahmed
    • Faculty of Medical ScienceUniversity College London
  • C. Coles
    • NHS Foundation Trust, Cambridge University Hospitals
  • T. Guerrero-Urbano
    • Guy’s and St Thomas’ Hospitals
  • C. Nutting
    • Royal Marsden Hospital
  • M. Powell
    • Barts Health NHS Trust, St Bartholomew’s Hospital
Open AccessSystematic Review

DOI: 10.1007/s40258-014-0115-8

Cite this article as:
Barbieri, M., Weatherly, H.L.A., Ara, R. et al. Appl Health Econ Health Policy (2014) 12: 497. doi:10.1007/s40258-014-0115-8

Abstract

Background

Breast, cervical and colorectal cancers are the three most frequent cancers in women, while lung, prostate and colorectal cancers are the most frequent in men. Much attention has been given to the economic evaluation of pharmaceuticals for treatment of cancer by the National Institute for Health and Care Excellence (NICE) in the UK and similar authorities internationally, while economic analysis developed for other types of anti-cancer interventions, including radiotherapy and surgery, are less common.

Objectives

Our objective was to review methods used in published cost-effectiveness studies evaluating radiotherapy for breast, cervical, colorectal, head and neck and prostate cancer, and to compare the economic evaluation methods applied with those defined in the guidelines used by the NICE technology appraisal programme.

Methods

A systematic search of seven databases (MEDLINE, EMBASE, CDSR, NHSEED, HTA, DARE, EconLit) as well as research registers, the NICE website and conference proceedings was conducted in July 2012. Only economic evaluations of radiotherapy interventions in individuals diagnosed with cancer that included quality-adjusted life-years (QALYs) or life-years (LYs) were included. Included studies were appraised on the basis of satisfying essential, preferred and UK-specific methods requirements, building on the NICE Reference Case for economic evaluations and on other methods guidelines.

Results

A total of 29 studies satisfied the inclusion criteria (breast 14, colorectal 2, prostate 10, cervical 0, head and neck 3). Only two studies were conducted in the UK (13 in the USA). Among essential methods criteria, the main issue was that only three (10 %) of the studies used clinical-effectiveness estimates identified through systematic review of the literature. Similarly, only eight (28 %) studies sourced health-related quality-of-life data directly from patients with the condition of interest. Other essential criteria (e.g. clear description of comparators, patient group indication and appropriate time horizon) were generally fulfilled, while most of the UK-specific requirements were not met.

Conclusion

Based on this review there is a dearth of up-to-date, robust evidence on the cost effectiveness of radiotherapy in cancer suitable to support decision making in the UK. Studies selected did not fully satisfy essential method standards currently recommended by NICE.

Key Points for Decision Makers

Few studies were found that examine the cost effectiveness of radiotherapy for breast, cervical, colorectal, head and neck and prostate cancer, and only two analyses were conducted in the UK.

Additional, high-quality evidence is required to inform decision making on the effectiveness and cost-effectiveness of radiotherapy in cancer.

Many recent publications identified in the review did not satisfy essential methods requirements. Forthcoming economic evaluations of radiotherapy in cancer should adhere to such requirements to better inform decision makers.

1 Introduction

Economic evaluations are increasingly used to inform decisions about the efficient allocation of healthcare resources. To date, these methods have mainly been applied to evaluate pharmaceutical interventions. Less evidence is available for other types of intervention, such as medical devices, surgical techniques and public health interventions, and decision making in these cases appears more challenging [1, 2]. The National Institute for Health and Care Excellence (NICE) in the UK represents an independent organisation tasked with assessing the clinical and cost effectiveness of health technologies and, to date, has mainly focused on new pharmaceutical and biopharmaceutical products. However, the scope of their responsibilities continues to expand and now includes diagnostics, public health interventions and social care.

NICE has issued a Reference Case for cost effectiveness in technnology appraisal, which specifies the methods standards considered to be consistent with a UK National Health Service (NHS) objective of improving population health given limited resources and a fixed budget. An element that distinguishes NICE from most other agencies is the explicit statement of a cost-effectiveness threshold, in terms of cost per quality-adjusted life-year (QALY) gained (£20,000–30,000), to classify interventions as good value for money.

Cancer is a leading cause of morbidity and mortality worldwide. An estimated 12.7 million new cases of cancer were diagnosed worldwide in 2008, with 7.6 million cancer-specific deaths (around 13 % of all deaths) [3]. Breast, cervical and colorectal cancers are the three most frequent cancers in women, while lung, prostate and colorectal cancers are the most frequent in men [3]. Much attention has been given to the economic evaluation of pharmaceuticals for treatment of cancer by NICE and similar authorities internationally. This contrasts markedly with the level of economic analysis developed for other types of anti-cancer interventions, including radiotherapy and surgery. For example, of the 35 NICE appraisals of breast cancer interventions, 28 (80 %) were technology appraisals of pharmaceuticals and seven (20 %) were non-pharmaceuticals.

Originally undertaken to inform the UK Department of Health, the aim of this review was to assess the quality of published economic evaluation studies of radiotherapy in breast, cervical, prostate, head and neck and colorectal cancers. Critical assessment of the studies was in terms of their reporting and conduct, including whether or not they adhered to methods standards for economic evaluation. There was also an assessment of whether the studies would provide relevant evidence for decisions in a UK setting.

2 Methods

Five systematic literature reviews were undertaken of the published evidence on the economic evaluation of radiotherapy in breast, cervical, prostate, head and neck, or colorectal cancer in July 2012. To identify relevant evidence, the following databases were searched: MEDLINE, EMBASE, Cochrane Database of Systematic Reviews (CDSR), NHS Economic Evaluation Database (NHSEED), Health Technology Assessment (HTA), Database of Abstracts of Reviews of Effects (DARE), and EconLit. Additionally, the Cost-Effectiveness Analysis (CEA) registry (https://research.tufts-nemc.org/cear4/), the NICE website (www.nice.org.uk), recent conference proceedings and reference lists of the included studies and existing reviews were also searched (see Appendix 1, with breast cancer as an example).

Citations retrieved by the searches were filtered by title and abstract, and the full manuscripts were retrieved if they satisfied the inclusion criteria. Studies were included in the review if they assessed the cost effectiveness of a radiotherapy intervention in individuals with breast, colorectal, cervical, head and neck or prostate cancer, and presented results in the form of either an incremental cost per QALY gained or an incremental cost per life-year (LY) gained. No language, publication or date restrictions were applied to the searches. Evidence was reviewed by a single researcher (MB) in close consultation with another author (HW) and, where there was any uncertainty, the issues were discussed within the review team until a consensus was achieved.

A data extraction template was designed to include relevant information from the studies identified including setting, patient population, intervention and comparators, type of economic analysis (e.g. model-based vs. based on a clinical study) and results. Included studies were appraised on the basis of satisfying essential, preferred and UK-specific methods requirements. The checklist draws heavily on the NICE Reference Case for economic evaluations. In addition, it draws on the authors’ experience of undertaking economic evaluations, working with decision makers to use economic evaluations to inform decisions and authors’ knowledge of the published literature on checklists and methods guidelines issued by other decision-making organisations. The aim was to review the report and conduct of included economic evaluations, focusing on the quality of each study to inform decision making. As such, the checklist is concise and designed as a decision aid, clearly distinguishing where essential and preferred requirements are met, as explained below. Those methods standards for economic evaluation that are generally recommended as best practice for undertaking a high-quality study were considered as essential requirements, and include the following seven:
  1. 1.

    Comparators clearly defined and justified: the study should provide a transparent and detailed description of the interventions considered, justifying the choice of comparators with relevance to, for example, clinical practice or best practice. For example NICE guidelines [4] state that, in order to define the decision problem “a definition and justification of the technologies being compared” (page 31) is needed. Similarly, the Canadian guidelines issued by the Canadian Agency for Drugs and Technologies in Health (CADTH) [5] state that “it is crucial to select the appropriate comparators for the analysis, as the choice will be important in determining the cost-effectiveness of the intervention” (page 16). Other guidelines, for example those of the Academy of Managed Care Pharmacy (AMCP) [6] in the USA and the Pharmaceutical Management Agency (PHARMAC) [7] in New Zealand, specify which comparator to use in the analysis (for example, “the treatment that most prescribers would replace in New Zealand,” page 8).

     
  2. 2.

    Patient group/indication clearly described: a clear description of the characteristics of the patient group receiving the interventions should be provided. For example, NICE states that the decision problem is supported by “a definition and justification of the relevant patient group (s) to be treated” (page 31). Some guidelines (e.g. NICE, CADTH, those issued by the Pharmaceutical Benefits Advisory Committee [PBAC] [8] in Australia) also highlight the importance of defining potential subgroups.

     
  3. 3.

    Effectiveness evidence based on a systematic review: evidence of a full literature search and, where appropriate, a synthesis of evidence on health effects informed by systematic review, are recommended by most economic guidelines (e.g. CADTH, PBAC, NICE, AMCP, the Healthcare Insurance Board (CVZ) [9] in the Netherlands). This ensures the data used represent all the relevant published evidence available at the time of the study, and helps to minimise bias. Data synthesis methods such as meta-analysis are recommended as appropriate. For example, the NICE reference case states that “evidence on outcomes should be obtained from a systematic review, defined as the systematic location, inclusion, appraisal and synthesis of evidence to obtain a reliable and valid overview of the data related to a clearly formulated question…. Synthesis of outcome data through meta-analysis is appropriate provided there are sufficient relevant and valid data that use measures of outcome that are comparable.”

     
  4. 4.

    Data for measurement of health-related quality of life (HRQoL) reported directly by patients and/or carers: it is generally recommended (e.g. NICE, Spanish guidelines [10], PHARMAC) that information on changes in HRQoL should be reported directly by patients with the condition under analysis, or by their carers when it is not possible to obtain these data directly from patients. The NICE reference case states that the “measurement of changes in HRQL should be reported directly from patients.”

     
  5. 5.

    Appropriate time horizon: the time horizon should be long enough to reflect all important differences in costs or outcomes between the interventions being compared. In the case of cancer, a lifetime horizon is typically the most appropriate, as the interventions could potentially have an effect on survival duration.

     
  6. 6.

    Relevant one-way sensitivity analyses: this is necessary to capture the full range of uncertainty that is relevant for key (model) inputs. In some circumstances, alternative scenarios and worst/best cases should be considered. Essentially, all guidelines for economic evaluation recommend the use of at least a one-way sensitivity analysis for key uncertain parameters.

     
  7. 7.

    Probabilistic sensitivity analysis (PSA) used to quantify full uncertainty: PSA is considered the most appropriate technique to analyse joint uncertainty in all parameters. Although only NICE recommends PSA for submissions, this is also encouraged by other agencies (AMCP, CADTH, CVZ).

     
Two methods features are more controversial but preferred for economic evaluation in many settings:
  1. (1)

    CEA using QALYs: although the use of a CEA based on QALYs is often stated as the preferred option to account for the impact of an intervention on HRQoL and survival duration using a consistent generic measure of outcome, this is not always seen as mandatory by all international agencies.

     
  2. (2)

    Preference data for valuation of HRQoL health states by a representative sample of the public: although some agencies (e.g. NICE, CADTH) recommend that the valuation of HRQoL should be based on the preferences of a representative sample of the general public, this is not always seen as mandatory, and valuation from patients, carers or health professionals are accepted alternatives by some agencies.

     
Finally, a secondary objective of the review was to assess the relevance of the studies identified from a UK decision-making perspective. The NICE Reference Case defines five areas of methods that can be considered specific to decision making in the NHS:
  1. (1)

    Comparators used in the UK NHS at the time each study was undertaken

     
  2. (2)

    Decision problem and analysis relating to a UK setting

     
  3. (3)

    Cost perspective of UK NHS and personal social services costs

     
  4. (4)

    Discount rate of 3.5 % per annum for costs and outcomes

     
  5. (5)

    Cost per QALY below the NICE threshold value (£20,000–30,000 per QALY).

     

The last point assesses if the intervention(s) analysed would be considered cost effective in the UK context

3 Results

3.1 Results of the Review

A flow diagram of the results of the searches is presented in Fig. 1. A total of 3,358 titles and abstracts were initially identified (breast 794, colorectal 623, prostate 707, cervical 325, head and neck 909). After excluding duplicates and studies that did not meet inclusion criteria, 116 full papers were retrieved (breast 18, colorectal 18, prostate 37, cervical 13, head and neck 30). Of these, 29 satisfied the inclusion criteria and were included in the review (breast 14 [1124], colorectal 2 [25, 26], prostate 10 [11, 2735], cervical 0, head and neck 3 [11, 29, 36]).
https://static-content.springer.com/image/art%3A10.1007%2Fs40258-014-0115-8/MediaObjects/40258_2014_115_Fig1_HTML.gif
Fig. 1

Flow diagram for study inclusion

Of the studies, 13 (45 %) were undertaken in the USA, and only two (7 %) studies were set in the UK. A total of 14 (48 %) of the studies selected were published a decade or more before this review. Most analyses expressed cost effectiveness in terms of incremental costs per QALY. There was some variation in the interventions considered in the analyses (especially for breast and prostate cancer). Study characteristics and the main results for each study are presented in Table 1. To judge the relevance of these findings to the UK setting and to assess the quality of the studies included, the numbers of studies that met essential, preferred and UK-specific requirements were examined. Figure 2 illustrates findings for each study included.
Table 1

Characteristics of studies included

References

Setting

Population

Intervention

Comparator

Study design

Main findings

Breast cancer

Alvegard et al., 2005 [22]

Sweden

Stage I–II BC

Postoperative RT

No RT

Model (not explicit)

Post-operative RT is cost effective when used as adjunction to no medical adjuvant tx and in high-risk pts

Dunscombe et al., 2000 [20]

Canada

Premenopausal node-positive BC

Adjuvant loco regional RT (RT plus surgery and chemotherapy)

No RT

Spreadsheet-based activity-costing model

Adjuvant RT is a cost-effective tx

Hayman et al., 1998 [13]

USA

Stage I–II BC

Adjuvant RT

No RT

Markov model

RT is cost effective following conservative surgery

Hayman et al., 2000 [14]

USA

Stage I–II BC

Adding an electron-beam boost to tangential RT

RT without tangential boost

Markov model

Boost is not cost effective using a threshold of $50,000 per QALY

Lee et al., 2002 [15]

USA

High-risk premenopausal node-positive BC

PMRT

No PMRT

Markov model

PMRT is cost-effective; results of model were robust

Lievens et al., 2005 [24]

Belgium

Stage I–III BC

Post-operative RT to the internal mammary and medial supraclavicular lymph node region

No RT

Markov model

RT was cost effective when evaluated over a long time period in tumours with a slow natural history and systemic tx for relapse

Liljegren et al., 1997 [23]

Sweden

Unifocal stage I BC post sector resection and axillary dissection

Post-operative RT

No RT

Model (decision tree)-based

Cost of RT is high; results show the importance of identifying risk factors for local recurrence

Lundkvist et al., 2005 [11]

Sweden

BC (not specified)

Proton RT

Conventional RT

Markov model

Likely that proton RT is more appropriate for individuals with higher than norm risk of CVD

Marks et al., 1999 [16]

USA

Local regional relapse node-positive BC

RT

No RT

EE using data from several clinical studies

Cost per local regional relapse prevented decreases as the number of positive axillary nodes increases

Patrice et al., 2007 [17]

USA

Early-stage BC

RT plus tamoxifen

Tamoxifen alone

Markov model

RT post conservative surgery was cost effective in older women

Prescott et al., 2007 [12]

UK

Minimum-risk (elderly women post breast-conserving surgery)

Whole breast RT

No RT

Markov model

While RT was well tolerated with no impairment on overall QoL at 3 years, the no RT intervention was cost effective

Samant et al., 2001 [21]

Canada

High-risk (postmenopausal) node-positive BC (post mastectomy)

Loco regional RT

No RT

Update of Dunscombe et al., 2000 [21]

RT appears cost effective, but further analyses needed

Sher et al., 2009 [18]

USA

Early-stage estrogen-receptor positive BC

EB-PBI; MS-PBI

WBRT

Markov model

EB-PBI is cost effective vs. WBRT, but MS-PBI is not and is unlikely to be cost effective unless the QoL after MS-PBI is superior

Suh et al., 2005 [19]

USA

Ductal carcinoma in situ

RT

No RT

Markov model

Addition of RT following BCS for pts with ductal carcinoma in situ should not be withheld because of concerns regarding its cost effectiveness

Colorectal cancer

Dahlberg et al., 2002 [26]

Sweden

Resectable rectal cancer

Preoperative RT followed by surgery within the next week

Surgery without RT

EE alongside clinical trial

Preoperative RT was cost effective even using their most pessimistic assumptions

van den Brink et al., 2004 [25]

The Netherlands

Resectable rectal cancer

Preoperative RT with TME

TME without preoperative RT

Model-based TME study

Preoperative RT was both effective and cost effective

Prostate cancer

Basu and Meltzer, 2005 [31]

USA

Early PC

RT

Surgery (not specified), WW

Model (NS)

WW was the most cost-effective option given a threshold of $50,000 per QALY

Hummel et al., 2003 [28]

UK

Early localised (TNM stages 1 and 2) PC

Traditional RT

WW, RP, brachytherapy, 3D-CRT, cryotherapy

Markov model

Brachytherapy and 3D-CRT appear the most cost-effective options but high variability was found

Konski and Watkins-Bruner, 2004 [32]

USA

Hormone-refractory PC with bone metastases

SFX, MFX

Pain medication only, chemotherapy (mitoxantrone or prednisone)

Markov model

SFX was the most cost-effective option for palliative tx

Konski et al., 2005 (RTOG) [34]

USA

Treatment of locally advanced with clinical stage T2b, T2c, T3, T4 PC without distant metastasis

RT plus hormone therapy

RT

Markov model

Adding hormone to RT provided good value for money

Konski and Konski, 2005 [33]

USA

Clinically localised PC

I-RT

3D-CRT

Markov model

RT generally cost effective, although it could be less good value for money for old pts and over a short-term time horizon

Konski et al., 2006 [35]

USA

PC eligible for surgery

I-RT

3D

Markov model

I-RT could be considered cost effective at the upper limit of acceptability

Konski et al., 2007 [29]

USA

Intermediate-risk PC

I-RT

PPT

Markov model

PPT is not cost effective in most pts

Lundkvist et al., 2005 [11]

Sweden

Unspecified PC

Proton therapy

Conventional RT

Markov model

Proton therapy appears a cost-effective option but high variation around mean values was found

Neymark et al., 2002 [27]

France

PC candidate for surgery

Hormonal therapy plus radiotherapy (COMB)

Conventional RT

EE with data from a clinical trial

COMB was dominant when mean survival time was estimated by a restricted means analysis

Samant et al., 2003 [30]

Canada

Locally advanced PC

Adjuvant goserelin in addition to RT

RT

EE with data from a clinical trial

Long-term adjuvant goserelin provided good value for money

Head and neck cancer

Higgins, 2011 [37]

Canada

Early glottis (T1 or T2) cancer

TOL

XRT

Decision tree model- based EE

TOL is dominant over XRT for this group of pts with early cancer

Konski and Watkins-Bruner, 2004 [32]

USA

Locally advanced HNSCC

AFXC, AHFXS, HFX

Standard fractionated RT

Markov model

HFX and AFXC appear the most cost-effective options

Lundkvist et al., 2005 [11]

Sweden

H&N (unspecified)

Proton therapy

Conventional RT

Markov model

Proton therapy was the cost-effective option

AFXC accelerated fractionated radiotherapy with concomitant boost, AHFXS accelerated and fractionated radiotherapy with split, BC breast cancer, BCS breast-conserving surgery, CVD cardiovascular disease, EB-PBI external beam partial breast irradiation, EE economic evaluation, HFX hyper fractionated RT, HNSCC head and neck squamous cell carcinoma, H&N head and neck, I-RT intensity-modulated RT, MFX multifraction RT, MS-PBI MammoSite partial breast irradiation, NS not specified, PC prostate cancer, PMRT post-mastectomy RT, PPT proton-beam therapy, pt(s) patient(s), QALY quality-adjusted life-year, QoL quality of life, RP radical prostatectomy, RT radiotherapy, SFX single fraction RT, TNM tumour, node, metastasis, TME total mesorectal excision, TOL trans oral CO2 laser incision, tx treatment, WBRT whole breast RT, WW watchful waiting, XRT external beam radiation, 3D-CRT three-dimensional conformal radiation therapy

https://static-content.springer.com/image/art%3A10.1007%2Fs40258-014-0115-8/MediaObjects/40258_2014_115_Fig2_HTML.gif
Fig. 2

Quality of included studies compared with the NICE reference case. HRQoL health-related quality of life, NHS UK National Health Service, NICE National Institute for Care and Excellence, QALY quality-adjusted life-year

3.2 Essential Requirements

Among the seven criteria that were defined as essential requirements, four were met in the majority of studies; these were comparators clearly defined, patient group/indication clearly defined, appropriate time horizon and relevant one-way sensitivity analyses. In 27 (93 %) of the studies, the interventions/comparators of the analysis were clearly defined. The two studies without a clear description of the interventions compared were both in prostate cancer (Neymark et al. [27] and Basu and Meltzer [31]). These studies focused on methods as opposed to generating results to inform decision making. As such, the failure to specify the options being compared can perhaps be understood.

The interventions compared could be grouped into eight broad categories: radiotherapy techniques in early-stage breast cancer patients following conserving surgery; post-mastectomy radiotherapy versus no radiotherapy in breast cancer patients; the comparison of alternative radiotherapy techniques in patients with breast cancer; radiotherapy options for colorectal cancer; radiotherapy versus surgery in early prostate cancer; different radiotherapy techniques in patients with prostate cancer; radiotherapy plus hormone therapies in locally advanced prostate cancer and palliative treatments; and comparison of radiotherapy techniques for head and neck cancer. Only in the methods paper by Basu and Meltzer [31] was there no clear description of the patient population (defined only as prostate cancer patients). In the other studies, patients’ general characteristics were reported, at least in terms of disease severity. Given the potential impact of radiotherapy on mortality risk, the most appropriate time horizon is likely to be patients’ lifetime. A long-term time horizon (longer than 10 years) was used in 23 (79 %) studies, while six (21 %) studies [12, 26, 31, 27, 32, 38] adopted a relatively short time horizon of between 1 and 8 years or did not clearly specify the time horizon of the analysis. Finally, one-way sensitivity analyses were conducted in the majority of studies (24 [83 %]), and the impact of variations in key parameters on cost-effectiveness results was reported.

Other accepted requirements for methods standards were not generally met. First, clinical data were obtained from a systematic review of the literature in only 3 of the 29 (10 %) studies (Hummel et al. [28] for prostate cancer; Konski and Watkins-Bruner [32] for both prostate and head and neck). In one study [15], data were obtained from a published meta-analysis; 13 studies (45 %) obtained clinical data from a single study (generally a clinical trial), while the remaining analyses were informed by data from various sources either selectively chosen by the authors or not fully described.

The data for measurement of HRQoL were reported directly by patients and/or carers in 5 of 26 studies (19 %). In three studies, HRQoL was not considered. In 7 of 26 (27 %) cases, data were taken from a mixture of patients’ and experts’ judgements. In the remaining 14 of 26 (54 %) cases, data were taken from previously published analyses and it was not possible to tell who was involved in the measurement task.

Finally, PSA was performed in only 10 of 29 studies. Six of the 19 studies without a PSA were conducted before 2001 when this technique was not commonly used, but the remaining 13 studies were published more recently.

3.3 Preferred Requirements

QALYs were used as the main outcome measure in 26 of 29 cases (90 %). The inclusion of HRQoL preference data valued by a representative sample of the public was rare, with only 3 of 26 (10 %) analyses [12, 25, 28] using these general population weights. In most cases (14 of 26, 54 %), little detail on sources of valuation of HRQoL data was provided and it is unclear whether these were taken from the general public.

3.4 UK-Specific Requirements

Only two studies were conducted in the UK (Hummel et al. [28] for prostate cancer and Prescott et al. [12] for breast cancer). Therefore, the UK NHS and personal social service costs and the UK setting were used in these analyses. Prescott et al. [12] adopted a 3.5 % per annum discount rate both for costs and for benefits while, in the Hummel et al. [12] study, the NICE discount rate of 6 % for costs and 1.5 % for benefits was used as they based their analysis on an earlier NICE Reference Case (2001). The majority of studies (24 of 29 [83 %]) used comparators relevant to the UK, even though they were not developed for UK decision makers specifically. Exceptions include studies by Lundqvist et al. [11] (both for prostate and for head and neck cancer) and Konski et al. [29] (who assessed the cost effectiveness of proton-pump therapy, which are not frequently used in the UK), and two studies on colorectal cancer [25, 26], which considered interventions whose use in the NHS currently varies widely.

As a number of the costs for the analyses were not undertaken using British pounds sterling, the currency for the cost-effectiveness estimates were converted using standard currency conversion exchange rates (year 2012). NICE’s cost-effectiveness threshold value was applied to the studies to assess the value for money on the interventions in the UK NHS. Incremental cost-effectiveness ratios were not inflated since the objective of the analysis was to assess whether these estimates were below the £20,000–30,000 threshold at the time of each study. The main intervention under analysis would be considered cost effective in 16 (55 %) studies using a £20,000 threshold and 19 (65 %) studies using a £30,000 threshold (Suh et al. [19], Konski et al. [35] and Lundqvist et al. [11] for prostate are those studies with cost-effectiveness ratios between £20,000 and £30,000). On the basis of the NICE threshold values, the following results could be synthesised: in the case of breast cancer, whole breast radiotherapy following breast conservation would be considered cost effective compared with no radiotherapy, but further studies are required to identify a very low-risk group of patients who derive minimal benefit and can, therefore, safely avoid radiotherapy. Post-mastectomy radiotherapy would be considered cost effective compared with no radiotherapy in patients at higher risk of recurrence. However, the definition of this level of risk is not clear. For colorectal cancer, preoperative radiotherapy followed by surgery compared with surgery alone would be considered cost effective. For prostate cancer, brachytherapy and 3-dimensional (3D) conformal radiation would be considered cost effective compared with traditional radiotherapy, but intensity-modulated radiotherapy would be cost effective compared with 3D conformal radiation; radiotherapy plus hormone therapy would be considered cost effective compared with radiotherapy alone in locally advanced patients; single or multi-fraction radiotherapy appears cost effective compared with palliative treatments (pain medication or chemotherapy) in hormone-refractory metastatic prostate cancer patients. Finally, for head and neck cancer, accelerated fractionated radiotherapy with concomitant boost, and hyper fractionated radiotherapy would be considered cost effective compared with standard fractionated radiotherapy in patients with local advanced disease.

4 Discussion

The objective of this review was to identify and summarise the evidence on the cost effectiveness of radiotherapy for various cancers, and to assess methods quality in those studies based on the NICE Reference Case. We observed substantial heterogeneity in the methods used. Overall, three key methods requirements considered essential for a good-quality evaluation (effectiveness evidence based on a systematic review, data for measurement of HRQoL reported directly by patients and/or carers, use of PSA to quantify the uncertainty) were generally not fulfilled. One of these criteria, the failure to use PSA or equivalent using individual-patient data from trials to characterise fully evidential and other forms of uncertainty, may reflect that a proportion of these studies was undertaken prior to the general use of these methods. Although there are also few examples in the later studies, it may be expected these methods will be more commonly used in the future.

The failure of a large proportion of studies (18 of 26 [69 %]) to source HRQoL measurements from patients probably reflects, in part, evidential weaknesses in the underlying clinical data being used by the economic evaluations in this review. In other words, when these studies were undertaken, few HRQoL data were collected in trials, particularly generic preference-based instruments suitable for estimating QALYs. More recently, methods to ‘map’ HRQoL weights from disease-specific instruments or clinical measures have been developed (Hernàndez Alava et al. [36]), and it may be expected they will be used in future studies where there are no direct estimates of suitable HRQoL weights. Where HRQoL weights are not available to estimate QALYs, use of high-quality mapping strategies can provide an informative approach to address the gap [43].

Arguably the most concerning method limitation is the failure of most studies (26 of 29 [90 %]) to incorporate the full extent of the available evidence on clinical effectiveness into economic analyses using systematic review. A general principle of evidence-based medicine, as well as economic evaluation, is not to be selective with evidence. However, in the review, many studies (12 of 29 [41 %]) used a single source of clinical-effectiveness evidence such as a randomised trial. It is possible that single clinical studies represent the only clinical evidence available in some cases, but this was not discussed explicitly in the papers. Where more than one clinical study has been identified through a systematic review, evidence on effectiveness for economic evaluation should be synthesised using, for example, meta-analysis. At the very least, a range of scenarios should be considered in the economic analysis using those different sources of clinical evidence. However, in this review, a minority of studies used meta-analysis or scenario analysis to bring in a fuller range of clinical evidence.

In terms of relevance for UK decision makers, only two studies were conducted in the UK and fully met the NICE UK-specific requirements. It is difficult to judge the relevance of the results of studies conducted in other jurisdictions, especially due to differences in clinical practice and sometimes in methods criteria specified by reimbursement and pricing agencies. Although most studies included interventions that were relevant to UK clinical practice at the time of publication, they might be dated due to improvements in clinical techniques and less important for current policy making. Radiotherapy techniques used in the cancer areas investigated have evolved rapidly over the last decade and many newer techniques are currently being evaluated in clinical trials. In addition, the clinical evidence in many of the studies is old and a number of new trials describing novel radiotherapy interventions have been reported, or are currently ongoing (e.g. CRO7 trial, SUPREMO trial, TARGIT-A trial, Warde et al. [3740]).

Essential and preferred methods requirements were mainly taken from the NICE Reference Case, supported by other guidelines (e.g. AMCP, CADTH, PBAC). NICE represents an oft-cited independent, centralised authority where decision making is informed by CEA. A strength of NICE is the provision of an explicit cost-effectiveness threshold to judge the value for money of the intervention considered. Another point that is not explicitly defined in the NICE Reference Case but is of considerable importance is the impact of heterogeneity in the cost effectiveness of treatments across different subgroups of patients. In the case of cancer, the identification of specific types of patients in whom radiotherapy is cost effective appears a key issue. For example, in the case of breast cancer, an on-going meta-analysis by the Early Breast Cancer Trialists’ Collaborative Group (EBCTCG) [41] of over 10,000 women treated in clinical trials with and without radiotherapy, has shown that, overall, breast radiotherapy reduces any cancer recurrence by 50 % [38]. However, although the relative advantage for breast radiotherapy is constant, the absolute benefit for individuals varies depending on their risk of recurrence and this, in turn, drives the cost effectiveness of radiotherapy in these patients. It is likely that, in a very low-risk group of patients, radiotherapy would not be cost effective. The same can be stated of prostate cancer, in which the relative merits of radiotherapy versus watchful waiting for low-risk disease has not been tested but is unlikely to demonstrate improvements in disease-specific survival up to 10 years of follow-up, since a comparison of radical prostatectomy and watchful waiting has shown no survival benefit [42].

In conclusion, based on this review, there is a dearth of up-to-date, robust evidence on the effectiveness and cost effectiveness of radiotherapy in cancer. The number of published economic evaluations of radiotherapy using QALYs or LYs saved appears quite low. For example, only two studies were found for colorectal cancer, three for head and neck cancer and none for cervical cancer. Some essential methodological standards were generally not met, especially in the identification of the clinical evidence. Consequently, the full uncertainty associated with the clinical benefits of the interventions was rarely captured. Finally, few of the studies were based in the UK, and international costs of interventions and healthcare use do not generally transfer to the UK. As new, relevant evidence becomes available, additional cost-effectiveness evaluations should be undertaken to inform decision makers for current and future clinical practice in this important area.

Acknowledgment/funder:

The research described in this article was funded by the Department of Health in England under the Policy Research Unit in Economic Evaluation of Health and Care Intervention (EEPRU) based at the University of York and the University of Sheffield. The views expressed are not necessarily those of the Department. The sponsors had no involvement in the analysis or interpretation of the data or findings presented here or the decision to submit the article for publication. We are grateful to our clinical advisors, who provided excellent input along the different stages of the research.

Conflict of Interest

The authors have no conflicts of interest to declare

Authors’ contributions to the paper

M. Barbieri is the lead author of the paper and the reviewer of all the articles included in the systematic review. H. Weatherly contributed in the preparation of the manuscript and was involved in the review for discussion of included/excluded papers. R. Ara, H. Basarir and M. Sculpher were involved in the preparation of the manuscript and in the discussion of the main issues of the review. R. Adams, H. Ahmed, C. Coles, T. Guerrero-Urbano, C. Nutting, and M. Powell provided clinical expertise in the different areas of cancer care. M. Barbieri is the guarantor for the overall content of the study.

Copyright information

© The Author(s) 2014

Open AccessThis article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.