Background

Given the constraints on health sector resources and the large demand for expanded or new health sector interventions, it is important to determine which interventions get the best value for money out of the limited resources available. As part of this prioritisation, cost-effectiveness is an important criterion for policymakers to consider, although it should be considered alongside other issues such as health equity, the size of the health gain, the timing of health gains and costs, issues around intervention feasibility and public acceptability, etc.

Previous international work has generated league tables for intervention cost-effectiveness from studies in multiple countries [1,2,3,4], for specific countries (e.g., the USA [5] and Australia [6, 7]), and for specific topic domains such as cancer care [8] and dietary sodium reduction [9]. All of this work suggests that there are many cost-saving and cost-effective interventions that policymakers could adopt or enhance use of. Nevertheless, there are often method differences between studies that may limit the coherence of such league tables [10].

The country in our case study presented here (New Zealand (NZ)) is a fairly typical high-income country in which demand for health services is constantly increasing relative to available resources. However, New Zealand does have a single national agency that successfully constrains rising pharmaceutical prices (PHARMAC) and conducts cost-utility analyses as part of its decision-making processes. But elsewhere, in the health sector, there is no agency that routinely does such analyses and systematically prioritises health sector interventions. There is also no government-wide threshold for when gaining a quality-adjusted life-year (QALY) is deemed “cost-effective” or the relative importance of gaining QALYs in different populations as part of reducing health inequalities (given the health gaps between Māori (indigenous) and non-Māori in this country).

Despite this, a league table has been published of 21 pharmaceutical interventions in New Zealand [11]. This work reported one intervention that was cost-saving and 20 with cost-effectiveness ratios ranging from NZ$771 to $142,000 per QALY (with a median value of the latter group of $9290, and mean of $25,300 [with these values calculated by us]). Other New Zealand-specific league tables include one for 12 dietary sodium reduction interventions (of which all but one were cost-saving [12]) and one for five tobacco control interventions (all cost-saving [13]). We have also developed an online interactive league table [14] that encompasses all the interventions modelled by the Burden of Disease Epidemiology, Equity and Cost-Effectiveness Programme (BODE3) Programme (covering tobacco control, nutrition, injury prevention, cancer prevention, and cancer treatment and management). However, in this particular study, we aimed to identify if other published cost-effectiveness analyses (CEAs) performed for New Zealand by different research teams could be included in a single reasonably coherent league table.

Methods

Literature searches

We first aimed to identify published New Zealand-specific CEAs that meet all of the following criteria: (i) the study was published in a peer-reviewed journal in the time-period 1 January 2010 to 8 October 2017; (ii) the study results included one of the following metrics: a cost per QALY or life-year (LY) gained or a cost per disability-adjusted life-year (DALY) averted; (iii) the study involved New Zealand epidemiological or cost data (even if New Zealand data was part of a combined study with other countries); and (iv) the study was not an output of the BODE3 Programme, since results from this programme all use a standard methodology and are detailed in an online interactive league table [14] (with a journal article about this league table pending).

The specific aspects of the search strategy are detailed in Table 1 and examples of excluded studies are detailed in Additional file 1: Table A1.

Table 1 Search strategy used to identify cost-effectiveness studies relevant to New Zealand
Full size table

Threshold for cost-effectiveness

A cost-effectiveness threshold from a policymaker's perspective will vary depending on what (dis)investment opportunities are being considered and what new (or requirement to reduce) funding is available, at a given time for a given investment horizon. Therefore, funding agencies usually do not have an official threshold—including in New Zealand. Nevertheless, as a guide, we note that on occasions, New Zealand Government agencies have quoted a value from the New Zealand Treasury of $38,110 per QALY (e.g., as per a Ministry of Health Report [15]), which is close to the World Health Organization guidance [16] of the per capita gross domestic product (GDP) of a country per QALY gained, i.e., $45,000 in NZ$ 2011 [17]).

Adjustment to NZ$ 2017

After abstracting the key information from each study, we then scaled the incremental cost-effectiveness ratio (ICER) values to 2017 NZ$ using the consumer price index (CPI) and purchasing power parity (PPP) (when translating from other currencies).

Results

A total of 21 cost-effectiveness studies were identified and their key characteristics are summarised in Table 2 (with more detailed summary data and commentary in Additional file 1 Tables A2, A4, and A5 in the Additional file 1). Of these, the reported incremental cost-effectiveness ratios (ICERs) suggest that 19% had cost-saving interventions, 71% had main results that can be classified as cost-effective, and 10% were not cost-effective using New Zealand GDP per capita as a threshold. The median value (2017 NZ$) was $5080 per QALY/DALY/LY as per the values in Table 2 and taking the first figure if multiple values are shown.

Table 2 League table of the 21 New Zealand cost-effectiveness studies identified and published in the period 1 January 2010 to 8 October 2017 (ordered by decreasing cost-effectiveness, with additional details on each study in the Additional file 1: Tables A2, A4, and A5)
Full size table

Table 3 shows additional details relating to the study methods including the variable discount rates (range 0 to 10%). Only 57% of the studies used a 3.0% or 3.5% discount rate (with 3.0% being the internationally recommended rate [18], but within New Zealand, 3.5% is recommended by the Pharmaceutical Management Agency (PHARMAC; the pharmaceutical and device funding agency for the New Zealand Government)). QALYs were used in 76% of the studies and around half (48%) used a lifetime time horizon. However, 24% used a very short time horizon of only 12 months (i.e., usually the length of the trial upon which the health economic analysis was based). Most studies used a health system perspective (86%), with the rest being a societal one (two studies had both). Only three of the studies included productivity losses, and none include greenhouse gas emissions, even though a number of interventions involved dietary changes and transport costs.

Table 3 Summary characteristics of the 21 studies with cost-effectiveness analyses for New Zealand in the period 1 January 2010 to 8 October 2017 (for the key interventions in each paper as shown in Table 2)
Full size table

The interventions studied were most commonly for primary prevention (38%), followed by treatment/management (33%) and then secondary prevention/screening (29%) (Table 2). Most of the interventions involved targeting a particular population group (90%) and were voluntary (95%), i.e., not involving regulations or taxes. Disease and condition topics were diverse, but 24% were related to cardiovascular disease and the next most common category was cervical cancer.

The search strategy also identified five CBAs (which did not have a cost-effectiveness component) and 12 other types of CEAs (albeit not using the QALY/DALY/LY metrics) (Table A3 in Additional file 1). Of the CBAs, only one included a cost-effectiveness component (i.e., cost-per-fall prevented [19]). Of the CEAs, only 17% (2/12) used a cost-per-death prevented, with the rest using other health metrics.

Discussion

Main findings and interpretation

It seemed possible to produce a reasonably coherent league table for the ICER values from different studies in this high-income country. However, given the method differences, such a league table is probably only useful as a general guide in terms of broad categorisations of “likely cost-saving”, “likely to be cost-effective”, and “unlikely to be cost-effective”. Indeed, relatively small differences between ICER results (e.g., less than $20,000 per QALY/DALY averted, or less than a twofold difference in ICER) may not be particularly critical to informed prioritisation of interventions (given the many other considerations needed in policymaking such as the size of the health gain, the size of the costs/cost-savings, impact on health inequalities, and intervention feasibility). However, differences larger than these, especially if premised on expert consideration of the assumptions and context of each study being compared, probably make the simple league table comparisons outlined in this study potentially useful for policymaking or at least guiding prioritisation around further research efforts.

The high levels of methodological diversity in the identified CEAs suggest the need for researchers to strive harder for methodological compatibility of their work, e.g., with the use of the ISPOR guidelines/CHEERS checklist [42] and other internationally agreed best practice health economic approaches [18]. This would allow their study results to be more readily integrated into methodologically compatible league tables. Similarly, if researchers perform CBAs, they should ideally also attempt to produce results in terms of cost per QALY gained from both a societal and health sector perspective.

The overall pattern of results of many cost-saving and cost-effective interventions was reasonably compatible with other league table work for this particular high-income country (the league tables in these three publications [11,12,13] and the online league table developed by BODE3 Programme [14]). They are also broadly compatible with Australian league table work [6, 7] and other international work on league tables—which has also identified both cost-saving and highly cost-effective interventions (see  Background ). This work collectively should therefore be considered by policymakers to guide research priorities and to inform at least the cost-effectiveness aspect of future intervention selection. Similarly, the league table work aimed at low and middle-income countries can sometimes identify interventions that could be considered for high-income countries like New Zealand (e.g., polypills for cardiovascular disease prevention, dietary salt reduction policies, and regulation around food advertisements) [4].

Study strengths and limitations

A strength of this study is that it was the first attempt to produce a league table of interventions covering methodologically diverse health interventions for this particular high-income country. It also included relatively few studies that had industry funding (Table 2) which may reduce bias in the results [43]. Nevertheless, this work has a number of limitations as detailed below.

Firstly, the search strategy might have missed some published studies given that occasionally cost-effectiveness results might only be found in appendicised information and hence are not readily identified in literature searches. Publication bias may have resulted in some studies not appearing in the peer-reviewed literature if authors or journal editors/reviewers considered the results to be of relatively little interest to policymakers (e.g., perhaps for studies which reported very cost-ineffective interventions).

Secondly, there are limitations with the threshold we used for categorising cost-effectiveness (in terms of GDP per capita as per the WHO CHOICE approach [16]). Such an approach has been criticised in the literature (e.g., in several articles [44,45,46]), with a particular weakness being that it is not linked to the shadow price of a health system’s budget constraint. Nevertheless, its use here can be justified given that it provides some link to real-world resourcing capacity and because of the absence of agreed alternative options proposed for this particular case study country.

Thirdly, this study did not attempt to include the components of the size of the health gain and cost impacts in the league table because this is very challenging due to the need to allow for varying population sizes, discerning total population and target population focus, and standardising the time horizon. There was also still variation in the outcome measures (i.e., QALY/DALY/LY) and, of course, the size of these varies with the different discount rates used (hence differentially impacting the ICER results).

Fourthly, more sophisticated critiques of the identified studies are possible (e.g., applying the CHEERS checklist [42] or detailed comparisons with state-of-the-art guidelines for CEAs [18]). However, this was not justified given the results in Table 3 that already show substantive deviation from established guidelines for many of the studies (e.g., in the discount rate).

Conclusions

It was possible to produce a reasonably coherent league table for the ICER values from different studies (within broad groupings) in this high-income country. Most interventions were cost-effective and a fifth were cost-saving. Nevertheless, study methodologies did vary widely and researchers need to pay more attention to using standardised methods that will allow their results to be included in future league tables.