1 Advantages of Net Benefit

Dr Paulden has made the strong statement that it is time to abandon the incremental cost-effectiveness ratio (ICER) [1]. His call is supported by an accompanying tutorial article on the advantages of the net benefit (NB) approach over the ICER [2]. Strong statements deserve cross-examination. This editorial questions whether it is necessary to abandon the ICER in order to fully embrace the benefits of NB. At the outset, I should make my disclosures clear that Mike and I are research collaborators and friends.

The central task of cost-effectiveness analysis (CEA) is to identify strategies that maximise health within finite budgets. Paulden notes that the ICER and NB perform this task equally well. He outlines the advantages NB has, both in terms of being easier to calculate and interpret as well as informing decision makers beyond simply picking the optimal strategy.

NB undoubtedly has important advantages over ICERs and these have been ably articulated by Paulden. Chief among these for me is the ability to appraise the cost effectiveness of dominated and dominant strategies. Dominated strategies may be of interest if they support other objectives such as satisfying patient preferences. Another substantive advantage is added clarity in sensitivity and scenario analysis. In particular, NB permits analysis of changes in relative cost effectiveness across multiple strategies in a way that is not possible with ICERs.

2 Critiquing Paulden’s Arguments

I want to briefly critically appraise some of Paulden’s other arguments in favour of NB before considering if there is any remaining role for the ICER. First, Paulden explains calculating ICERs is more laborious than NB. While true, in the context of the significant analytical effort of a typical CEA, this extra effort is arguably marginal.

Paulden also states that interpretation of the ICER is not intuitive, with the tutorial supporting this with examinations of ICERs in the southwest quadrant of the cost-effectiveness plane and negative ICERs. I consider this criticism a little overstated and its exposition unnecessarily complex. It is established that negative ICERs for the northwest and southeast quadrants should not be reported as they have no meaningful interpretation [3]. Accordingly, there are no negative ICERs to misinterpret; rather, the advantage of NB appears to be its ability to quantify the extent of dominance. Regarding the southwest quadrant, the complication seems to stem from Paulden’s assumption that the ICER belongs to the less effective strategy, rather than that strategy serving as the comparator. If, as commonly assumed, the ICER belongs to the more effective strategy, then this confusion disappears.

Equity weighting is likely to become an increasingly important part of CEA. Paulden’s commentary explains that this is better handled using NB than using ICERs, although this issue is not explored in the tutorial article. It is my understanding that the relevant distinction here is not between NB and ICERs but whether equity weights are applied to health outcomes or the cost-effectiveness threshold, with the former approach being superior. It should be possible to generate equivalent equity-weighted results with ICERs and NB. Accordingly, I do not believe an advantage of handling equity weights is inherent to NB.

3 Benefits of the Incremental Cost-Effectiveness Ratio

I now turn to consider what might be some of the residual benefits of the ICER. These are admittedly not ‘hard’ benefits of technical superiority but rather ‘soft’ benefits of ease of interpretation. Paulden notes that an advantage of ICERs is that their calculation does not require the specification of a threshold but that they cannot be meaningfully interpreted without a threshold. This is manifestly true, yet it seems to overlook the potential advantages of ICERs without the definition of a threshold. The adoption of the cost-effectiveness ratio decades ago at the birth of CEA rather than monetising health benefits may have been a shrewd step to avoid the thorny question placing a price on life. In part, this permits analysis while allowing the commitment to a threshold to remain behind the closed doors of decision makers’ committee rooms. Furthermore, it permits analyses in contexts in which there is no official cost-effectiveness ratio, regrettably which is still most countries worldwide.

ICERs permit an added degree of transparency. On occasion while reading US CEAs, I have performed a double-take as interventions with ICERs well in excess of $100,000/quality-adjusted life-year (QALY) are declared cost effective; reading further I see the authors have conveniently assumed a threshold of $200,000/QALY. Using NB means that such unrealistic thresholds will be ‘baked-in’ to cost-effectiveness estimates.

The recent and overdue efforts of CEAs to estimate empirical thresholds may finally put an end to the era of undefined, fudged and excessive thresholds. This would facilitate greater application of NB. Despite this, I still see some advantages in reporting ICERs. An analyst can, using the ICERs alone, determine which interventions are more effective and more expensive than others. In my specific research context of cancer screening, this typically permits rapid identification of which strategies are too intense to justify and which are suboptimally intense. Furthermore, ICERs allow the analyst to determine at a glance which intervention would be optimal if a different threshold was applied. Of course, this can be achieved with NB, as Paulden explains, but it requires the tabulation of additional results for different thresholds. The analyst clearly cannot anticipate all thresholds that may be of interest to readers from different contexts.

It has always puzzled me why Stinnet and Mullahy’s NB metric has not been more widely adopted. A possible explanation is that NB estimates do not immediately convey anything in isolation. A positive NB indicates an intervention is cost effective relative to no intervention, but it is only when compared with NB of other strategies can we determine if the intervention is cost effective or not. Conversely, the ICER appears to convey something intuitive to us: ICERs above the threshold indicate interventions too expensive to justify, while ICERs below the threshold indicate we can possibly do better.

4 Other Benefits of Net Benefit

I see some additional benefits of NB that Paulden did not record. While the ICER may appear intuitive, it is likely we are prone to over-interpret it. NB can help avoid this. For instance, an intervention with an ICER exceeding the threshold on the basis of only a small difference in costs and effects relative to the optimal strategy will only bring a small reduction in NB relative to the optimum. Conversely, a second intervention could have the same ICER on the basis of a very large difference in costs and effects. ICERs may lead us to the mistaken conclusion that each intervention is equally cost ineffective, whereas the NB shows the first intervention is less cost ineffective than the second.

Another benefit I see of NB is a reduction in the misidentification of cost-ineffective strategies as cost effective due to incorrectly calculated ICERs. Although mathematically simple, ICERs are still widely miscalculated. A recent review found nearly one-third of colorectal screening CEAs had failed to present appropriately incrementally assessed outcomes [4]. Analysts often compare interventions with no treatment or dominated strategies. Such errors cannot occur using NB.

A soft advantage of NB is that it may concentrate decision makers’ minds on the opportunity cost of cost-ineffective interventions. Decision makers have been known to struggle with health economic metrics [5, 6]. I suspect for non-health economists the natural resistance to placing a money value on health is reinforced by the ICER as cost appears explicitly in the numerator. Indeed, I speculate that some decision makers may believe that by approving treatments with high ICERs, they demonstrate compassion, generosity and a high valuation of health. To make the opportunity costs of cost-ineffective approvals clearer, some analysts have explicitly quantified the cost of care in terms of QALYs foregone [7]. Similarly, using NB and expressing it in terms of net health benefit may assist health economists in their mission to convince others that the choices are ultimately about health not money.

5 Conclusion

NB clearly has important advantages over ICERs in the estimation and communication of cost effectiveness. It seems to me there are some remaining benefits of using ICERs. The question then seems, do we do any harm to decision making by continuing to use ICERs alongside NB? I cannot see any serious, active harm posed by ICERs. An indirect harm may result from ongoing reliance on ICER that inhibits greater adoption of NB. However, in my view, I see sufficient remaining usefulness of the ICER to merit its retention alongside NB.