Incorporation of uncertainty in health economic modelling studies

Abstract

In a recent leading article in PharmacoEconomics, Nuijten described some methods for incorporating uncertainty into health economic models and for utilising the information on uncertainty regarding the cost effectiveness of a therapy in resource allocation decision-making. His proposals are found to suffer from serious flaws in statistical and health economic reasoning.

Nuijten’s suggestions for incorporating uncertainty: (a) wrongly interpret the p-value as the probability that the null hypothesis is true; (b) represent this probability wrongly by truncating the input distribution; and (c) in the specific example of an antiparkinsonian drug uses a completely inappropriate p-value of 0.05 when the null hypothesis would, in reality, be emphatically disproved by the data.

His suggestions regarding minimum important differences in cost effectiveness: (a) introduce areas of indifference that suggest inappropriate reliance on cost minimisation while failing to recognise that decisions should be based on expected costs versus benefits; and (b) offer no guidance on how the probabilities associated with these areas could be used in decision-making. Furthermore, Nuijten’s model for Parkinson’s disease is over-simplified to the point of providing a bad example of modelling practice, which may mislead the readers of PharmacoEconomics.

The rationale for this paper is to ensure that readers do not apply inappropriate analyses as a result of following the proposals contained in Nuijten’s paper. In addition to a detailed critique of Nuijten’s proposals, we provide brief summaries of the currently accepted best practice in cost-effectiveness decision-making under uncertainty.

Appendix A: A Brief Critique of Nuijten’s Illustrative Model

There are various ambiguities and imprecisions in Nuijten’s detailed description of the model, such that we have been unable to replicate the figures he reports in his table II. However, the description is sufficiently clear for various deficiencies to be apparent. The most notable problem is that the treatment effect is assumed to last only as long as the trial. It is not plausible that the biochemical processes that drive the treatment effect would stop simply because the trial follow-up had stopped. Whilst we may be increasingly uncertain about the scale of the effect as we extrapolate over time, the correct approach is to incorporate this uncertainty into the analysis, not to embed an indefensible assumption into the analysis.

A further problem with Nuijten’s model is the assumption that there is no mortality over the 5-year model time horizon. It is increasingly accepted that models of chronic conditions such as Parkinson’s disease should adopt a lifetime time horizon and incorporate mortality.[2,7,8]

There are further errors and confusions in the application of probabilistic sensitivity analysis in section 2.3. The rationale given for running 10 000 Monte Carlo simulations is wrong. The number of runs should be chosen to reduce the Monte Carlo sampling error in any required probabilistic sensitivity analysis output to a sufficiently small value for the desired inferences or decisions. The suggestion that one should use a log-normal distribution to represent uncertainty about mean costs because costs are log-normally distributed at the individual patient level is also misguided.