Dear Editor,

Vreman et al. [1] regard a probabilistic deterministic sensitivity analysis (DSA) as “the most appropriate method for providing insight on the effect of uncertainty in individual parameters on the estimate of cost effectiveness”. Furthermore, they “advocate to at least implement distributional DSA in all cases”. Distributional DSA aims at demonstrating outcomes for a number of intermediate steps between the base case and the minimum and maximum values that are based on percentiles of the probability density function of the parameter. The authors present the results of the distributional and probabilistic DSA in a tornado diagram.

In my opinion, the approach suggested by the authors is intuitively appealing if a probability distribution is known. Nevertheless, the authors implicitly assume that assigning probabilities is always preferable to not assigning probabilities. This assumption, however, is contentious when decision-makers must decide under complete ignorance, that is, when they do not have information about probabilities [2]. In this situation, assigning equal probabilities (based on the Laplace rule) can lead to preference reversals [3]. Hence, when deciding under ignorance it is not rational to apply a distributional or probabilistic DSA (based on an equiprobability assumption). Instead, under ignorance, it is rational to make decisions only based on the extreme values, without assigning probabilities [3]. This approach aligns well with the original purpose of a tornado diagram, which has been described as a graphical illustration of the pessimistic and optimistic values for each uncertain variable [4]. That is, the classic tornado diagram does not intend to make a statement about the probability of the values that occur in-between the extreme values.

The authors do caution against assigning equal probabilities. Instead of dropping probabilities, however, they suggest using (true) probability distributions. Yet, under ignorance it would be preferable not to assign any probabilities and, instead, make decisions based on the extreme values analyzed in univariate sensitivity analyses and illustrated by a classical tornado diagram. While it is true that correlations between variables need to be considered, their formal consideration needs to rely on probabilities. Instead, under ignorance, decision-makers may incorporate them implicitly in the optimistic and pessimistic values.

Finally, I would like to provide examples of variables for which information on probability distributions is lacking. Examples are long-term effectiveness, side effects, or downstream costs of new technologies [5]. Furthermore, some authors have advocated that cost-effectiveness analysis should go beyond the consideration of present real-world conditions and incorporate life-cycle or dynamic aspects. Life-cycle aspects include entry of biosimilars in an orphan drug designation and the arrival of technological advancements in the future [6]. While it may be appropriate to assign a specific probability distribution for the former event based on experience, for the latter it may not be. Instead, it seems more appropriate to use a pessimistic–optimistic range, that is, to model 'yes' and 'no' scenarios. In more general terms, all variables to which uniform probability distributions would be applied otherwise due to a lack of information on probabilities appear to be candidates for the classic tornado diagram. Another, very recent example of decision making under ignorance is the coronavirus crisis. Current uncertainties revolve around the ability of the vaccine(s) to prevent transmission of new variant strains, the harms and benefits in children (e.g., in terms of preventing long COVID), and whether new variant strains will render approved vaccines less effective.

In summary, I argue that the classic tornado diagram still has its place for displaying optimistic and pessimistic values for those variables and in those situations for which information on probability distributions is lacking. Still, more research needs to be conducted to define the conditions in which it is not appropriate to assume a probability distribution. In this respect, the coronavirus crisis may serve as an opportunity to fine-tune our modelling assumptions and methods.