The myth and fallacy of simple extrapolation in medicine

Abstract

Simple extrapolation is the orthodox approach to extrapolating from clinical trials in evidence-based medicine: extrapolate the relative effect size (e.g. the relative risk) from the trial unless there is a compelling reason not to do so. I argue that this method relies on a myth and a fallacy. The myth of simple extrapolation is the idea that the relative risk is a ‘golden ratio’ that is usually transportable due to some special mathematical or theoretical property. The fallacy of simple extrapolation is an unjustified argument from ignorance: we conclude that the relative effect size is transportable in the absence of evidence to the contrary. In short, simple extrapolation is a deeply problematic solution to the problem of extrapolation.

Appendix

We can model the question of whether or not to extrapolate the effect size using decision theory, by comparing the expected utility of extrapolating and intervening (EU_I) with the expected utility of not extrapolating and not intervening (EU_¬I). If we intervene with I, there are two possible scenarios: the intervention’s major benefit is generalizable (G), or the intervention’s major benefit is not generalizable (¬G). There is a partial expected utility associated with G (EU_G), as well as a partial expected utility associated with ¬G (EU_¬G). The EU_G depends on the benefits and harms of the intervention, as well as their probabilities. We can predict the probability of a beneficial outcome (B) or harmful outcome (H) using the effect size and the untreated outcome probability (e.g. RR = p(B|I)/p(B|¬I), thus: *p(B|I) = RR × p(B|¬I)). If we assume one major benefit B (the primary outcome targeted by the intervention) and one major harm H (a major side effect or adverse event), then *EU_G = p(B|I)u(B) + p(H|I)u(H), where p(B|I) and p(H|I) are the probabilities and u(B) and u(H) are the utilities (u(B) is positive and u(H) is negative). We can further assume that EU_¬G < EU_G; otherwise, there is no point in even considering intervention. Meanwhile, the total expected utility of intervening is: EU_I = p(G)EU_G + p(¬G)EU_¬G. Substituting the previous equations for *EU_G and *p(B|I):

$${\text{EU}}_{\text{I}} = {\text{p}}\left( {\text{G}} \right)\left[ {\left( {{\text{RR x p}}\left( {{\text{B}}|\neg {\text{I}}} \right)} \right){\text{u}}\left( {\text{B}} \right) + {\text{p}}\left( {{\text{H}}|{\text{I}}} \right){\text{u}}\left( {\text{H}} \right)} \right] + {\text{p}}\left( {\neg {\text{G}}} \right){\text{EU}}_{{\neg {\text{G}}}}$$

According to decision theory, we should implement the intervention if EU_I > EU_¬I. Simple extrapolation implies that this will usually be the case unless compelling evidence substantially raises the p(¬G) (and thus substantially lowers EU_I). However, EU_I also depends on EU_G and EU_¬G, and thus on the benefits and harms of intervention and their effect sizes. Therefore, even if p(¬G) is low to begin with (a risky assumption), EU_I might only be slightly greater than EU_¬I to begin with, and raising p(¬G) by just a little might tip the balance in favour of EU_¬I.