Taking the reins: how regulatory decision-makers can stop being hijacked by uncertainty

Abstract

Several decades after the mechanics of quantitative uncertainty analysis (QUA) for risk assessment and regulatory cost analysis were developed and refined, QUA still rarely reaches the minds of decision-makers. The most common justification for this situation is that “decision-makers want a number, not a set of statistical distributions.” This may be an accurate assessment of their druthers, but one obvious though perhaps impractical retort is to say that if decision-makers insist on misleading point estimates, then we need new and better decision-makers. This article offers a way out of this dilemma. Decision-makers do not have to understand (or even receive) all the information contained in a complete QUA, but they do have to drive the QUA. They need to instruct analysts how to approach the phenomena they analyze (parameter uncertainty, model uncertainty, interindividual variability, offsetting and second-order effects, and the monetary value of future uncertainty reductions), they need to insist that uncertainties in cost be treated a priori as exactly as important as uncertainties in risk, and—even more importantly—they need to instruct analysts which estimator(s) to seek, report, and explain. Here we offer 10 detailed principles to guide decision-makers into a new relationship with risk and cost analysts—10 observations about how “eyes wide open” point estimates can vastly outperform point estimates handed to the decision-maker without context, justification, or honesty about the value judgments they impose upon the decision. A decision-maker who explains “I chose Option A because its benefits of 2.345 exceed its costs of 1.234” can be replaced by a dollar-store calculator. We need decision-makers who can say “I chose Option A because the spectrum of benefits it likely offers, to these citizens, considering the range of costs it likely imposes, makes it a superior choice to any other.” QUA, performed carefully and following clear policy instructions, can empower decision-makers to earn their influential roles.