The Mismeasure of Risk

Abstract

Tsunamis, volcanic ash clouds, financial crashes, and oil blowouts are recent disasters that have caught us by surprise and dominated world headlines. It seems that we are just not very good at predicting such risks or dealing with their consequences. This chapter looks at one reason this might be so – that we are not measuring risk appropriately, either in how we assess a risk or how we then make decisions about risk-related problems. Our measurements are conditioned by expectations of precision and simplicity which, we will argue, are too often lacking in the real world.

In assessing a risk we can miss potential outcomes, especially when a system changes over time. We can treat our theories as true irrespective of empirical support, and we can set impractical thresholds of acceptable risk. When we do measure the risk, we may not have enough data of sufficient quality, or behavioral responses may reduce the measured risk yet increase risk that is not measured. In short, our current assessments of risk are too blinkered.

The associated blunders in decision-making are also all too familiar – disregarding a risk because the model says the chance of the event occurring is low even though we have little confidence in the model; conversely, taking an unduly “precautionary” approach to avoid risks when the potential for harm is negligible; worst of all, perhaps, missing the wider decision-making context needed for a sound judgment.

Put together, these factors mean that simple measures of risk can be a poor guide for decision-makers. This chapter advocates five extensions to current methods in order to avoid these pitfalls in the future. In risk assessment, we need to:

• 1. Estimate unknown unknown risk

• 2. Quantify model risk

• 3. Use multiple measures and thresholds; and in decision support

• 4. Balance risk and reward; and

• 5. Examine ROB (Risk Outside the Box)

We will describe examples where, had certain measures of risk been in place, disasters might have been averted. We will then show how each of the extended methods may be applied. Two particular applications of the proposed approach will be provided – “RAG” statuses in project and risk management, and Solvency II in the insurance industry.