The impact of biases on simulation-based risk aggregation: modeling cognitive influences on risk assessment
- 255 Downloads
This paper develops a systematic approach to quantifying the effect of judgmental biases on aggregate risk measures. Starting with the standard risk management process, we derive the areas that require expert judgment as input in order to aggregate risk into risk measures such as Earnings at Risk. We specify three possible gateways for biases and identify several psychological theories to quantify deviations of expert judgments from objective probabilities. The impact of these cognitive biases on the aggregate risk measure is investigated via Monte Carlo simulation experiments. Through experimental design, we can determine the size of both the average and the possible interaction effects of the different biases. The results show that aggregate risk is systematically underestimated if it is based on biased subjective judgment. Moreover, the existence of interaction effects indicates potential problems of simple debiasing strategies.
KeywordsCognitive biases Experimental design Expert judgment Risk management Simulation
- Antony, J. (2003). Design of experiments for engineers and scientists. Amsterdam: Butterworth-Heinemann. Google Scholar
- Aragonés, J. R., Blanco, C., & Dowd, K. (2001). Incorporating stress tests into market risk modeling. Derivatives Quarterly, 7(3), 44–49. Google Scholar
- Baron, J. (2008). Thinking and deciding (4th ed.). Cambridge: Cambridge Univ. Press. Google Scholar
- Basel Committee on Banking Supervision (2009). Principles for sound stress testing practices and supervision: [final paper]. Bank for International Settlements, Basel. http://www.bis.org/publ/bcbs147.pdf?noframes=1. Accessed 28 April 2010.
- Bedford, T., & Cooke, R. (2001). Probabilistic risk analysis: foundations and methods. Cambridge: Cambridge Univ. Press. Google Scholar
- Berkowitz, J. (1999). A coherent framework for stress-testing. Board of Governors of the Federal Reserve, Washington, 1–14. Google Scholar
- Brenner, L. A., Koehler, D. J., & Rottenstreich, Y. (2008). Remarks on support theory: recent advances and future directions. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases: the psychology of intuitive judgment (pp. 489–509). Cambridge: Cambridge Univ. Press. Google Scholar
- Clemen, R. T., & Lichtendahl, K. C. Jr. (2002). Debiasing expert overconfidence: a Bayesian calibration model. In Conference proceedings of the 6th international conference on probabilistic safety assessment and management, pp. 1–16. Google Scholar
- Cooke, R. M. (1991). Experts in uncertainty: opinion and subjective probability in science. New York: Oxford Univ. Press. Google Scholar
- COSO (2004). Enterprise risk management—integrated framework: executive summary. http://www.coso.org/documents/COSO_ERM_ExecutiveSummary.pdf. Accessed 14 June 2011.
- Dowd, K. (2003). Beyond value at risk: the new science of risk management. Chichester: Wiley. Google Scholar
- Gilbert, N., & Troitzsch, K. G. (2005). Simulation for the social scientist (2nd ed.). Maidenhead: Open Univ. Press. Google Scholar
- Griffin, D., & Brenner, L. A. (2005). Perspectives on probability judgment calibration. In D. J. Koehler & N. Harvey (Eds.), Blackwell handbook of judgment and decision making (pp. 177–199). Oxford: Blackwell. Google Scholar
- Griffin, D., Gonzalez, R., & Varey, C. (2001). The heuristics and biases approach to judgment under uncertainty. In A. Tesser & N. Schwarz (Eds.), Blackwell handbook of social psychology: intra individual processes (pp. 207–235). Malden: Blackwell. Google Scholar
- Institut der Wirtschaftsprüfer [IDW] (1999). IDW Prüfungsstandard: Die Prüfung des Risikofrüherkennungssystems nach § 317 Abs. 4 HGB (IDW PS 340). WPg, 52(16), 658–662. Google Scholar
- Institut der Wirtschaftsprüfer [IDW] (2006). WP Handbuch 2006. Düsseldorf: IDW Verlag GmbH. Google Scholar
- Jennings, D. L., Amabile, T. M., & Ross, L. (1982). Informal covariation assessment: data-based versus theory-based judgments. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: heuristics and biases (23rd ed., pp. 211–230). Cambridge: Cambridge Univ. Press. Google Scholar
- Jorion, P. (2007). Value at risk: the new benchmark for managing financial risk (3rd ed.). New York: McGraw-Hill. Google Scholar
- Kelton, D. W., & Barton, R. R. (2003). Experimental design for simulation. In Proceedings of the 2003 winter simulation conference (pp. 59–65). Google Scholar
- Keppel, G., & Wickens, T. D. (2004). Design and analysis: a researcher’s handbook (4th ed.). Upper Saddle River: Pearson Prentice Hall. Google Scholar
- Kleijnen, J. P. C. (2008). Design and analysis of simulation experiments. Boston: Springer. Google Scholar
- Koehler, D. J., Brenner, L. A., & Griffin, D. (2008). The calibration of expert judgment: heuristics and biases beyond the laboratory. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases: the psychology of intuitive judgment (pp. 686–715). Cambridge: Cambridge Univ. Press. Google Scholar
- Law, A. M. (2007). Simulation modeling and analysis (4th ed.). Boston: McGraw-Hill. Google Scholar
- Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982). Calibration of probabilities: the state of the art to 1980. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: heuristics and biases (23rd ed., pp. 306–334). Cambridge: Cambridge Univ. Press. Google Scholar
- Lorscheid, I., Heine, B., & Meyer, M. (2011). Opening the ‘black box’ of simulations: increased transparency of simulation models and effective results reports through the systematic design of experiments. CMOT, forthcoming. Google Scholar
- McClelland, A. G. R., & Bolger, F. (1994). The calibration of subjective probabilities: theories and models 1980–94. In G. Wright & P. Ayton (Eds.), Subjective probability (pp. 453–482). Chichester: Wiley. Google Scholar
- Meyer, M. A., & Booker, J. M. (1991). Eliciting and analyzing expert judgment: a practical guide. London: Academic Press. Google Scholar
- Montgomery, D. C. (2005). Design and analysis of experiments (6th ed.). Hoboken: Wiley. Google Scholar
- SOX (2002). Sarbanes-Oxley act of 2002 (Enrolled Bill [Final as Passed Both House and Senate]—ENR), The Library of Congress. http://thomas.loc.gov/cgi-bin/query/z?c107:H.R.3763.ENR. Accessed 11 April 2011.
- Vose, D. (1996). Quantitative risk analysis: a guide to Monte Carlo simulation modelling. Chichester: Wiley. Google Scholar
- Vose, D. (2008). Risk analysis: a quantitative guide (3rd ed.). Chichester: Wiley. Google Scholar
- Wieske, D., & Van der Meer, R. (2006). Monte Carlo simulations and corporate risk management in Germany. http://api.ning.com/files/og1Ma9CrBKYp6Mt-3cQ2LZvZEjnTYEKkQOb3dFu7nwbkgIMriyO*NH2rCvwbfYwuBcx6pUdL*IiiYhdlDWqMQfJrKnowuNB2/MonteCarloandearningsatrisk.pdf. Accessed 14 June 2011.
- Yates, J. F. (1990). Judgment and decision making. Englewood Cliffs: Prentice Hall. Google Scholar