Advertisement

Decision-Making Under Uncertainty

  • Dominik R. BachEmail author
Chapter
Part of the Studies in Neuroscience, Psychology and Behavioral Economics book series (SNPBE)

Abstract

All decision-making takes place under uncertainty, even in controlled laboratory circumstances. The Bayesian brain hypothesis, a widely accepted theoretical framework of brain function, prescribes that the brain uses probability distributions to store parameter values, rather than point estimates, and is thus able to use uncertainty on various parameters. This allows for investigating value-based decision-making under natural circumstances when information needs to be extracted from noisy input, and it may also impact on decisions based on propositional information. In this chapter, I present experimental approaches to neural representations of uncertainty in value-based decision-making.

Keywords

Outcome Distribution Expect Utility Theory Ambiguity Aversion White Ball Player Type 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgments

I would like to thank Quentin Huys, Marc Guitart Masip, Deborah Talmi and Matthias Staib, for helpful comments on a first draft of this manuscript.

References

  1. Abler B, Herrnberger B, Gron G, Spitzer M (2009) From uncertainty to reward: BOLD characteristics differentiate signaling pathways. BMC. Neurosci. 10:154Google Scholar
  2. Aron AR, Shohamy D, Clark J, Myers C, Gluck MA, Poldrack RA (2004) Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning. J Neurophysiol 92:1144–1152CrossRefPubMedGoogle Scholar
  3. Bach DR, Dolan RJ (2012) Knowing how much you don’t know: a neural organization of uncertainty estimates. Nat Rev Neurosci 13:572–586PubMedGoogle Scholar
  4. Bach DR, Seymour B, Dolan RJ (2009) Neural activity associated with the passive prediction of ambiguity and risk for aversive events. J Neurosci 29:1648–1656CrossRefPubMedPubMedCentralGoogle Scholar
  5. Bach DR, Hulme O, Penny WD, Dolan RJ (2011) The known unknowns: neural representation of second-order uncertainty, and ambiguity. J Neurosci 31:4811–4820CrossRefPubMedPubMedCentralGoogle Scholar
  6. Battaglia PW, Schrater PR (2007) Humans trade off viewing time and movement duration to improve visuomotor accuracy in a fast reaching task. J Neurosci 27:6984–6994CrossRefPubMedGoogle Scholar
  7. Becker SW, Brownson FO (1964) What price ambiguity? Or the role of ambiguity in decision making. J Polit Econ 72:62–73CrossRefGoogle Scholar
  8. Behrens TE, Woolrich MW, Walton ME, Rushworth MF (2007) Learning the value of information in an uncertain world. Nat Neurosci 10:1214–1221CrossRefPubMedGoogle Scholar
  9. Bernoulli D (1738) Specimen theoriae novae de mensura sortis. Commentarii Academiae Scientiarum Imperialis Petropolitanae 5:175–192Google Scholar
  10. Brouwer AM, Knill DC (2009) Humans use visual and remembered information about object location to plan pointing movements. J Vis 9:24–19CrossRefPubMedPubMedCentralGoogle Scholar
  11. Camerer C (1995) Individual decision making. In: Kagel JH, Roth AE (eds) Handbook of experimental economics. Princeton University Press, Princeton, pp 587–704Google Scholar
  12. Camerer C (1999) Ambiguity aversion and non-additive probability: experimental evidence, models and applications. In: Luini L (ed) Uncertain decisions: bridging theory and experiments. Kluwer, Boston, pp 53–80CrossRefGoogle Scholar
  13. Chow CC, Sarin RK (2002) Known, unknown, and unknowable certainties. Theor Decis 52:127–138CrossRefGoogle Scholar
  14. Curley SP, Yates F, Abrams RA (1986) Psychological sources of ambiguity avoidance. Organ Behav Hum Decis Process 38:230–256CrossRefGoogle Scholar
  15. d’Acremont M, Lu ZL, Li X, Van der LM, Bechara A (2009) Neural correlates of risk prediction error during reinforcement learning in humans. NeuroImage 47:1929–1939CrossRefPubMedGoogle Scholar
  16. d’Acremont M, Bossaerts P (2008) Neurobiological studies of risk assessment: a comparison of expected utility and mean-variance approaches. Cogn Affect Behav Neurosci 8:363–374CrossRefPubMedGoogle Scholar
  17. Dayan P, Hinton GE (1996) Varieties of Helmholtz machine. Neural Netw 9:1385–1403CrossRefPubMedGoogle Scholar
  18. Dayan P, Hinton GE, Neal RM, Zemel RS (1995) The Helmholtz machine. Neural Comput 7:889–904CrossRefPubMedGoogle Scholar
  19. Dreher JC, Kohn P, Berman KF (2006) Neural coding of distinct statistical properties of reward information in humans. Cereb Cortex 16:561–573CrossRefPubMedGoogle Scholar
  20. Ellsberg D (1961) Risk, ambiguity, and the savage axioms. Quart J Econ 75:643–669CrossRefGoogle Scholar
  21. Faisal AA, Wolpert DM (2009) Near optimal combination of sensory and motor uncertainty in time during a naturalistic perception-action task. J Neurophysiol 101:1901–1912CrossRefPubMedGoogle Scholar
  22. Fitzgerald TH, Seymour B, Bach DR, Dolan RJ (2010) Differentiable neural substrates for learned and described value and risk. Curr Biol 20:1823–1829CrossRefPubMedPubMedCentralGoogle Scholar
  23. Friston K (2009) The free-energy principle: a rough guide to the brain? Trends Cogn Sci 13:293–301CrossRefPubMedGoogle Scholar
  24. Friston K (2010) The free-energy principle: a unified brain theory? Nat Rev Neurosci 11:127–138CrossRefPubMedGoogle Scholar
  25. Huettel SA, Stowe CJ, Gordon EM, Warner BT, Platt ML (2006) Neural signatures of economic preferences for risk and ambiguity. Neuron 49:765–775CrossRefPubMedGoogle Scholar
  26. Keren G, Gerritsen LEM (1999) On the robustness and possible accounts of ambiguity aversion. Acta Psychol 103:149–172CrossRefGoogle Scholar
  27. Klibanoff P, Marinacci M, Mukerji S (2005) A smooth model of decision making under ambiguity. Econometrica 73:1849–1892CrossRefGoogle Scholar
  28. Knill DC, Pouget A (2004) The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci 27:712–719CrossRefPubMedGoogle Scholar
  29. Kording KP, Wolpert DM (2004) Bayesian integration in sensorimotor learning. Nature 427:244–247CrossRefPubMedGoogle Scholar
  30. Kording KP, Wolpert DM (2006) Bayesian decision theory in sensorimotor control. Trends in cognitive sciences 10:319–326CrossRefPubMedGoogle Scholar
  31. Kording KP, Ku SP, Wolpert DM (2004) Bayesian integration in force estimation. J Neurophysiol 92:3161–3165CrossRefPubMedGoogle Scholar
  32. Larson JR (1980) Exploring the external validity of a subjectively weighted utility model of decision making. Organ Behav Hum Performance 26:293–304CrossRefGoogle Scholar
  33. MacCrimmon KR, Larson S (1979) Utility theory: axioms versus ‘paradoxes’. In: Allais M, Hagen O (eds) Expected utility hypotheses and the allais paradox. D. Reidel, Dordrecht, pp 333–410CrossRefGoogle Scholar
  34. Mackintosh NJ (1983) Conditioning and associative learning. Oxford University Press, OxfordGoogle Scholar
  35. Markowitz H (1952) Portfolio selection. J Finan 7:77–91Google Scholar
  36. Mathys C, Daunizeau J, Friston KJ, Stephan KE (2011) A Bayesian foundation for individual learning under uncertainty. Front Hum Neurosci 5:39CrossRefPubMedPubMedCentralGoogle Scholar
  37. Mohr PN, Biele G, Krugel LK, Li SC, Heekeren HR (2010) Neural foundations of risk-return trade-off in investment decisions. NeuroImage 49:2556–2563CrossRefPubMedGoogle Scholar
  38. Muthukrishnan AV, Wathieu L, Xu AJ (2009) Ambiguity aversion and the preference for established brands. Manage Sci 55:1933–1941CrossRefGoogle Scholar
  39. Ogawa M, van der Meer MA, Esber GR, Cerri DH, Stalnaker TA, Schoenbaum G (2013) Risk-responsive orbitofrontal neurons track acquired salience. Neuron 77:251–258CrossRefPubMedPubMedCentralGoogle Scholar
  40. O’Neill M, Schultz W (2010) Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68:789–800CrossRefPubMedGoogle Scholar
  41. Orban G, Wolpert DM (2011) Representations of uncertainty in sensorimotor control. Curr Opin Neurobiol 21:629–635CrossRefPubMedGoogle Scholar
  42. Parise CV, Knorre K, Ernst MO (2014) Natural auditory scene statistics shapes human spatial hearing. Proc Natl Acad Sci USA 111:6104–6108CrossRefPubMedPubMedCentralGoogle Scholar
  43. Pearce JM, Hall G (1980) A model for Pavlovian learning—variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532–552CrossRefPubMedGoogle Scholar
  44. Pouget A, Beck JM, Ma WJ, Latham PE (2013) Probabilistic brains: knowns and unknowns. Nat Neurosci 16:1170–1178CrossRefPubMedPubMedCentralGoogle Scholar
  45. Preuschoff K, Bossaerts P, Quartz SR (2006) Neural differentiation of expected reward and risk in human subcortical structures. Neuron 51:381–390CrossRefPubMedGoogle Scholar
  46. Pulford BD, Colman AM (2008) Size doesn’t really matter. ambiguity aversion in Ellsberg urns with few balls. Exp Psychol 55:31–37CrossRefPubMedGoogle Scholar
  47. Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, pp 64–99Google Scholar
  48. Rolls ET, McCabe C, Redoute J (2008) Expected value, reward outcome, and temporal difference error representations in a probabilistic decision task. Cereb Cortex 18:652–663CrossRefPubMedGoogle Scholar
  49. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599CrossRefPubMedGoogle Scholar
  50. Shafir S, Reich T, Tsur E, Erev I, Lotem A (2008) Perceptual accuracy and conflicting effects of certainty on risk-taking behaviour. Nature 453:917–920CrossRefPubMedGoogle Scholar
  51. Slovic P, Tversky A (1974) Who accepts Savage’s axiom? Behav Sci 19:368–373CrossRefGoogle Scholar
  52. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgeGoogle Scholar
  53. Symmonds M, Bossaerts P, Dolan RJ (2010) A behavioral and neural evaluation of prospective decision-making under risk. J Neurosci 30:14380–14389CrossRefPubMedPubMedCentralGoogle Scholar
  54. Symmonds M, Wright ND, Bach DR, Dolan RJ (2011) Deconstructing risk: separable encoding of variance and skewness in the brain. NeuroImageGoogle Scholar
  55. Tassinari H, Hudson TE, Landy MS (2006) Combining priors and noisy visual cues in a rapid pointing task. J Neurosci 26:10154–10163CrossRefPubMedGoogle Scholar
  56. Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642–1645CrossRefPubMedGoogle Scholar
  57. Tobler PN, O’Doherty JP, Dolan RJ, Schultz W (2007) Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J Neurophysiol 97:1621–1632CrossRefPubMedGoogle Scholar
  58. Trautmann ST, Vieider FM, Wakker PP (2008) Causes of ambiguity aversion: known versus unknown preferences. J Risk Uncertainty 36:225–243CrossRefGoogle Scholar
  59. Trommershauser J, Kording K, Landy MS (2011) Sensory cue integration. Oxford University Press, OxfordCrossRefGoogle Scholar
  60. Vilares I, Kording K (2011) Bayesian models: the structure of the world, uncertainty, behavior, and the brain. Ann NY Acad Sci 1224:22–39CrossRefPubMedPubMedCentralGoogle Scholar
  61. von Neumann J, Morgenstern O (1944) Theory of games and economic behavior. Princeton University Press, PrincetonGoogle Scholar
  62. Yates JF, Zukowski LG (1976) Characterization of ambiguity in decision-making. Behav Sci 21:19–25CrossRefGoogle Scholar
  63. Yu AJ, Dayan P (2005) Uncertainty, neuromodulation, and attention. Neuron 46:681–692CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.University of ZurichZurichSwitzerland
  2. 2.University College LondonLondonUK

Personalised recommendations