Physiological and behavioral signatures of reflective exploratory choice
Physiological arousal, a marker of emotional response, has been demonstrated to accompany human decision making under uncertainty. Anticipatory emotions have been portrayed as basic and rapid evaluations of chosen actions. Instead, could these arousal signals stem from a “cognitive” assessment of value that utilizes the full environment structure, as opposed to merely signaling a coarse, reflexive assessment of the possible consequences of choices? Combining an exploration–exploitation task, computational modeling, and skin conductance measurements, we find that physiological arousal manifests a reflective assessment of the benefit of the chosen action, mirroring observed behavior. Consistent with the level of computational sophistication evident in these signals, a follow-up experiment demonstrates that anticipatory arousal is modulated by current environment volatility, in accordance with the predictions of our computational account. Finally, we examine the cognitive costs of the exploratory choice behavior these arousal signals accompany by manipulating concurrent cognitive demand. Taken together, these results demonstrate that the arousal that accompanies choice under uncertainty arises from a more reflective and “cognitive” assessment of the chosen action’s consequences than has been revealed previously.
KeywordsDecision-making Reward Reinforcement learning Emotion Arousal
The experiments reported here were part of A.R.O.’s doctoral dissertation at the University of Texas at Austin. During this period A.R.O. was supported by a Mike Hogg Endowment Fellowship from the University of Texas at Austin. The authors thank Todd Gureckis, Russ Poldrack, Alex Huk, Nathaniel Daw, Tom Schönberg, Yael Niv and Tyler Davis for helpful conversations.
- Cassandra, A., Littman, M. L., & Zhang, N. L. (1997). Incremental pruning: A simple, fast, exact method for partially observable Markov decision processes. In Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence, 54–61.Google Scholar
- Damasio, A. (1994). Descartes’ error: Emotion, reason, and the human brain. New York: Putnam.Google Scholar
- Gershman, S. J., Markman, A. B., & Otto, A. R. (2014). Retrospective revaluation in sequential decision making: A tale of two systems. Journal of Experimental Psychology: General, 143(1), 182–194.Google Scholar
- Humphries, M. D., Khamassi, M., & Gurney, K. (2012). Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. Frontiers in Decision Neuroscience, 6, 9.Google Scholar
- Ledoux, J. (1996). The emotional brain: The mysterious underpinnings of emotional life. New York: Simon and Schuster.Google Scholar
- Otto, A. R., Raio, C. M., Chiang, A., Phelps, E. A., & Daw, N. D. (2013b). Working-memory capacity protects model-based learning from stress. Proceedings of the National Academy of Sciences, 110(52), 20941–20946.Google Scholar
- Preuschoff, K., Hart, B., & Einhäuser, W. (2011). Pupil dilation signals surprise: Evidence for noradrenaline’s role in decision making. Frontiers in Decision Neuroscience, 5, 115.Google Scholar
- Rolls, E. T. (1999). The brain and emotion. Oxford: Oxford University Press.Google Scholar
- Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning. Cambridge, MA: MIT Press.Google Scholar
- Worthy, D. A., Hawthorne, M. J., & Otto, A. R. (2013). Heterogeneity of strategy use in the Iowa gambling task: A comparison of win-stay/lose-shift and reinforcement learning models. Psychonomic Bulletin & Review, 20(2), 364–371.Google Scholar
- Wunderlich, K., Dayan, P., & Dolan, R. J. (2012). Mapping value based planning and extensively trained choice in the human brain. Nature Neuroscience. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/22406551