Opposing pupil responses to offered and anticipated reward values

Abstract

Previous studies have shown that the pupils dilate more in anticipation of larger rewards. This finding raises the possibility of a more general association between reward amount and pupil size. We tested this idea by characterizing macaque pupil responses to offered rewards during evaluation and comparison in a binary choice task. To control attention, we made use of a design in which offers occurred in sequence. By looking at pupil responses after choice but before reward, we confirmed the previously observed positive association between pupil size and anticipated reward values. Surprisingly, however, we find that pupil size is negatively correlated with the value of offered gambles before choice, during both evaluation and comparison stages of the task. These results demonstrate a functional distinction between offered and anticipated rewards and present evidence against a narrow version of the simulation hypothesis; the idea that we represent offers by reactivating states associated with anticipating them. They also suggest that pupil size is correlated with relative, not absolute, values of offers, suggestive of an accept–reject model of comparison.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Azab H, Hayden BY (2017) Correlates of decisional dynamics in the dorsal anterior cingulate cortex. PLoS Biol 15:e2003091. https://doi.org/10.1371/journal.pbio.2003091

    Article  PubMed  PubMed Central  Google Scholar 

  2. Azab H, Hayden BY (2018) Correlates of economic decisions in the dorsal and subgenual anterior cingulate cortices. Eur J Neursoci 47:979–993. https://doi.org/10.1111/ejn.13865

    Article  Google Scholar 

  3. Blanchard TC, Hayden BY (2015) Monkeys are more patient in a foraging task than in a standard intertemporal choice task. PloS one 10:e0117057. https://doi.org/10.1371/journal.pone.0117057

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  4. Blanchard TC, Wilke A, Hayden BY (2014a) Hot-hand bias in rhesus monkeys. J Exp Psychol: Anim Learn Cognit 40:280–286. https://doi.org/10.1037/xan0000033

    Article  Google Scholar 

  5. Blanchard TC, Wolfe LS, Vlaev I, Winston JS, Hayden BY (2014b) Biases in preferences for sequences of outcomes in monkeys. Cognition 130:289–299. https://doi.org/10.1016/j.cognition.2013.11.012

    Article  PubMed  PubMed Central  Google Scholar 

  6. Blanchard TC, Strait CE, Hayden BY (2015) Ramping ensemble activity in dorsal anterior cingulate neurons during persistent commitment to a decision. J Neurophysiol 114:2439–2449. https://doi.org/10.1152/jn.00711.2015

    Article  PubMed  PubMed Central  Google Scholar 

  7. Blanchard TC, Piantadosi ST, Hayden BY (2018) Robust mixture modeling reveals category-free selectivity in reward region neuronal ensembles. J Neurophysiol 119:1305–1318. https://doi.org/10.1152/jn.00808.2017

    Article  PubMed  Google Scholar 

  8. Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433–436

    Article  PubMed  CAS  Google Scholar 

  9. Britten KH, Shadlen MN, Newsome WT, Movshon JA (1992) The analysis of visual motion: a comparison of neuronal and psychophysical performance. J Neurosci 12:4745–4765

    Article  PubMed  CAS  Google Scholar 

  10. Britten KH, Newsome WT, Shadlen MN, Celebrini S, Movshon JA (1996) A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis Neurosci 13:87–100

    Article  PubMed  CAS  Google Scholar 

  11. Cisek P (2012) Making decisions through a distributed consensus. Curr Opin Neurobiol 22:927–936. https://doi.org/10.1016/j.conb.2012.05.007

    Article  PubMed  CAS  Google Scholar 

  12. Cornelissen FW, Peters EM, Palmer J (2002) The eyelink toolbox: eye tracking with MATLAB and the psychophysics toolbox. Behav Res Methods Instrum Comput 34:613–617

    Article  PubMed  Google Scholar 

  13. de Gee JW, Knapen T, Donner TH (2014) Decision-related pupil dilation reflects upcoming choice and individual bias. Proc Natl Acad Sci USA 111:E618–E625. https://doi.org/10.1073/pnas.1317557111

    Article  PubMed  CAS  Google Scholar 

  14. Ebitz RB, Hayden BY (2016) Dorsal anterior cingulate: a Rorschach test for cognitive neuroscience. Nat Neurosci 19:1278–1279. https://doi.org/10.1038/nn.4387

    Article  PubMed  CAS  Google Scholar 

  15. Efron B, Tibshirani RJ (1993) Permutation tests. In: An introduction to the bootstrap. Springer, pp 202–219

  16. Einhauser W, Stout J, Koch C, Carter O (2008) Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proc Natl Acad Sci USA 105:1704–1709. https://doi.org/10.1073/pnas.0707727105

    Article  PubMed  Google Scholar 

  17. Einhauser W, Koch C, Carter OL (2010) Pupil dilation betrays the timing of decisions. Front Hum Neurosci 4:18. https://doi.org/10.3389/fnhum.2010.00018

    PubMed  PubMed Central  Article  Google Scholar 

  18. Farashahi S, Azab H, Hayden B, Soltani A (2018) On the flexibility of basic risk attitudes in monkeys. J Neurosci 38:4383–4398. https://doi.org/10.1523/JNEUROSCI.2260-17.2018

    Article  PubMed  Google Scholar 

  19. Farovik A, Place RJ, McKenzie S, Porter B, Munro CE, Eichenbaum H (2015) Orbitofrontal cortex encodes memories within value-based schemas and represents contexts that guide memory retrieval. J Neurosci 35:8333–8344. https://doi.org/10.1523/JNEUROSCI.0134-15.2015

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  20. Freidin E, Aw J, Kacelnik A (2009) Sequential and simultaneous choices: testing the diet selection and sequential choice models. Behav Processes 80:218–223. https://doi.org/10.1016/j.beproc.2008.12.001

    Article  Google Scholar 

  21. Gamlin PD, Zhang H, Harlow A, Barbur JL (1998) Pupil responses to stimulus color, structure and light flux increments in the rhesus monkey. Vis Res 38:3353–3358

    Article  PubMed  CAS  Google Scholar 

  22. Geng JJ, Blumenfeld Z, Tyson TL, Minzenberg MJ (2015) Pupil diameter reflects uncertainty in attentional selection during visual search. Front Hum Neurosci 9:435. https://doi.org/10.3389/fnhum.2015.00435

    Article  PubMed  PubMed Central  Google Scholar 

  23. Hayden BY (2018) Economic choice: the foraging perspective. Curr Opin Behav Sci 24:1–6. https://doi.org/10.1016/j.cobeha.2017.12.002

    Article  Google Scholar 

  24. Hayden BY, Gallant JL (2013) Working memory and decision processes in visual area v4. Front Neurosci 7:18. https://doi.org/10.3389/fnins.2013.00018

    Article  PubMed  PubMed Central  Google Scholar 

  25. Hayden BY, Moreno-Bote R (2018) A neuronal theory of sequential economic choice. Brain Neurosci Adv 2:1–15. https://doi.org/10.1177/2398212818766675

    Article  Google Scholar 

  26. Hoeks B, Levelt WJ (1993) Pupillary dilation as a measure of attention: a quantitative system analysis. Behav Res Methods 25:16–26

    Article  Google Scholar 

  27. Howard JD, Gottfried JA, Tobler PN, Kahnt T (2015) Identity-specific coding of future rewards in the human orbitofrontal cortex. Proc Natl Acad Sci USA 112:5195–5200. https://doi.org/10.1073/pnas.1503550112

    Article  PubMed  CAS  Google Scholar 

  28. Hunt LT, Hayden BY (2017) A distributed, hierarchical and recurrent framework for reward-based choice. Nat Rev Neurosci 18:172–182. https://doi.org/10.1038/nrn.2017.7

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  29. Just MA, Carpenter PA, Miyake A (2003) Neuroindices of cognitive workload: neuroimaging, pupillometric and event-related potential studies of brain work. Theor Issues Ergon Sci 4:56–88

    Article  Google Scholar 

  30. Kacelnik A, Vasconcelos M, Monteiro T, Aw J (2011) Darwin’s “tug-of-war” vs. starlings’“horse-racing”: how adaptations for sequential encounters drive simultaneous choice. Behav Ecol Sociobiol 65:547–558

    Article  Google Scholar 

  31. Kahneman D, Beatty J (1966) Pupil diameter and load on memory. Science 154:1583–1585

    Article  PubMed  CAS  Google Scholar 

  32. Kahnt T, Heinzle J, Park SQ, Haynes JD (2010) The neural code of reward anticipation in human orbitofrontal cortex. Proc Natl Acad Sci USA 107:6010–6015. https://doi.org/10.1073/pnas.0912838107

    Article  PubMed  Google Scholar 

  33. Krajbich I, Armel C, Rangel A (2010) Visual fixations and the computation and comparison of value in simple choice. Nat Neurosci 13:1292–1298. https://doi.org/10.1038/nn.2635

    Article  PubMed  CAS  Google Scholar 

  34. Lavín C, San Martín R, Jubal ER (2013) Pupil dilation signals uncertainty and surprise in a learning gambling task. Front Behav Neurosci 7

  35. Levy DJ, Glimcher PW (2012) The root of all value: a neural common currency for choice. Curr Opin Neurobiol 22:1027–1038. https://doi.org/10.1016/j.conb.2012.06.001

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  36. Lui LL, Pasternak T (2011) Representation of comparison signals in cortical area MT during a delayed direction discrimination task. J Neurophysiol 106:1260–1273. https://doi.org/10.1152/jn.00016.2011

    Article  PubMed  PubMed Central  Google Scholar 

  37. Machens CK, Romo R, Brody CD (2005) Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science 307:1121–1124. https://doi.org/10.1126/science.1104171

    Article  PubMed  CAS  Google Scholar 

  38. McNamee D, Liljeholm M, Zika O, O’Doherty JP (2015) Characterizing the associative content of brain structures involved in habitual and goal-directed actions in humans: a multivariate FMRI study. J Neurosci 35:3764–3771. https://doi.org/10.1523/JNEUROSCI.4677-14.2015

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  39. Miller EK, Desimone R (1994) Parallel neuronal mechanisms for short-term memory. Science 263:520–522

    Article  PubMed  CAS  Google Scholar 

  40. Miller EK, Li L, Desimone R (1991) A neural mechanism for working and recognition memory in inferior temporal cortex. Science 254:1377–1379

    Article  PubMed  CAS  Google Scholar 

  41. Mirabella G, Bertini G, Samengo I, Kilavik BE, Frilli D, Della Libera C, Chelazzi L (2007) Neurons in area V4 of the macaque translate attended visual features into behaviorally relevant categories. Neuron 54:303–318. https://doi.org/10.1016/j.neuron.2007.04.007

    Article  PubMed  CAS  Google Scholar 

  42. Nassar MR, Rumsey KM, Wilson RC, Parikh K, Heasly B, Gold JI (2012) Rational regulation of learning dynamics by pupil-linked arousal systems. Nat Neurosci 15:1040–1046. https://doi.org/10.1038/nn.3130

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  43. O’Doherty JP (2011) Contributions of the ventromedial prefrontal cortex to goal-directed action selection. Ann N Y Acad Sci 1239:118–129. https://doi.org/10.1111/j.1749-6632.2011.06290.x

    Article  PubMed  Google Scholar 

  44. Pearson JM, Watson KK, Platt ML (2014) Decision making: the neuroethological turn. Neuron 82:950–965. https://doi.org/10.1016/j.neuron.2014.04.037

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  45. Preuschoff K, Marius’t Hart B, Einhäuser W (2011) Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making. Front Neurosci 5

  46. Rich EL, Wallis JD (2016) Decoding subjective decisions from orbitofrontal cortex. Nat Neurosci 19:973–980. https://doi.org/10.1038/nn.4320

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  47. Rigato S, Rieger G, Romei V (2016) Multisensory signalling enhances pupil dilation. Sci Rep 6:26188. https://doi.org/10.1038/srep26188

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  48. Rudebeck PH, Putnam PT, Daniels TE, Yang T, Mitz AR, Rhodes SE, Murray EA (2014) A role for primate subgenual cingulate cortex in sustaining autonomic arousal. Proc Natl Acad Sci USA 111:5391–5396. https://doi.org/10.1073/pnas.1317695111

    Article  PubMed  CAS  Google Scholar 

  49. Schoenbaum G, Setlow B, Saddoris MP, Gallagher M (2003) Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39:855–867

    Article  PubMed  CAS  Google Scholar 

  50. Schultz W (2008) Introduction. Neuroeconomics: the promise and the profit. Philos Trans R Soc Lond B Biol Sci 363:3767–3769. https://doi.org/10.1098/rstb.2008.0153

    Article  PubMed  PubMed Central  Google Scholar 

  51. Sleezer BJ, Castagno MD, Hayden BY (2016) Rule Encoding in Orbitofrontal Cortex and Striatum Guides Selection. J Neurosci 36:11223–11237. https://doi.org/10.1523/JNEUROSCI.1766-16.2016

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  52. Stalnaker TA, Roesch MR, Franz TM, Burke KA, Schoenbaum G (2006) Abnormal associative encoding in orbitofrontal neurons in cocaine-experienced rats during decision-making. Eur J Neurosci 24:2643–2653. https://doi.org/10.1111/j.1460-9568.2006.05128.x

    Article  PubMed  PubMed Central  Google Scholar 

  53. Strait CE, Blanchard TC, Hayden BY (2014) Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron 82:1357–1366. https://doi.org/10.1016/j.neuron.2014.04.032

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  54. Strait CE, Sleezer BJ, Hayden BY (2015) Signatures of value comparison in ventral striatum neurons. PLoS Biol 13:e1002173. https://doi.org/10.1371/journal.pbio.1002173

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  55. Strait CE, Sleezer BJ, Blanchard TC, Azab H, Castagno MD, Hayden BY (2016) Neuronal selectivity for spatial positions of offers and choices in five reward regions. J Neurophysiol 115:1098–1111. https://doi.org/10.1152/jn.00325.2015

    Article  PubMed  Google Scholar 

  56. Suzuki TW, Kunimatsu J, Tanaka M (2016) Correlation between pupil size and subjective passage of time in non-human primates. J Neurosci 36:11331–11337

    Article  PubMed  CAS  Google Scholar 

  57. Tsujimoto S, Genovesio A, Wise SP (2012) Neuronal activity during a cued strategy task: comparison of dorsolateral, orbital, and polar prefrontal cortex. J Neurosci 32:11017–11031. https://doi.org/10.1523/JNEUROSCI.1230-12.2012

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  58. van den Brink RL, Murphy PR, Nieuwenhuis S (2016) Pupil diameter tracks lapses of attention. PloS One 11:e0165274. https://doi.org/10.1371/journal.pone.0165274

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  59. Varazzani C, San-Galli A, Gilardeau S, Bouret S (2015) Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys. J Neurosci 35:7866–7877. https://doi.org/10.1523/JNEUROSCI.0454-15.2015

    Article  PubMed  CAS  Google Scholar 

  60. Wallis JD, Rich EL (2011) Challenges of interpreting frontal neurons during value-based decision-making. Front Neurosci 5:124. https://doi.org/10.3389/fnins.2011.00124

    Article  PubMed  PubMed Central  Google Scholar 

  61. Wang MZ, Hayden BY (2017) Reactivation of associative structure specific outcome responses during prospective evaluation in reward-based choices. Nat Commun 8:15821. https://doi.org/10.1038/ncomms15821

    Article  PubMed  PubMed Central  Google Scholar 

  62. Xie J, Frazier PI, Chick SE (2016) Bayesian optimization via simulation with pairwise sampling and correlated prior beliefs. Oper Res 64:542–559. https://doi.org/10.1287/opre.2016.1480

    Article  Google Scholar 

  63. Yamada H, Tymula A, Louie K, Glimcher PW (2013) Thirst-dependent risk preferences in monkeys identify a primitive form of wealth. Proc Natl Acad Sci USA 110:15788–15793. https://doi.org/10.1073/pnas.1308718110

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This work is supported by a CAREER award from NSF (BCS1253576) and a R01 from NIH (DA038615) to BYH. We thank Meghan Castagno, Marc Mancarella and Caleb Strait for assistance with data collection, and the rest of the Hayden lab for valuable discussions.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Tyler Cash-Padgett.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Statement on the Welfare of Animals

All procedures performed in this study involving animals were in accordance with the ethical standards of the University of Rochester.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (TIF 8092 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Cash-Padgett, T., Azab, H., Yoo, S.B.M. et al. Opposing pupil responses to offered and anticipated reward values. Anim Cogn 21, 671–684 (2018). https://doi.org/10.1007/s10071-018-1202-2

Download citation

Keywords

  • Monkey
  • Decision-making
  • Pupil size
  • Value