Abstract
Volitional selection of action is subject to continuous adjustment under the influence of information obtained by monitoring behavioral consequences and by exploiting behavioral context based on prior knowledge about the environment. The rostral cingulate motor area (CMAr) is thought to be responsible for adjusting behavior by monitoring its consequences. To investigate whether the CMAr also plays a role in exploitation of behavioral context in action selection, we recorded neuronal activities from the CMAr while monkeys performed a reward-based motor selection task that required them to switch from one action to the other based on the amount of reward. We examined both the behavior of monkeys and the activity of CMA neurons quantitatively by constructing a hybrid reinforcement learning model incorporating context-based and outcome-based action values into a new action value. We found that CMAr neurons encoded the context-based action value by increasing or decreasing their firing rates gradually with the number of repetitions of the same movement (i.e., behavioral context). We also found that CMAr neurons encoded the context-based and outcome-based action values in two distinct time windows at single neuron and population levels. Our findings indicate that the CMAr is involved in behavioral adjustment of action selection by exploiting the behavioral context and not merely by monitoring reward outcome.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abe H, Lee D (2011) Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 70:731–741
Amiez C, Joseph JP, Procyk E (2005) Anterior cingulate error-related activity is modulated by predicted reward. Eur J Neurosci 21:3447–3452
Behrens TEJ, Woolrich MW, Walton ME, Rushworth MFS (2007) Learning the value of information in an uncertain world. Nat Neurosci 10:1214–1221
Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8:1704–1711
Doya K, Samejima K, Katagiri K, Kawato M (2002) Multiple model-based reinforcement learning. Neural Comput 14:1347–1369
Fujii N, Mushiake H, Tanji J (2002) Distribution of eye- and arm-movement-related neuronal activity in the SEF and in the SMA and pre-SMA of monkeys. J Neurophysiol 87:2158–2166
Gläscher J, Daw N, Dayan P, O’Doherty JP (2010) States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66:585–595
Gold JI, Shadlen MN (2007) The neural basis of decision making. Annu Rev Neurosci 30:535–574
Hadland KA, Rushworth MFS, Gaffan D, Passingham RE (2003) The anterior cingulate and reward-guided selection of actions. J Neurophysiol 89:1161–1164
Hayden BY, Pearson JM, Platt ML (2009) Fictive reward signals in the anterior cingulate cortex. Science 324:948–950
Hayden BY, Pearson JM, Platt ML (2011) Neuronal basis of sequential foraging decisions in a patchy environment. Nat Neurosci 14:933–939
Isoda M, Hikosaka O (2007) Switching from automatic to controlled action by monkey medial frontal cortex. Nat Neurosci 10:240–248
Jocham G, Neumann J, Klein TA, Danielmeier C, Ullsperger M (2009) Adaptive coding of action values in the human rostral cingulate zone. J Neurosci 29:7489–7496
Kennerley SW, Walton ME, Behrens TEJ, Buckley MJ, Rushworth MFS (2006) Optimal decision making and the anterior cingulate cortex. Nat Neurosci 9:940–947
Longo MR, Iannetti GD, Mancini F, Driver J, Haggard P (2012) Linking pain and the body: neural correlates of visually induced analgesia. J Neurosci 32:2601–2607
Matelli M, Luppino G, Rizzolatti G (1991) Architecture of superior and mesial area 6 and the adjacent cingulate cortex in the macaque monkey. J Comp Neurol 311:445–462
Matsumoto M, Matsumoto K, Abe H, Tanaka K (2007) Medial prefrontal cell activity signaling prediction errors of action values. Nat Neurosci 10:647–656
Matsuzaka Y, Aizawa H, Tanji J (1992) A motor area rostral to the supplementary motor area (presupplementary motor area) in the monkey: neuronal activity during a learned motor task. J Neurophysiol 68:653–662
Mongillo G, Barak O, Tsodyks M (2008) Synaptic theory of working memory. Science 319:1543–1546
Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936–1947
Nachev P, Kennard C, Husain M (2008) Functional role of the supplementary and pre-supplementary motor areas. Nat Rev Neurosci 9:856–869
Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O (2004) Dopamine Neurons can represent context-dependent prediction error. Neuron 41:269–280
Niki H, Watanabe M (1979) Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res 171:213–224
O’Doherty JP (2012) Beyond simple reinforcement learning: the computational neurobiology of reward-learning and valuation. Eur J Neurosci 35:987–990
Picard N, Strick PL (1996) Motor areas of the medial wall: a review of their location and functional activation. Cereb Cortex 6:342–353
Picard N, Strick PL (1997) Activation on medial wall during remembered sequences of reaching movements in monkeys. J Neurophysiol 77:2197–2201
Quilodran R, Rothe M, Procyk E (2008) Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57:314–325
Ridderinkhof KR, Ullsperger M, Crone EA, Nieuwenhuis S (2004) The role of the medial frontal cortex in cognitive control. Science 306:443–447
Roitman JD, Shadlen MN (2002) Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 22:9475–9489
Romo R, Hernandez A, Zainos A (2004) Neuronal correlates of a perceptual decision in ventral premotor cortex. Neuron 41:165–173
Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27
Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599
Seo H, Lee D (2007) Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J Neurosci 27:8366–8377
Shima K, Tanji J (1998) Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282:1335–1338
Shima K, Tanji J (2000) Neuronal activity in the supplementary and presupplementary motor areas for temporal organization of multiple movements. J Neurophysiol 84:2148–2160
Shima K, Aya K, Mushiake H, Inase M, Aizawa H, Tanji J (1991) Two movement-related foci in the primate cingulate cortex observed in signal-triggered and self-paced forelimb movements. J Neurophysiol 65:188–202
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
Swick D, Turken AU (2002) Dissociation between conflict detection and error monitoring in the human anterior cingulate cortex. Proc Natl Acad Sci USA 99:16354–16359
Tachibana K, Suzuki K, Mori E, Miura N, Kawashima R, Horie K, Sato S, Tanji J, Mushiake H (2009) Neural activity in the human brain signals logical rule identification. J Neurophysiol 102:1526–1537
Tanji J (1994) The supplementary motor area in the cerebral cortex. Neurosci Res 19:251–268
Tanji J (2001) Sequential organization of multiple movements: involvement of cortical motor areas. Annu Rev Neurosci 24:631–651
Ullsperger M, von Cramon DY (2003) Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J Neurosci 23:4308–4314
Walton ME, Devlin JT, Rushworth MFS (2004) Interactions between decision making and performance monitoring within prefrontal cortex. Nat Neurosci 7:1259–1265
Wang Y, Isoda M, Matsuzaka Y, Shima K, Tanji J (2005) Prefrontal cortical cells projecting to the supplementary eye field and presupplementary motor area in the monkey. Neurosci Res 53:1–7
Acknowledgments
This work was supported by Core research for Evolutional Science and Technology (CREST) of Japan Science and Technology Agency (JST) and by Grants-in-Aid for Scientific Research, Ministry of Education, Science, and Culture, Japan (Grant No. 22300124). We thank M. Kurama and M. Takahashi for technical assistance.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Iwata, Ji., Shima, K., Tanji, J. et al. Neurons in the cingulate motor area signal context-based and outcome-based volitional selection of action. Exp Brain Res 229, 407–417 (2013). https://doi.org/10.1007/s00221-013-3442-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-013-3442-3