Psychonomic Bulletin & Review

, Volume 25, Issue 1, pp 302–321 | Cite as

Distributed representations of action sequences in anterior cingulate cortex: A recurrent neural network approach

Theoretical Review

Abstract

Anterior cingulate cortex (ACC) has been the subject of intense debate over the past 2 decades, but its specific computational function remains controversial. Here we present a simple computational model of ACC that incorporates distributed representations across a network of interconnected processing units. Based on the proposal that ACC is concerned with the execution of extended, goal-directed action sequences, we trained a recurrent neural network to predict each successive step of several sequences associated with multiple tasks. In keeping with neurophysiological observations from nonhuman animals, the network yields distributed patterns of activity across ACC neurons that track the progression of each sequence, and in keeping with human neuroimaging data, the network produces discrepancy signals when any step of the sequence deviates from the predicted step. These simulations illustrate a novel approach for investigating ACC function.

Keywords

Anterior cingulate cortex Recurrent neural network Sequence learning Surprise 

Notes

Acknowledgements

The order of authorship is arbitrary; both authors made equal contributions to the research and preparation of this article. This research was supported in part by funding from the Canada Research Chairs program and a Natural Sciences and Engineering Research Council of Canada Discovery Grant (312409–05) awarded to C.B.H.

Supplementary material

13423_2017_1280_Fig10_ESM.gif (94 kb)
ESM 1

(GIF 93 kb)

13423_2017_1280_MOESM1_ESM.tif (664 kb)
High resolution image (TIF 663 kb)
13423_2017_1280_Fig11_ESM.gif (44 kb)
ESM 2

(GIF 43 kb)

13423_2017_1280_MOESM2_ESM.tif (225 kb)
High resolution image (TIF 225 kb)

References

  1. Abbott, L. F., DePasquale, B., & Memmesheimer, R.-M. (2016). Building functional networks of spiking model neurons. Nature Neuroscience, 19, 350–355. doi: 10.1038/nn.4241
  2. Alexander, W. H., & Brown, J. W. (2010). Competition between learned reward and error outcome predictions in anterior cingulate cortex. NeuroImage, 49, 3210–3218. doi: 10.1111/j.1756-8765.2010.01085.x PubMedCrossRefGoogle Scholar
  3. Alexander, W. H., & Brown, J. W. (2011). Medial prefrontal cortex as an action-outcome predictor. Nature Neuroscience, 14, 1338–1344. doi: 10.1038/nn.2921 PubMedPubMedCentralCrossRefGoogle Scholar
  4. Alexander, W. H., Fukunaga, R., Finn, P., & Brown, J. W. (2015). Reward salience and risk aversion underlie differential ACC activity in substance dependence. NeuroImage: Clinical, 8, 59–71. doi: 10.1016/j.nicl.2015.02.025 CrossRefGoogle Scholar
  5. Amiez, C., Joseph, J.-P., & Procyk, E. (2005). Anterior cingulate error-related activity is modulated by predicted reward. European Journal of Neuroscience, 21, 3447–3452. doi: 10.1111/j.1460-9568.2005.04170.x PubMedPubMedCentralCrossRefGoogle Scholar
  6. Angel, R. W. (1976). Efference copy in the control of movement. Neurology, 26, 1164–1168. doi: 10.1212/wnl.26.12.1164 PubMedCrossRefGoogle Scholar
  7. Ardid, S., & Wang, X.-J. (2013). A tweaking principle for executive control: Neuronal circuit mechanism for rule-based task switching and conflict resolution. Journal of Neuroscience, 33, 19504–19517. doi: 10.1523/jneurosci.1356-13.2013 PubMedCrossRefGoogle Scholar
  8. Backus, D. A., Ye, S., Russo, G. S., & Crutcher, M. D. (2001). Neural activity correlated with the preparation and execution of visually guided arm movements in the cingulate motor area of the monkey. Experimental Brain Research, 140, 182–189. doi: 10.1007/s002210100807 PubMedCrossRefGoogle Scholar
  9. Badre, D., Frank, M. J., & Moore, C. I. (2015). Interactionist neuroscience. Neuron, 88, 855–860. doi: 10.1016/j.neuron.2015.10.021 PubMedCrossRefGoogle Scholar
  10. Baeg, E. H., Kim, Y. B., Huh, K., Mook-Jung, I., Kim, H. T., & Jung, M. W. (2003). Dynamics of population code for working memory in the prefrontal cortex. Neuron, 40, 177–188. doi: 10.1016/s0896-6273(03)00597-x PubMedCrossRefGoogle Scholar
  11. Balaguer-Ballester, E., Lapish, C. C., Seamans, J. K., & Durstewitz, D. (2011). Attracting dynamics of frontal cortex ensembles during memory-guided decision-making. PLoS Computational Biology, 7, e1002057. doi: 10.1371/journal.pcbi.1002057 PubMedPubMedCentralCrossRefGoogle Scholar
  12. Berns, G. S., Cohen, J. D., & Mintun, M. A. (1997). Brain regions responsive to novelty in the absence of awareness. Science, 276, 1272–1275. doi: 10.1126/science.276.5316.1272 PubMedCrossRefGoogle Scholar
  13. Blanchard, T. C., Strait, C. E., & Hayden, B. Y. (2015). Ramping ensemble activity in dorsal anterior cingulate neurons during persistent commitment to a decision. Journal of Neurophysiology, 114, 2439–2449.PubMedPubMedCentralCrossRefGoogle Scholar
  14. Botvinick, M. M. (2005). Modeling routine sequential action with recurrent neural nets. In J. J. Bryson, T. J. Prescott, & A. K. Seth (Eds.), Modeling natural action selection (pp. 180–187). Cambridge: Cambridge University Press.Google Scholar
  15. Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108, 624–652. doi: 10.1037/0033-295x.108.3.624 PubMedCrossRefGoogle Scholar
  16. Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and anterior cingulate cortex: An update. Trends in Cognitive Sciences, 8, 539–546. doi: 10.1016/j.tics.2004.10.003 PubMedCrossRefGoogle Scholar
  17. Botvinick, M., & Plaut, D. C. (2002). Representing task context: Proposals based on a connectionist model of action. Psychological Research, 66, 298–311. doi: 10.1007/s00426-002-0103-8 PubMedCrossRefGoogle Scholar
  18. Botvinick, M., & Plaut, D. C. (2004). Doing without schema hierarchies: A recurrent connectionist approach to normal and impaired routine sequential action. Psychological Review, 111, 395–429. doi: 10.1037/0033-295x.111.2.395 PubMedCrossRefGoogle Scholar
  19. Braver, T. S., Barch, D. M., Gray, J. R., Molfese, D. I., & Snyder, A. (2001). Anterior cingulate cortex and response conflict: Effects of frequency, inhibition and errors. Cerebral Cortex, 11, 825–836. doi: 10.1093/cercor/11.9.825 PubMedCrossRefGoogle Scholar
  20. Brown, J. W. (2009). Conflict effects without conflict in anterior cingulate cortex: Multiple response effects and context specific representations. NeuroImage, 47, 334–341. doi: 10.1016/j.neuroimage.2009.04.034 PubMedPubMedCentralCrossRefGoogle Scholar
  21. Brown, J. W., & Braver, T. S. (2005). Learned predictions of error likelihood in the anterior cingulate cortex. Science, 307, 1118–1121. doi: 10.1126/science.1105783 PubMedCrossRefGoogle Scholar
  22. Brown, J. W., & Braver, T. S. (2007). Risk prediction and aversion by anterior cingulate cortex. Cognitive, Affective, & Behavioral Neuroscience, 7, 266–277. doi: 10.3758/cabn.7.4.266 CrossRefGoogle Scholar
  23. Brown, J. W., & Braver, T. S. (2008). A computational model of risk, conflict, and individual difference effects in the anterior cingulate cortex. Brain Research, 1202, 99–108. doi: 10.1016/j.brainres.2007.06.080 PubMedCrossRefGoogle Scholar
  24. Bryden, D. W., Johnson, E. E., Tobia, S. C., Kashtelyan, V., & Roesch, M. R. (2011). Attention for learning signals in anterior cingulate cortex. Journal of Neuroscience, 31, 18266–18274. doi: 10.1523/jneurosci.4715-11.2011 PubMedPubMedCentralCrossRefGoogle Scholar
  25. Bush, G. (2009). Dorsal anterior midcingulate cortex: Roles in normal cognition and disruption in attention-deficit/hyperactivity disorder. In B. A. Vogt (Ed.), Cingulate neurobiology and disease (pp. 245–274). Oxford: Oxford University Press.Google Scholar
  26. Cai, X., & Padoa-Schioppa, C. (2012). Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex. Journal of Neuroscience, 32, 3791–3808. doi: 10.1523/jneurosci.3864-11.2012 PubMedPubMedCentralCrossRefGoogle Scholar
  27. Caracheo, B. F., Emberly, E., Hadizadeh, S., Hyman, J. M., & Seamans, J. K. (2013). Abrupt changes in the patterns and complexity of anterior cingulate cortex activity when food is introduced into an environment. Frontiers in Neuroscience, 7, 74. doi: 10.3389/fnins.2013.00074 PubMedPubMedCentralCrossRefGoogle Scholar
  28. Cavanagh, J. F., & Shackman, A. J. (2015). Frontal midline theta reflects anxiety and cognitive control: Meta-analytic evidence. Journal of Physiology, Paris, 109, 3–15. doi: 10.1016/j.jphysparis.2014.04.003 PubMedCrossRefGoogle Scholar
  29. Cavanagh, J. F., Zambrano-Vazquez, L., & Allen, J. J. B. (2012). Theta lingua franca: A common mid-frontal substrate for action monitoring processes. Psychophysiology, 49, 220–238. doi: 10.1111/j.1469-8986.2011.01293.x PubMedCrossRefGoogle Scholar
  30. Churchland, P. S., & Sejnowski, T. J. (1992). The computational brain. Cambridge: MIT Press.Google Scholar
  31. Cleeremans, A., & McClelland, J. L. (1991). Learning the structure of event sequences. Journal of Experimental Psychology: General, 120, 235–253. doi: 10.1037/0096-3445.120.3.235 CrossRefGoogle Scholar
  32. Cooper, R., & Glasspool, D. (2001). Learning action affordances and action schemas. In R. M. French & J. P. Sougné (Eds.), Connectionist models of learning, development and evolution (pp. 133–42). London: Springer. doi: 10.1007/978-1-4471-0281-6_14 CrossRefGoogle Scholar
  33. Cooper, R. P., Ruh, N., & Mareschal, D. (2014). The goal circuit model: A hierarchical multi-route model of the acquisition and control of routine sequential action in humans. Cognitive Science, 38, 244–274. doi: 10.1111/cogs.12067 PubMedCrossRefGoogle Scholar
  34. Cooper, R., & Shallice, T. (2000). Contention scheduling and the control of routine activities. Cognitive Neuropsychology, 17, 297–338. doi: 10.1080/026432900380427 PubMedCrossRefGoogle Scholar
  35. Cooper, R. P., & Shallice, T. (2006). Hierarchical schemas and goals in the control of sequential behavior. Psychological Review, 113, 887–916. doi: 10.1037/0033-295x.113.4.887 PubMedCrossRefGoogle Scholar
  36. Cowen, S. L., Davis, G. A., & Nitz, D. A. (2012). Anterior cingulate neurons in the rat map anticipated effort and reward to their associated action sequences. Journal of Neurophysiology, 107, 2393–2407. doi: 10.1152/jn.01012.2011 PubMedCrossRefGoogle Scholar
  37. Cowen, S. L., & McNaughton, B. L. (2007). Selective delay activity in the medial prefrontal cortex of the rat: Contribution of sensorimotor information and contingency. Journal of Neurophysiology, 98, 303–316. doi: 10.1152/jn.00150.2007 PubMedCrossRefGoogle Scholar
  38. Cunningham, J. P., & Yu, B. M. (2014). Dimensionality reduction for large-scale neural recordings. Nature Neuroscience, 17, 1500–1509. doi: 10.1038/nn.3776 PubMedPubMedCentralCrossRefGoogle Scholar
  39. Davis, K. D., Hutchinson, W. D., Lozano, A. M., Tasker, R. R., & Dostrovsky, J. O. (2000). Human anterior cingulate cortex neurons modulated by attention-demanding tasks. Journal of Neurophysiology, 83, 3575–3577.PubMedCrossRefGoogle Scholar
  40. Davis, K. D., Taylor, K. S., Hutchison, W. D., Dostrovsky, J. O., McAndrews, M. P., Richter, E. O., & Lozano, A. M. (2005). Human anterior cingulate cortex neurons encode cognitive and emotional demands. Journal of Neuroscience, 25, 8402–8406. doi: 10.1523/jneurosci.2315-05.2005 PubMedCrossRefGoogle Scholar
  41. Donkers, F. C. L., Nieuwenhuis, S., & van Boxtel, G. J. M. (2005). Mediofrontal negativities in the absence of responding. Cognitive Brain Research, 25, 777–787. doi: 10.1016/j.cogbrainres.2005.09.007 PubMedCrossRefGoogle Scholar
  42. Donoso, M., Collins, A. G. E., & Koechlin, E. (2014). Foundations of human reasoning in the prefrontal cortex. Science, 344, 1481–1486. doi: 10.1126/science.1252254 PubMedCrossRefGoogle Scholar
  43. Durstewitz, D., & Deco, G. (2008). Computational significance of transient dynamics in cortical networks. European Journal of Neuroscience, 27, 217–227. doi: 10.1111/j.1460-9568.2007.05976.x PubMedCrossRefGoogle Scholar
  44. Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000). Neurocomputational models of working memory. Nature Neuroscience, 3, 1184–1191. doi: 10.1038/81460 PubMedCrossRefGoogle Scholar
  45. Durstewitz, D., Vittoz, N. M., Floresco, S. B., & Seamans, J. K. (2010). Abrupt transitions between prefrontal neural ensemble states accompany behavioral transitions during rule learning. Neuron, 66, 438–448. doi: 10.1016/j.neuron.2010.03.029 PubMedCrossRefGoogle Scholar
  46. Ebitz, R. B., & Hayden, B. Y. (2016). Dorsal anterior cingulate: A Rorschach test for cognitive neuroscience. Nature Neuroscience, 19, 1278–1279.PubMedCrossRefGoogle Scholar
  47. Ebitz, R. B., & Platt, M. L. (2015). Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal. Neuron, 85, 628–640. doi: 10.1016/j.neuron.2014.12.053 PubMedPubMedCentralCrossRefGoogle Scholar
  48. Eimer, M., Goschke, T., Schlaghecken, F., & Stürmer, B. (1996). Explicit and implicit learning of event sequences: Evidence from event-related brain potentials. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 970–987. doi: 10.1037/0278-7393.22.4.970 PubMedGoogle Scholar
  49. Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179–211. doi: 10.1207/s15516709cog1402_1 CrossRefGoogle Scholar
  50. Erlich, J. C., & Brody, C. D. (2013). What to do and how. Nature, 503, 45–47. doi: 10.1038/503045a PubMedCrossRefGoogle Scholar
  51. Euston, D. R., Gruber, A. J., & McNaughton, B. L. (2012). The role of medial prefrontal cortex in memory and decision making. Neuron, 76, 1057–1070. doi: 10.1016/j.neuron.2012.12.002 PubMedPubMedCentralCrossRefGoogle Scholar
  52. Euston, D. R., & McNaughton, B. L. (2006). Apparent encoding of sequential context in rat medial prefrontal cortex is accounted for by behavioral variability. Journal of Neuroscience, 26, 13143–13155. doi: 10.1523/jneurosci.3803-06.2006 PubMedCrossRefGoogle Scholar
  53. Falkenstein, M., Hohnsbein, J., Hoormann, J., & Blanke, L. (1990). Effects of errors in choice reaction tasks on the ERP under focused and divided attention. In C. H. M. Brunia, A. W. K. Gaillard, & A. Kok (Eds.), Psychophysiological brain research (pp. 192–195). Tilberg: Tilburg University Press.Google Scholar
  54. Ferdinand, N. K., Mecklinger, A., Kray, J., & Gehring, W. J. (2012). The processing of unexpected positive response outcomes in the mediofrontal cortex. Journal of Neuroscience, 32, 12087–12092. doi: 10.1523/jneurosci.1410-12.2012 PubMedCrossRefGoogle Scholar
  55. Ferdinand, N. K., Mecklinger, A., & Kray, J. (2008). Error and deviance processing in implicit and explicit sequence learning. Journal of Cognitive Neuroscience, 20, 629–642. doi: 10.1162/jocn.2008.20046 PubMedCrossRefGoogle Scholar
  56. Ferdinand, N. K., & Opitz, B. (2014). Different aspects of performance feedback engage different brain areas: Disentangling valence and expectancy in feedback processing. Scientific Reports, 4, 5986. doi: 10.1038/srep05986 PubMedPubMedCentralCrossRefGoogle Scholar
  57. Folstein, J. R., & Van Petten, C. (2008). Influence of cognitive control and mismatch on the N2 component of the ERP: A review. Psychophysiology, 45, 152–170. doi: 10.1111/j.1469-8986.2007.00602.x PubMedCrossRefGoogle Scholar
  58. Forster, S. E., & Brown, J. W. (2011). Medial prefrontal cortex predicts and evaluates the timing of action outcomes. NeuroImage, 55, 253–265. doi: 10.1016/j.neuroimage.2010.11.035 PubMedCrossRefGoogle Scholar
  59. Fu, Q., Bin, G., Dienes, Z., Fu, X., & Gao, X. (2013). Learning without consciously knowing: Evidence from event-related potentials in sequence learning. Consciousness and Cognition, 22, 22–34. doi: 10.1016/j.concog.2012.10.008 PubMedCrossRefGoogle Scholar
  60. Fujisawa, S., Amarasingham, A., Harrison, M. T., & Buzsáki, G. (2008). Behavior-dependent short-term assembly dynamics in the medial prefrontal cortex. Nature Neuroscience, 11, 823–833. doi: 10.1038/nn.2134 PubMedPubMedCentralCrossRefGoogle Scholar
  61. Fusi, S., Miller, E. K., & Rigotti, M. (2016). Why neurons mix: High dimensionality for higher cognition. Current Opinion in Neurobiology, 37, 66–74. doi: 10.1016/j.conb.2016.01.010 PubMedCrossRefGoogle Scholar
  62. Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., & Donchin, E. (1993). A neural system for error detection and compensation. Psychological Science, 4, 385–390. doi: 10.1111/j.1467-9280.1993.tb00586.x CrossRefGoogle Scholar
  63. Gehring, W. J., Gratton, G., Coles, M. G. H., & Donchin, E. (1992). Probability effects on stimulus evaluation and response processes. Journal of Experimental Psychology: Human Perception and Performance, 18, 198–216. doi: 10.1037/0096-1523.18.1.198 PubMedGoogle Scholar
  64. Gehring, W. J., Liu, Y., Orr, J. M., & Carp, J. (2012). The error-related negativity (ERN/Ne). In S. J. Luck & E. Kappenm (Eds.), The Oxford handbook of event-related potential components (pp. 231–291). Oxford: Oxford University Press. doi: 10.1093/oxfordhb/9780195374148.013.0120 Google Scholar
  65. HajiHosseini, A., & Holroyd, C. B. (2013). Frontal midline theta and N200 amplitude reflect complementary information about expectancy and outcome evaluation. Psychophysiology, 50, 550–562. doi: 10.1111/psyp.12040 PubMedCrossRefGoogle Scholar
  66. Hammer, A., Rautzenberg, A., Heldmann, M., & Münte, T. F. (2011). Challenging the error-likelihood model with a recognition paradigm: An electrophysiological study. Biological Psychology, 88, 180–187. doi: 10.1016/j.biopsycho.2011.07.010 PubMedCrossRefGoogle Scholar
  67. Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L. (2011). Surprise signals in anterior cingulate cortex: Neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. Journal of Neuroscience, 31, 4178–4187. doi: 10.1523/jneurosci.4652-10.2011 PubMedPubMedCentralCrossRefGoogle Scholar
  68. Hayden, B. Y., Pearson, J. M., & Platt, M. L. (2011). Neuronal basis of sequential foraging decisions in a patchy environment. Nature Neuroscience, 14, 933–939. doi: 10.1038/nn.2856 PubMedPubMedCentralCrossRefGoogle Scholar
  69. Hayden, B. Y., & Platt, M. L. (2010). Neurons in anterior cingulate cortex multiplex information about reward and action. Journal of Neuroscience, 30, 3339–3346. doi: 10.1523/jneurosci.4874-09.2010 PubMedPubMedCentralCrossRefGoogle Scholar
  70. Hengst, B. (2012). Hierarchical approaches. In M. Wiering & M. Van Otterlo (Eds.), Reinforcement learning (pp. 293–323). Berlin: Springer. doi: 10.1007/978-3-642-27645-3 CrossRefGoogle Scholar
  71. Hillman, K. L., & Bilkey, D. K. (2010). Neurons in the rat anterior cingulate cortex dynamically encode cost-benefit in a spatial decision-making task. Journal of Neuroscience, 30, 7705–7713. doi: 10.1523/jneurosci.1273-10.2010 PubMedCrossRefGoogle Scholar
  72. Hillman, K. L., & Bilkey, D. K. (2012). Neural encoding of competitive effort in the anterior cingulate cortex. Nature Neuroscience, 15, 1290–1297. doi: 10.1038/nn.3187 PubMedCrossRefGoogle Scholar
  73. Hoerzer, G. M., Legenstein, R., & Maass, W. (2014). Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. Cerebral Cortex, 24, 677–690. doi: 10.1093/cercor/bhs348 PubMedCrossRefGoogle Scholar
  74. Holroyd, C. B. (2004). A note on the oddball N200 and the feedback ERN. In M. Ullsperger & M. Falkenstein (Eds.), Errors, conflicts, and the brain: Current opinions on performance monitoring (pp. 211–218). Leipzig: MPI of Cognitive Neuroscience.Google Scholar
  75. Holroyd, C. B. (2016). The waste disposal problem of effortful control. In T. Braver (Ed.), Motivation and cognitive control (pp. 235–260). New York: Psychology Press.Google Scholar
  76. Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679–709. doi: 10.1037//0033-295x.109.4.679 PubMedCrossRefGoogle Scholar
  77. Holroyd, C. B., & Coles, M. G. H. (2008). Dorsal anterior cingulate cortex integrates reinforcement history to guide voluntary behavior. Cortex, 44, 548–559. doi: 10.1016/j.cortex.2007.08.013 PubMedCrossRefGoogle Scholar
  78. Holroyd, C. B., & McClure, S. M. (2015). Hierarchical control over effortful behavior by rodent medial frontal cortex: A computational model. Psychological Review, 122, 54–83. doi: 10.1037/a0038339 PubMedCrossRefGoogle Scholar
  79. Holroyd, C. B., Pakzad-Vaezi, K. L., & Krigolson, O. E. (2008). The feedback correct-related positivity: Sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology, 45, 688–697. doi: 10.1111/j.1469-8986.2008.00668.x PubMedCrossRefGoogle Scholar
  80. Holroyd, C. B., & Yeung, N. (2011). An integrative theory of anterior cingulate cortex function: Option selection in hierarchical reinforcement learning. In R. B. Mars, J. Sallet, M. F. S. Rushworth, & N. Yeung (Eds.), Neural basis of motivational and cognitive control (pp. 333–349). Cambridge: MIT Press. doi: 10.7551/mitpress/9780262016438.003.0018 Google Scholar
  81. Holroyd, C. B., & Yeung, N. (2012). Motivation of extended behaviors by anterior cingulate cortex. Trends in Cognitive Sciences, 16, 122–128. doi: 10.1016/j.tics.2011.12.008 PubMedCrossRefGoogle Scholar
  82. Holroyd, C. B., Yeung, N., Coles, M. G. H., & Cohen, J. D. (2005). A mechanism for error detection in speeded response time tasks. Journal of Experimental Psychology: General, 134, 163–191. doi: 10.1037/0096-3445.134.2.163 CrossRefGoogle Scholar
  83. Hornik, K. (1991). Approximation capabilities of multilayer feedforward networks. Neural Networks, 4, 251–257. doi: 10.1016/0893-6080(91)90009-t CrossRefGoogle Scholar
  84. Hoshi, E., Sawamura, H., & Tanji, J. (2005). Neurons in the rostral cingulate motor area monitor multiple phases of visuomotor behavior with modest parametric selectivity. Journal of Neurophysiology, 94, 640–656. doi: 10.1152/jn.01201.2004 PubMedCrossRefGoogle Scholar
  85. Hyman, J. M., Ma, L., Balaguer-Ballester, E., Durstewitz, D., & Seamans, J. K. (2012). Contextual encoding by ensembles of medial prefrontal cortex neurons. Proceedings of the National Academy of Sciences, 109, 5086–5091. doi: 10.1073/pnas.1114415109 CrossRefGoogle Scholar
  86. Hyman, J. M., Whitman, J., Emberly, E., Woodward, T. S., & Seamans, J. K. (2013). Action and outcome activity state patterns in the anterior cingulate cortex. Cerebral Cortex, 23, 1257–1268. doi: 10.1093/cercor/bhs104 PubMedCrossRefGoogle Scholar
  87. Jacobs, J., Weidemann, C. T., Miller, J. F., Solway, A., Burke, J. F., Wei, X.-X., & Kahana, M. J. (2013). Direct recordings of grid-like neuronal activity in human spatial navigation. Nature Neuroscience, 16, 1188–1190. doi: 10.1038/nn.3466 PubMedPubMedCentralCrossRefGoogle Scholar
  88. Jessup, R. K., Busemeyer, J. R., & Brown, J. W. (2010). Error effects in anterior cingulate cortex reverse when error likelihood is high. Journal of Neuroscience, 30, 3467–3472. doi: 10.1523/jneurosci.4130-09.2010 PubMedPubMedCentralCrossRefGoogle Scholar
  89. Jia, S., Li, H., Luo, Y., Chen, A., Wang, B., & Zhou, X. (2007). Detecting perceptual conflict by the feedback-related negativity in brain potentials. NeuroReport, 18, 1385–1388. doi: 10.1097/wnr.0b013e3282c48a90 PubMedCrossRefGoogle Scholar
  90. Johnston, K., Levin, H. M., Koval, M. J., & Everling, S. (2007). Top-down control-signal dynamics in anterior cingulate and prefrontal cortex neurons following task switching. Neuron, 53, 453–462. doi: 10.1016/j.neuron.2006.12.023 PubMedCrossRefGoogle Scholar
  91. Jongsma, M. L. A., van Rijn, C. M., Gerrits, N. J. H. M., Eichele, T., Steenbergen, B., Maes, J. H. R., & Quiroga, R. Q. (2013). The learning-oddball paradigm: Data of 24 separate individuals illustrate its potential usefulness as a new clinical tool. Clinical Neurophysiology, 124, 514–521. doi: 10.1016/j.clinph.2012.09.009 PubMedCrossRefGoogle Scholar
  92. Kaping, D., Vinck, M., Hutchison, R. M., Everling, S., & Womelsdorf, T. (2011). Specific contributions of ventromedial, anterior cingulate, and lateral prefrontal cortex for attentional selection and stimulus valuation. PLoS Biology, 9, e1001224. doi: 10.1371/journal.pbio.1001224 PubMedPubMedCentralCrossRefGoogle Scholar
  93. Kennerley, S. W., Behrens, T. E. J., & Wallis, J. D. (2011). Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nature Neuroscience, 14, 1581–1589. doi: 10.1038/nn.2961 PubMedPubMedCentralCrossRefGoogle Scholar
  94. Kennerley, S. W., Dahmubed, A. F., Lara, A. H., & Wallis, J. D. (2009). Neurons in the frontal lobe encode the value of multiple decision variables. Journal of Cognitive Neuroscience, 21, 1162–1178. doi: 10.1162/jocn.2009.21100 PubMedPubMedCentralCrossRefGoogle Scholar
  95. Kennerley, S. W., & Wallis, J. D. (2009). Evaluating choices by single neurons in the frontal lobe: Outcome value encoded across multiple decision variables. European Journal of Neuroscience, 29, 2061–2073. doi: 10.1111/j.1460-9568.2009.06743.x PubMedPubMedCentralCrossRefGoogle Scholar
  96. Khamassi, M., Lallee, S., Enel, P., Procyk, E., & Dominey, P. F. (2011). Robot cognitive control with a neurophysiologically inspired reinforcement learning model. Frontiers in Neurorobotics, 6(1), 1–14.Google Scholar
  97. Khamassi, M., Quilodran, R., Enel, P., Dominey, P. F., & Procyk, E. (2015). Behavioral regulation and the modulation of information coding in the lateral prefrontal and cingulate cortex. Cerebral Cortex, 25, 3197–3218. doi: 10.1093/cercor/bhu114 PubMedCrossRefGoogle Scholar
  98. Klavir, O., Genud-Gabai, R., & Paz, R. (2013). Functional connectivity between amygdala and cingulate cortex for adaptive aversive learning. Neuron, 80, 1290–1300. doi: 10.1016/j.neuron.2013.09.035 PubMedCrossRefGoogle Scholar
  99. Koechlin, E., Danek, A., Burnod, Y., & Grafman, J. (2002). Medial prefrontal and subcortical mechanisms underlying the acquisition of motor and cognitive action sequences in humans. Neuron, 35, 371–381. doi: 10.1016/s0896-6273(02)00742-0 PubMedCrossRefGoogle Scholar
  100. Kopp, B., & Wolff, M. (2000). Brain mechanisms of selective learning: Event-related potentials provide evidence for error-driven learning in humans. Biological Psychology, 51, 223–246. doi: 10.1016/s0301-0511(99)00039-3 PubMedCrossRefGoogle Scholar
  101. Koyama, T., Tanaka, Y. Z., & Mikami, A. (1998). Nociceptive neurons in the macaque anterior cingulate activate during anticipation of pain. NeuroReport, 9, 2663–2667. doi: 10.1097/00001756-199808030-00044 PubMedCrossRefGoogle Scholar
  102. Kriegeskorte, N., & Kievit, R. A. (2013). Representational geometry: Integrating cognition, computation and the brain. Trends in Cognitive Sciences, 17, 401–412. doi: 10.1016/j.tics.2013.06.007 PubMedPubMedCentralCrossRefGoogle Scholar
  103. Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity analysis—Connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, Article 4, 1–28. doi: 10.3389/neuro.06.004.2008 Google Scholar
  104. Kuwabara, M., Mansouri, F. A., Buckley, M. J., & Tanaka, K. (2014). Cognitive control functions of anterior cingulate cortex in macaque monkeys performing a Wisconsin card sorting test analog. Journal of Neuroscience, 34, 7531–7547. doi: 10.1523/jneurosci.3405-13.2014 PubMedPubMedCentralCrossRefGoogle Scholar
  105. Lang, S., & Kotchoubey, B. (2000). Learning effects on event-related brain potentials. NeuroReport, 11, 3327–3331. doi: 10.1097/00001756-200010200-00013 PubMedCrossRefGoogle Scholar
  106. Lapish, C. C., Durstewitz, D., Chandler, L. J., & Seamans, J. K. (2008). Successful choice behavior is associated with distinct and coherent network states in anterior cingulate cortex. Proceedings of the National Academy of Sciences, 105, 11963–11968. doi: 10.1073/pnas.0804045105 CrossRefGoogle Scholar
  107. Luk, C.-H., & Wallis, J. D. (2009). Dynamic encoding of responses and outcomes by neurons in medial prefrontal cortex. Journal of Neuroscience, 29, 7526–7539. doi: 10.1523/jneurosci.0386-09.2009 PubMedPubMedCentralCrossRefGoogle Scholar
  108. Ma, L., Hyman, J. M., Lindsay, A. J., Phillips, A. G., & Seamans, J. K. (2014). Differences in the emergent coding properties of cortical and striatal ensembles. Nature Neuroscience, 17, 1100–1106. doi: 10.1038/nn.3753 PubMedPubMedCentralCrossRefGoogle Scholar
  109. Ma, L., Hyman, J. M., Phillips, A. G., & Seamans, J. K. (2014). Tracking progress toward a goal in corticostriatal ensembles. Journal of Neuroscience, 34, 2244–2253. doi: 10.1523/jneurosci.3834-13.2014 PubMedCrossRefGoogle Scholar
  110. Mante, V., Sussillo, D., Shenoy, K. V., & Newsome, W. T. (2013). Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature, 503, 78–84. doi: 10.1038/nature12742 PubMedPubMedCentralCrossRefGoogle Scholar
  111. Martens, J. (2010). Deep learning via Hessian-free optimization. Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel.Google Scholar
  112. Martens, J., & Sutskever, I. (2011). Learning recurrent neural networks with hessian-free optimization. Proceedings of the 28th International Conference on Machine Learning (ICML-11), Bellevue, WA.Google Scholar
  113. Mathalon, D. H., Whitfield, S. L., & Ford, J. M. (2003). Anatomy of an error: ERP and fMRI. Biological Psychology, 64, 119–141. doi: 10.1016/s0301-0511(03)00105-4 PubMedCrossRefGoogle Scholar
  114. Matsumoto, M., Matsumoto, K., Abe, H., & Tanaka, K. (2007). Medial prefrontal cell activity signaling prediction errors of action values. Nature Neuroscience, 10, 647–656. doi: 10.1038/nn1890 PubMedCrossRefGoogle Scholar
  115. Metereau, E., & Dreher, J.-C. (2013). Cerebral correlates of salient prediction error for different rewards and punishments. Cerebral Cortex, 23, 477–487. doi: 10.1093/cercor/bhs037 PubMedCrossRefGoogle Scholar
  116. Miyawaki, K., Sato, A., Yasuda, A., Kumano, H., & Kuboki, T. (2005). Explicit knowledge and intention to learn in sequence learning: An event-related potential study. NeuroReport, 16, 705–708. doi: 10.1097/00001756-200505120-00010 PubMedCrossRefGoogle Scholar
  117. Moody, S. L., Wise, S. P., di Pellegrino, G., & Zipser, D. (1998). A model that accounts for activity in primate frontal cortex during a delayed matching-to-sample task. Journal of Neuroscience, 18, 399–410.PubMedGoogle Scholar
  118. Mulder, A. B., Nordquist, R. E., Örgüt, O., & Pennartz, C. M. A. (2003). Learning-related changes in response patterns of prefrontal neurons during instrumental conditioning. Behavioural Brain Research, 146, 77–88. doi: 10.1016/j.bbr.2003.09.016 PubMedCrossRefGoogle Scholar
  119. Nakahara, H., & Doya, K. (1998). Near-saddle-node bifurcation behavior as dynamics in working memory for goal-directed behavior. Neural Computation, 10, 113–132. doi: 10.1162/089976698300017917 PubMedCrossRefGoogle Scholar
  120. Namikawa, J., Nishimoto, R., & Tani, J. (2011). A neurodynamic account of spontaneous behaviour. PLoS Computational Biology, 7, e1002221. doi: 10.1371/journal.pcbi.1002221 PubMedPubMedCentralCrossRefGoogle Scholar
  121. Nee, D. E., Kastner, S., & Brown, J. W. (2011). Functional heterogeneity of conflict, error, task-switching, and unexpectedness effects within medial prefrontal cortex. NeuroImage, 54, 528–540. doi: 10.1016/j.neuroimage.2010.08.027 PubMedCrossRefGoogle Scholar
  122. Nieuwenhuis, S., Schweizer, T. S., Mars, R. B., Botvinick, M. M., & Hajcak, G. (2007). Error-likelihood prediction in the medial frontal cortex: A critical evaluation. Cerebral Cortex, 17, 1570–1581. doi: 10.1093/cercor/bhl068 PubMedCrossRefGoogle Scholar
  123. Niki, H., & Watanabe, M. (1976). Cingulate unit activity and delayed response. Brain Research, 110, 381–386. doi: 10.1016/0006-8993(76)90412-1 PubMedCrossRefGoogle Scholar
  124. Nishijo, H., Yamamoto, Y., Ono, T., Uwano, T., Yamashita, J., & Yamashima, T. (1997). Single neuron responses in the monkey anterior cingulate cortex during visual discrimination. Neuroscience Letters, 227, 79–82. doi: 10.1016/s0304-3940(97)00310-8 PubMedCrossRefGoogle Scholar
  125. Nishimoto, R., & Tani, J. (2004). Learning to generate combinatorial action sequences utilizing the initial sensitivity of deterministic dynamical systems. Neural Networks, 17, 925–933. doi: 10.1016/j.neunet.2004.02.003 PubMedCrossRefGoogle Scholar
  126. O’Reilly, J. X., Schüffelgen, U., Cuell, S. F., Behrens, T. E. J., Mars, R. B., & Rushworth, M. F. S. (2013). Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proceedings of the National Academy of Sciences, 110, E3660–E3669. doi: 10.1073/pnas.1305373110 CrossRefGoogle Scholar
  127. Oliveira, F. T. P., McDonald, J. J., & Goodman, D. (2007). Performance monitoring in the anterior cingulate is not all error related: Expectancy deviation and the representation of action-outcome associations. Journal of Cognitive Neuroscience, 19, 1994–2004. doi: 10.1162/jocn.2007.19.12.1994 PubMedCrossRefGoogle Scholar
  128. Procyk, E., & Joseph, J. P. (2001). Characterization of serial order encoding in the monkey anterior cingulate sulcus. European Journal of Neuroscience, 14, 1041–1046. doi: 10.1046/j.0953-816x.2001.01738.x PubMedPubMedCentralCrossRefGoogle Scholar
  129. Procyk, E., Tanaka, Y. L., & Joseph, J. P. (2000). Anterior cingulate activity during routine and non-routine sequential behaviors in macaques. Nature Neuroscience, 3, 502–508. doi: 10.1038/74880 PubMedCrossRefGoogle Scholar
  130. Quilodran, R., Rothé, M., & Procyk, E. (2008). Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron, 57, 314–325. doi: 10.1016/j.neuron.2007.11.031 PubMedCrossRefGoogle Scholar
  131. Rao, R. P. N., & Sejnowski, T. J. (2000). Predictive learning of temporal sequences in recurrent neocortical circuits. In S. A. Solla, T. K. Leen, & K.-R. Müller (Eds.), Advances in neural information processing systems 12 (pp. 164–70). Cambridge: MIT Press.Google Scholar
  132. Remondes, M., & Wilson, M. A. (2013). Cingulate-hippocampus coherence and trajectory coding in a sequential choice task. Neuron, 80, 1277–1289. doi: 10.1016/j.neuron.2013.08.037 PubMedCrossRefGoogle Scholar
  133. Ridderinkhof, K. R., van den Wildenberg, W. P. M., Segalowtiz, S. J., & Carter, C. S. (2004). Neurocognitive mechanisms of cognitive control: The role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning. Brain and Cognition, 56, 129–140. doi: 10.1016/j.bandc.2004.09.016 PubMedCrossRefGoogle Scholar
  134. Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013). The importance of mixed selectivity in complex cognitive tasks. Nature, 497, 585–590. doi: 10.1038/nature12160 PubMedPubMedCentralCrossRefGoogle Scholar
  135. Rolls, E. G. (2009). The anterior and midcingulate cortices and reward. In B. A. Vogt (Ed.), Cingulate neurobiology and disease (pp. 191–218). Oxford: Oxford University Press.Google Scholar
  136. Ruh, N., Cooper, R. P., & Mareschal, D. (2005). Routine action: Combining familiarity and goal orientedness. In J. J. Bryson, T. J. Prescott, & A. K. Seth (Eds.), Modeling natural action selection: Proceedings of an international workshop (pp. 174–179). Edinburgh: AISB Press.Google Scholar
  137. Rumelhart, D. E., & McClelland, J. L. (1986). Parallel distributed processing: Explorations in the microstructure of cognition (vol. 1). Cambridge: MIT Press.Google Scholar
  138. Rushworth, M. F. S., & Behrens, T. E. J. (2008). Choice, uncertainty and value in prefrontal and cingulate cortex. Nature Neuroscience, 11, 389–397. doi: 10.1038/nn2066 PubMedCrossRefGoogle Scholar
  139. Rushworth, M. F. S., Kolling, N., Sallet, J., & Mars, R. B. (2012). Valuation and decision-making in frontal cortex: One or many serial or parallel systems? Current Opinion in Neurobiology, 22, 946–955. doi: 10.1016/j.conb.2012.04.011 PubMedCrossRefGoogle Scholar
  140. Rüsseler, J., Hennighausen, E., Münte, T. F., & Rösler, F. (2003). Differences in incidental and intentional learning of sensorimotor sequences as revealed by event-related brain potentials. Cognitive Brain Research, 15, 116–126. doi: 10.1016/s0926-6410(02)00145-3 PubMedCrossRefGoogle Scholar
  141. Rüsseler, J., Kuhlicke, D., & Münte, T. F. (2003). Human error monitoring during implicit and explicit learning of a sensorimotor sequence. Neuroscience Research, 47, 233–240. doi: 10.1016/s0168-0102(03)00212-8 PubMedCrossRefGoogle Scholar
  142. Rüsseler, J., & Rösler, F. (2000). Implicit and explicit learning of event sequences: Evidence for distinct coding of perceptual and motor representations. Acta Psychologica, 104, 45–67. doi: 10.1016/s0001-6918(99)00053-0 PubMedCrossRefGoogle Scholar
  143. Russo, G. S., Backus, D. A., Ye, S., & Crutcher, M. D. (2002). Neural activity in monkey dorsal and ventral cingulate motor areas: Comparison with the supplementary motor area. Journal of Neurophysiology, 88, 2612–2629. doi: 10.1152/jn.00306.2002 PubMedCrossRefGoogle Scholar
  144. Sallet, J., Quilodran, R., Rothe, M., Vezoli, J., Joseph, J.-P., & Procyk, E. (2007). Expectations, gains, and losses in the anterior cingulate cortex. Cognitive, Affective, & Behavioral Neuroscience, 7, 327–336. doi: 10.3758/cabn.7.4.327 CrossRefGoogle Scholar
  145. Sams, M., Alho, K., & Näätänen, R. (1983). Sequential effects on the ERP in discriminating two stimuli. Biological Psychology, 17, 41–58. doi: 10.1016/0301-0511(83)90065-0 PubMedCrossRefGoogle Scholar
  146. Schlaghecken, F., Stürmer, B., & Eimer, M. (2000). Chunking processes in the learning of event sequences: Electrophysiological indicators. Memory & Cognition, 28, 821–831. doi: 10.3758/bf03198417 CrossRefGoogle Scholar
  147. Schuck, N. W., Gaschler, R., Wenke, D., Heinzle, J., Frensch, P. A., Haynes, J.-D., & Reverberi, C. (2015). Medial prefrontal cortex predicts internally driven strategy shifts. Neuron, 86, 331–340. doi: 10.1016/j.neuron.2015.03.015 PubMedPubMedCentralCrossRefGoogle Scholar
  148. Seo, H., & Lee, D. (2008). Cortical mechanisms for reinforcement learning in competitive games. Philosophical Transactions of the Royal Society B, 363, 3845–3857. doi: 10.1098/rstb.2008.0158 CrossRefGoogle Scholar
  149. Servan-Schreiber, D., Cleeremans, A., & McClelland, J. L. (1991). Graded state machines: The representation of temporal contingencies in simple recurrent networks. Machine Learning, 7, 57–89. doi: 10.1007/978-1-4615-4008-3_4 Google Scholar
  150. Shackman, A. J., Salomons, T. V., Slagter, H. A., Fox, A. S., Winter, J. J., & Davidson, R. J. (2011). The integration of negative affect, pain and cognitive control in the cingulate cortex. Nature Reviews Neuroscience, 12, 154–167. doi: 10.1038/nrn2994 PubMedPubMedCentralCrossRefGoogle Scholar
  151. Shadmehr, R., & Wise, S. P. (2005). The computational neurobiology of reaching and pointing. Cambridge: MIT Press.Google Scholar
  152. Shen, C., Ardid, S., Kaping, D., Westendorff, S., Everling, S., & Womelsdorf, T. (2015). Anterior cingulate cortex cells identify process-specific errors of attentional control prior to transient prefrontal-cingulate inhibition. Cerebral Cortex, 25, 2213–2228. doi: 10.1093/cercor/bhu028 PubMedCrossRefGoogle Scholar
  153. Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013). The expected value of control: An integrative theory of anterior cingulate cortex function. Neuron, 79, 217–240. doi: 10.1016/j.neuron.2013.07.007 PubMedPubMedCentralCrossRefGoogle Scholar
  154. Shidara, M., Mizuhiki, T., & Richmond, B. J. (2005). Neuronal firing in anterior cingulate neurons changes modes across trials in single states of multitrial reward schedules. Experimental Brain Research, 163, 242–245. doi: 10.1007/s00221-005-2232-y PubMedCrossRefGoogle Scholar
  155. Shidara, M., & Richmond, B. J. (2002). Anterior cingulate: Single neuronal signals related to degree of reward expectancy. Science, 296, 1709–1711. doi: 10.1126/science.1069504 PubMedCrossRefGoogle Scholar
  156. Shima, K., Aya, K., Mushiake, H., Inase, M., Aizawa, H., & Tanji, J. (1991). Two movement-related foci in the primate cingulate cortex observed in signal-triggered and self-paced forelimb movements. Journal of Neurophysiology, 65, 188–202.PubMedCrossRefGoogle Scholar
  157. Shima, K., & Tanji, J. (1998). Role for cingulate motor area cells in voluntary movement selection based on reward. Science, 282, 1335–1338. doi: 10.1126/science.282.5392.1335 PubMedCrossRefGoogle Scholar
  158. Silvetti, M., Seurinck, R., & Verguts, T. (2011). Value and prediction error in medial frontal cortex: Integrating the single-unit and systems levels of analysis. Frontiers in Human Neuroscience, 5, 75. doi: 10.3389/fnhum.2011.00075 PubMedPubMedCentralCrossRefGoogle Scholar
  159. Song, H. F., Yang, G. R., & Wang, X.-J. (2016). Training excitatory-inhibitory recurrent neural networks for cognitive tasks: A simple and flexible framework. PLOS Computational Biology, 1–30. doi: 10.1371/journal.pcbi.1004792
  160. Starzyk, J. A., & He, H. (2007). Anticipation-based temporal sequences learning in hierarchical structure. IEEE Transactions on Neural Networks, 18, 344–358. doi: 10.1109/tnn.2006.884681 PubMedCrossRefGoogle Scholar
  161. Stokes, M. G., Kusunoki, M., Sigala, N., Nili, H., Gaffan, D., & Duncan, J. (2013). Dynamic coding for cognitive control in prefrontal cortex. Neuron, 78, 364–375. doi: 10.1016/j.neuron.2013.01.039 PubMedPubMedCentralCrossRefGoogle Scholar
  162. Sussillo, D. (2014). Neural circuits as computational dynamical systems. Current Opinion in Neurobiology, 25, 156–163. doi: 10.1016/j.conb.2014.01.008 PubMedCrossRefGoogle Scholar
  163. Sussillo, D., & Abbott, L. F. (2009). Generating coherent patterns of activity from chaotic neural networks. Neuron, 63, 544–557. doi: 10.1016/j.neuron.2009.07.018 PubMedPubMedCentralCrossRefGoogle Scholar
  164. Sussillo, D., & Barak, O. (2013). Opening the black box: Low-dimensional dynamics in high-dimensional recurrent neural networks. Neural Computation, 25, 626–649. doi: 10.1162/neco_a_00409 PubMedCrossRefGoogle Scholar
  165. Takenouchi, K., Nishijo, H., Uwano, T., Tamura, R., Takigawa, M., & Ono, T. (1999). Emotional and behavioral correlates of the anterior cingulate cortex during associative learning in rats. Neuroscience, 93, 1271–1287. doi: 10.1016/s0306-4522(99)00216-x PubMedCrossRefGoogle Scholar
  166. Tanji, J., Shima, K., & Matsuzaka, Y. (2002). Reward-based planning of motor selection in the rostral cingulate motor area. In S. C. Gandevia, U. Proske, & D. G. Stuart (Eds.), Sensorimotor control of movement and posture (pp. 417–423). New York: Springer. doi: 10.1007/978-1-4615-0713-0_47 CrossRefGoogle Scholar
  167. Tervo, D. G. R., Proskurin, M., Manakov, M., Kabra, M., Vollmer, A., Branson, K., & Karpova, A. Y. (2014). Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. Cell, 159, 21–32. doi: 10.1016/j.cell.2014.08.037 PubMedCrossRefGoogle Scholar
  168. Totah, N. K. B., Kim, Y. B., Homayoun, H., & Moghaddam, B. (2009). Anterior cingulate neurons represent errors and preparatory attention within the same behavioral sequence. Journal of Neuroscience, 29, 6418–6426. doi: 10.1523/jneurosci.1142-09.2009 PubMedPubMedCentralCrossRefGoogle Scholar
  169. Ursu, S., Clark, K. A., Aizenstein, H. J., Stenger, V. A., & Carter, C. S. (2009). Conflict-related activity in the caudal anterior cingulate cortex in the absence of awareness. Biological Psychology, 80, 279–286. doi: 10.1016/j.biopsycho.2008.10.008 PubMedCrossRefGoogle Scholar
  170. Verguts, T. (2017). Binding by random bursts: A computational model of cognitive control. Journal of Cognitive Neuroscience. doi: 10.1162/jocn_a_01117 PubMedGoogle Scholar
  171. Verguts, T., Vassena, E., & Silvetti, M. (2015). Adaptive effort investment in cognitive and physical tasks: A neurocomputational model. Frontiers in Behavioral Neuroscience, 9, 57. doi: 10.3389/fnbeh.2015.00057 PubMedPubMedCentralCrossRefGoogle Scholar
  172. Walton, M. E., & Mars, R. B. (2007). Probing human and monkey anterior cingulate cortex in variable environments. Cognitive, Affective, & Behavioral Neuroscience, 7, 413–422. doi: 10.3758/cabn.7.4.413 CrossRefGoogle Scholar
  173. Wang, X.-J. (2008). Decision making in recurrent neuronal circuits. Neuron, 60, 215–234. doi: 10.1016/j.neuron.2008.09.034 PubMedPubMedCentralCrossRefGoogle Scholar
  174. Wang, Y., Cui, L., Wang, H., Tian, S., & Zhang, X. (2004). The sequential processing of visual feature conjunction mismatches in the human brain. Psychophysiology, 41, 21–29. doi: 10.1111/j.1469-8986.2003.00134.x PubMedCrossRefGoogle Scholar
  175. Wang, C., Ulbert, I., Schomer, D. L., Marinkovic, K., & Halgren, E. (2005). Responses of human anterior cingulate cortex microdomains to error detection, conflict monitoring, stimulus–response mapping, familiarity, and orienting. Journal of Neuroscience, 25, 604–613. doi: 10.1523/jneurosci.4151-04.2005 PubMedCrossRefGoogle Scholar
  176. Warren, C. M., & Holroyd, C. B. (2012). The impact of deliberative strategy dissociates ERP components related to speeded responding vs. reinforcement learning. Frontiers in Decision Neuroscience, 6, 43. doi: 10.3389/fnins.2012.00043 Google Scholar
  177. Weible, A. P., Rowland, D. C., Monaghan, C. K., Wolfgang, N. T., & Kentros, C. G. (2012). Neural correlates of long-term object memory in the mouse anterior cingulate cortex. Journal of Neuroscience, 32, 5598–5608. doi: 10.1523/jneurosci.5265-11.2012 PubMedCrossRefGoogle Scholar
  178. Wessel, J. R., Danielmeier, C., Morton, J. B., & Ullsperger, M. (2012). Surprise and error: Common neuronal architecture for the processing of errors and novelty. Journal of Neuroscience, 32, 7528–7537. doi: 10.1523/jneurosci.6352-11.2012 PubMedCrossRefGoogle Scholar
  179. Williams, R. J., & Zipser, D. (1995). Gradient-based learning algorithms for recurrent networks and their computational complexity. In Y. Chauvin & D. E. Rumelhart (Eds.), Back-propagation: Theory, architectures and applications (pp. 433–486). Hillsdale: Erlbaum.Google Scholar
  180. Wolf, F., Engelken, R., Puelma-Touzel, M., Weidinger, J. D. F., & Neef, A. (2014). Dynamical models of cortical circuits. Current Opinion in Neurobiology, 25, 228–236. doi: 10.1016/j.conb.2014.01.017 PubMedCrossRefGoogle Scholar
  181. Wolpert, D. M., & Ghahramani, A. (2000). Computational principles of movement neuroscience. Nature Neuroscience Supplement, 3, 1212–1217.CrossRefGoogle Scholar
  182. Yamashita, Y., & Tani, J. (2008). Emergence of functional hierarchy in a multiple timescale neural network model: A humanoid robot experiment. PLoS Computational Biology, 4, e1000220. doi: 10.1371/journal.pcbi.1000220 PubMedPubMedCentralCrossRefGoogle Scholar
  183. Yeung, N. (2013). Conflict monitoring and cognitive control. In K. Ochsner & S. Kosslyn (Eds.), Oxford handbook of cognitive neuroscience (pp. 275–299). Oxford: Oxford University Press. doi: 10.1093/oxfordhb/9780199988709.013.0018 Google Scholar
  184. Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004). The neural basis of error detection: Conflict monitoring and the error-related negativity. Psychological Review, 111, 931–959. doi: 10.1037/0033-295x.111.4.931 PubMedCrossRefGoogle Scholar
  185. Yeung, N., & Nieuwenhuis, S. (2009). Dissociating response conflict and error likelihood in anterior cingulate cortex. Journal of Neuroscience, 29, 14506–14510. doi: 10.1523/jneurosci.3615-09.2009 PubMedPubMedCentralCrossRefGoogle Scholar
  186. Yuste, R. (2015). From the neuron doctrine to neural networks. Nature Reviews Neuroscience, 16, 487–497. doi: 10.1038/nrn3962 PubMedCrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of VictoriaVictoriaCanada

Personalised recommendations