Abstract
The basal ganglia control body movements, mainly, based on their values. Critical for this mechanism is dopamine neurons, which sends unpredicted value signals, mainly, to the striatum. This mechanism enables animals to change their behaviors flexibly, eventually choosing a valuable behavior. However, this may not be the best behavior, because the flexible choice is focused on recent, and, therefore, limited, experiences (i.e., short-term memories). Our old and recent studies suggest that the basal ganglia contain separate circuits that process value signals in a completely different manner. They are insensitive to recent changes in value, yet gradually accumulate the value of each behavior (i.e., movement or object choice). These stable circuits eventually encode values of many behaviors and then retain the value signals for a long time (i.e., long-term memories). They are innervated by a separate group of dopamine neurons that retain value signals, even when no reward is predicted. Importantly, the stable circuits can control motor behaviors (e.g., hand or eye) quickly and precisely, which allows animals to automatically acquire valuable outcomes based on historical life experiences. These behaviors would be called ‘skills’, which are crucial for survival. The stable circuits are localized in the posterior part of the basal ganglia, separately from the flexible circuits located in the anterior part. To summarize, the flexible and stable circuits in the basal ganglia, working together but independently, enable animals (and humans) to reach valuable goals in various contexts.
Similar content being viewed by others
References
Aarts H, Dijksterhuis A (2000) Habits as knowledge structures: automaticity in goal-directed behavior. J Pers Soc Psychol 78:53–63
Alexander GE, DeLong MR, Strick PL (1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 9:357–381
Ammons RB, Farr RG, Bloch E, Neumann E, Dey M, Marion R, Ammons CH (1958) Long-term retention of perceptual-motor skills. J Exp Psychol 55:318–328
Balleine BW, Dickinson A (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacol 37:407–419
Balleine BW, O'Doherty JP (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacol 35:48–69
Bergman H, Feingold A, Nini A, Raz A, Slovin H, Abeles M, Vaadia E (1998) Physiological aspects of information processing in the basal ganglia of normal and parkinsonian primates. Trends Neurosci 21:32–38
Bhatia KP, Marsden CD (1994) The behavioural and motor consequences of focal lesions of the basal ganglia in man. Brain 117:859–876
Bichot NP, Schall JD (1999) Effects of similarity and history on neural mechanisms of visual selection. Nat Neurosci 2:549–554
Bromberg-Martin ES, Hikosaka O (2009) Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron 63:119–126
Brown LL, Schneider JS, Lidsky TI (1997) Sensory and cognitive functions of the basal ganglia. Curr Opin Neurobiol 7:157–163
Cheramy A, Leviel V, Glowinski J (1981) Dendritic release of dopamine in the substantia nigra. Nature 289:537–542
Cowan N (2001) The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav Brain Sci 24:87–114 (discussion 114–185)
Ericsson KA, Lehmann AC (1996) Expert and exceptional performance: evidence of maximal adaptation to task constraints. Annu Rev Psychol 47:273–305
Flaherty AW, Graybiel AM (1991) Corticostriatal transformations in the primate somatosensory system. Projections from physiologically mapped body-part representations. J Neurophysiol 66:1249–1263
François C, Yelnik J, Percheron G, Fénelon G (1994) Topographic distribution of the axonal endings from the sensorimotor and associative striatum in the macaque pallidum and substantia nigra. Exp Brain Res 102:305–318
Ghazizadeh A, Griggs W, Hikosaka O (2016a) Object-finding skill created by repeated reward experience. J Vis 16:17
Ghazizadeh A, Griggs W, Hikosaka O (2016b) Ecological origins of object salience: reward, uncertainty, aversiveness, and novelty. Front Neurosci 10:378
Grupe DW, Nitschke JB (2013) Uncertainty and anticipation in anxiety: an integrated neurobiological and psychological perspective. Nat Rev Neurosci 14:488–501
Haber SN, Kim KS, Mailly P, Calzavara R (2006) Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J Neurosci 26:8368–8376
Hikosaka O, Wurtz RH (1983) Visual and oculomotor functions of monkey substantia nigra pars reticulata. III. Memory-contingent visual and saccade responses. J Neurophysiol 49:1268–1284
Hikosaka O, Sakamoto M, Usui S (1989a) Functional properties of monkey caudate neurons. I. Activities related to saccadic eye movements. J Neurophysiol 61:780–798
Hikosaka O, Sakamoto M, Usui S (1989b) Functional properties of monkey caudate neurons. II. Visual and auditory responses. J Neurophysiol 61:799–813
Hikosaka O, Sakamoto M, Usui S (1989c) Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol 61:814–832
Hikosaka O, Rand MK, Miyachi S, Miyashita K (1995) Learning of sequential movements in the monkey: process of learning and retention of memory. J Neurophysiol 74:1652–1661
Hikosaka O, Nakahara H, Rand MK, Sakai K, Lu X, Nakamura K, Miyachi S, Doya K (1999) Parallel neural networks for learning sequential procedures. Trends Neurosci 22:464–471
Hikosaka O, Takikawa Y, Kawagoe R (2000) Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol Rev 80:953–978
Hikosaka O, Nakamura K, Sakai K, Nakahara H (2002a) Central mechanisms of motor skill learning. Curr Opin Neurobiol 12:217–222
Hikosaka O, Rand MK, Nakamura K, Miyachi S, Kitaguchi K, Sakai K, Lu X, Shimo Y (2002b) Long-term retention of motor skill in macaque monkeys and humans. Exp Brain Res 147:494–504
Hikosaka O, Nakamura K, Nakahara H (2006) Basal ganglia orient eyes to reward. J Neurophysiol 95:567–584
Hikosaka O, Yamamoto S, Yasuda M, Kim HF (2013) Why skill matters. Trends Cogn Sci 17:434–441
Hikosaka O, Kim HF, Yasuda M, Yamamoto S (2014) Basal Ganglia circuits for reward value-guided behavior. Annu Rev Neurosci 37:289–306
Isoda M, Hikosaka O (2007) Switching from automatic to controlled action by monkey medial frontal cortex. Nat Neurosci 10:240–248
Isoda M, Hikosaka O (2008) Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement. J Neurosci 28:7209–7218
Jankowski J, Scheef L, Huppe C, Boecker H (2009) Distinct striatal regions for planning and executing novel and automated movement sequences. NeuroImage 44:1369–1379
Kaneda K, Nambu A, Tokuno H, Takada M (2002) Differential processing patterns of motor information via striatopallidal and striatonigral projections. J Neurophysiol 88:1420–1432
Kato M, Miyashita N, Hikosaka O, Matsumura M, Usui S, Kori A (1995) Eye movements in monkeys with local dopamine depletion in the caudate nucleus. I. Deficits in spontaneous saccades. J Neurosci 15:912–927
Kawagoe R, Takikawa Y, Hikosaka O (1998) Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1:411–416
Kawagoe R, Takikawa Y, Hikosaka O (2004) Reward-predicting activity of dopamine and caudate neurons—a possible mechanism of motivational control of saccadic eye movement. J Neurophysiol 91:1013–1024
Kim HF, Hikosaka O (2013) Distinct basal ganglia circuits controlling behaviors guided by flexible and stable values. Neuron 79:1001–1010
Kim HF, Hikosaka O (2015) Parallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards. Brain 138:1776–1800
Kim HF, Ghazizadeh A, Hikosaka O (2014) Separate groups of dopamine neurons innervate caudate head and tail encoding flexible and stable value memories. Front Neuroanat 8:120
Kim HF, Ghazizadeh A, Hikosaka O (2015) Dopamine neurons encoding long-term memory of object value for habitual behavior. Cell 163:1165–1175
Kori A, Miyashita N, Kato M, Hikosaka O, Usui S, Matsumura M (1995) Eye movements in monkeys with local dopamine depletion in the caudate nucleus. II. Deficits in voluntary saccades. J Neurosci 15:928–941
Lauwereyns J, Watanabe K, Coe B, Hikosaka O (2002) A neural correlate of response bias in monkey caudate nucleus. Nature 418:413–417
Lehericy S, Benali H, Van de Moortele PF, Pelegrini-Issac M, Waechter T, Ugurbil K, Doyon J (2005) Distinct basal ganglia territories are engaged in early and advanced motor sequence learning. Proc Natl Acad Sci USA 102:12566–12571
Lerner TN, Shilyansky C, Davidson TJ, Evans KE, Beier KT, Zalocusky KA, Crow AK, Malenka RC, Luo L, Tomer R, Deisseroth K (2015) Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits. Cell 162:635–647
Lu X, Hikosaka O, Miyachi S (1998) Role of monkey cerebellar nuclei in skill for sequential movement. J Neurophysiol 79:2245–2254
Matsumoto M, Hikosaka O (2009) Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459:837–841
Menegas W, Bergan JF, Ogawa SK, Isogai Y, Umadevi Venkataraju K, Osten P, Uchida N, Watabe-Uchida M (2015) Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass. Elife 4:e10032
Menegas W, Babayan BM, Uchida N, Watabe-Uchida M (2017) Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice. elife 6:e21886
Middleton FA, Strick PL (1996) The temporal lobe is a target of output from the basal ganglia. Proc Natl Acad Sci USA 93:8683–8687
Miyachi S, Hikosaka O, Miyashita K, Karádi Z, Rand MK (1997) Differential roles of monkey striatum in learning of sequential hand movement. Exp Brain Res 115:1–5
Miyachi S, Hikosaka O, Lu X (2002) Differential activation of monkey striatal neurons in the early and late stages of procedural learning. Exp Brain Res 146:122–126
Miyashita N, Hikosaka O, Kato M (1995) Visual hemineglect induced by unilateral striatal dopamine deficiency in monkeys. NeuroReport 6:1257–1260
Miyashita K, Rand MK, Miyachi S, Hikosaka O (1996) Anticipatory saccades in sequential procedural learning in monkeys. J Neurophysiol 76:1361–1366
Myers RE (1976) Comparative neurology of vocalization and speech: proof of a dichotomy. Ann N Y Acad Sci 280:745–760
Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O (2004) Dopamine neurons can represent context-dependent prediction error. Neuron 41:269–280
Nakamura K, Hikosaka O (2006) Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J Neurosci 26:5360–5369
Nakamura K, Sakai K, Hikosaka O (1998) Neuronal activity in medial frontal cortex during learning of sequential procedures. J Neurophysiol 80:2671–2687
Nakamura K, Sakai K, Hikosaka O (1999) Effects of local inactivation of monkey medial frontal cortex in learning of sequential procedures. J Neurophysiol 82:1063–1068
Nicola SM, Surmeier DJ, Malenka RC (2000) Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu Rev Neurosci 23:185–215
Parent A (1990) Extrinsic connections of the basal ganglia. Trends Neurosci 13:254–258
Rand MK, Hikosaka O, Miyachi S, Lu X, Miyashita K (1998) Characteristics of a long-term procedural skill in the monkey. Exp Brain Res 118:293–297
Rand MK, Hikosaka O, Miyachi S, Lu X, Nakamura K, Kitaguchi K, Shimo Y (2000) Characteristics of sequential movements during early learning period in monkeys. Exp Brain Res 131:293–304
Redgrave P, Rodriguez M, Smith Y, Rodriguez-Oroz MC, Lehericy S, Bergman H, Agid Y, DeLong MR, Obeso JA (2010) Goal-directed and habitual control in the basal ganglia: implications for Parkinson’s disease. Nat Rev Neurosci 11:760–772
Reynolds JN, Wickens JR (2002) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 15:507–521
Richfield EK, Young AB, Penney JB (1987) Comparative distribution of dopamine D-1 and D-2 receptors in the basal ganglia of turtles, pigeons, rats, cats, and monkeys. J Comp Neurol 262:446–463
Saint-Cyr JA, Ungerleider LG, Desimone R (1990) Organization of visual cortical inputs to the striatum and subsequent outputs to the pallido-nigral complex in the monkey. J Comp Neurol 298:129–156
Sakai K, Hikosaka O, Miyauchi S, Takino R, Sasaki Y, Pütz B (1998) Transition of brain activation from frontal to parietal areas in visuo-motor sequence learning. J Neurosci 18:1827–1840
Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27
Schultz W, Apicella P, Scarnati E, Ljungberg T (1992) Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12:4595–4610
Selemon LD, Goldman-Rakic PS (1985) Longitudinal topography and interdigitation of corticostriatal projections in the rhesus monkey. J Neurosci 5:776–794
Sidibe M, Pare JF, Smith Y (2002) Nigral and pallidal inputs to functionally segregated thalamostriatal neurons in the centromedian/parafascicular intralaminar nuclear complex in monkey. J Comp Neurol 447:286–299
Skinner BF (1963) Operant behavior. Am Psychol 18:503–515
Smith Y, Parent A (1986) Differential connections of caudate nucleus and putamen in the squirrel monkey (Saimiri sciureus). Neuroscience 18:347–371
Strasburger H, Rentschler I, Juttner M (2011) Peripheral vision and pattern recognition: a review. J Vis 11:13
Takada M, Tokuno H, Nambu A, Inase M (1998) Corticostriatal projections from the somatic motor areas of the frontal cortex in the macaque monkey: segregation versus overlap of input zones from the primary motor cortex, the supplementary motor area, and the premotor cortex. Exp Brain Res 120:114–128
Takara S, Hatanaka N, Takada M, Nambu A (2011) Differential activity patterns of putaminal neurons with inputs from the primary motor cortex and supplementary motor area in behaving monkeys. J Neurophysiol 106:1203–1217
Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O (2002) Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142:284–291
Takikawa Y, Kawagoe R, Hikosaka O (2004) A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward mapping. J Neurophysiol 92:2520–2529
Tanaka K (1996) Inferotemporal cortex and object vision. Annu Rev Neurosci 19:109–139
Tepper JM, Martin LP, Anderson DR (1995) GABAA receptor-mediated inhibition of rat substantia nigra dopaminergic neurons by pars reticulata projection neurons. J Neurosci 15:3092–3103
Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12:97–136
Tricomi E, Balleine BW, O’Doherty JP (2009) A specific role for posterior dorsolateral striatum in human habit learning. Eur J Neurosci 29:2225–2232
Van Hoesen GW, Yeterian EH, Lavizzo-Mourey R (1981) Widespread corticostriate projections from temporal cortex of the rhesus monkey. J Comp Neurol 199:205–219
Waszcak BL, Walters JR (1983) Dopamine modulation of the effects of gamma-aminobutyric acid on substantia nigra pars reticulata neurons. Science 220:218–221
Watanabe K, Lauwereyns J, Hikosaka O (2003) Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus. J Neurosci 23:10052–10057
Wolfe JM (1994) Guided Search 2.0 A revised model of visual search. Psychon Bull Rev 1:202–238
Wood W, Neal DT (2007) A new look at habits and the habit–goal interface. Psychol Rev 114:843–863
Worbe Y, Baup N, Grabli D, Chaigneau M, Mounayar S, McCairn K, Feger J, Tremblay L (2009) Behavioral and movement disorders induced by local inhibitory dysfunction in primate striatum. Cereb Cortex 19:1844–1856
Wunderlich K, Dayan P, Dolan RJ (2012) Mapping value based planning and extensively trained choice in the human brain. Nat Neurosci 15:786–791
Wymbs NF, Bassett DS, Mucha PJ, Porter MA, Grafton ST (2012) Differential recruitment of the sensorimotor putamen and frontoparietal cortex during motor chunking in humans. Neuron 74:936–946
Yamamoto S, Monosov IE, Yasuda M, Hikosaka O (2012) What and where information in the caudate tail guides saccades to visual objects. J Neurosci 32:11005–11016
Yamamoto S, Kim HF, Hikosaka O (2013) Reward value-contingent changes of visual responses in the primate caudate tail associated with a visuomotor skill. J Neurosci 33:11227–11238
Yasuda M, Hikosaka O (2015) Functional territories in primate substantia nigra pars reticulata separately signaling stable and flexible values. J Neurophysiol 113:1681–1696
Yasuda M, Yamamoto S, Hikosaka O (2012) Robust representation of stable object values in the oculomotor basal ganglia. J Neurosci 32:16917–16932
Yin HH, Mulcare SP, Hilario MR, Clouse E, Holloway T, Davis MI, Hansson AC, Lovinger DM, Costa RM (2009) Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat Neurosci 12:333–341
Acknowledgements
This research was supported by the Intramural Research Program at the National Institutes of Health, National Eye Institute.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hikosaka, O., Ghazizadeh, A., Griggs, W. et al. Parallel basal ganglia circuits for decision making. J Neural Transm 125, 515–529 (2018). https://doi.org/10.1007/s00702-017-1691-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00702-017-1691-1