Encyclopedia of Computational Neuroscience

Living Edition
| Editors: Dieter Jaeger, Ranu Jung

Basal Ganglia: Decision-Making

  • Wei WeiEmail author
  • Xiao-Jing Wang
Living reference work entry
DOI: https://doi.org/10.1007/978-1-4614-7320-6_519-1

Keywords

Deep Brain Stimulation Superior Colliculus Reward Rate Evidence Accumulation Superior Colliculus Neuron 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Definition

The basal ganglia (BG) participate not only in the selection of motor plans but also in perceptual decision-making. The functional structure of the BG and their close interconnections with the cortex and dopamine system allow the BG to be actively involved in the perceptual decision-making processes.

Detailed Description

The role of the BG in perceptual decision-making processes has recently attracted much attention (Ding and Gold 2013). Although the BG were much earlier proposed to be the central substrate for action selection and habit learning (Graybiel 1995; Mink 1996; Redgrave et al. 2010), their active participation in perceptual decision-making has been investigated only more recently (Ding and Gold 2010). The BG have close interaction with the frontal cortex and the lateral intraparietal area (LIP), where are believed to be the sites of evidence accumulation in decision processes (Gold and Shadlen 2007; Huk and Shadlen 2005; Kim and Shadlen 1999). The BG also have prominent interaction with the downstream subcortical motor-related areas, such as the thalamus and the superior colliculus (SC), and the midbrain dopamine system (Hikosaka et al. 2000). As diagrammed in Fig. 1, information from the cortex can be transmitted to the thalamus and SC through three BG pathways: (1) the direct pathway, from the striatum directly to the output nuclei GPi/SNr; (2) the indirect pathway, from the striatum through the GPe and STN to the output nuclei; and (3) the hyper-direct pathway, from the cortex directly to the STN then to the output nuclei. The opposing effect of dopamine signals on striatum medium spiny neurons that project through the direct and indirect pathways differentially makes the BG suitable for implementing reward dependence learning during the training process in decision-making tasks (Gerfen and Surmeier 2011). In addition, the hyper-direct pathway provides a natural substrate for inhibitory control when an evidence accumulation process needs to be cancelled. Parkinson’s disease (PD), originating from depleted dopamine levels within the BG, shows significant disruption of BG activity (DeLong and Wichmann 2009; Obeso et al. 2008). Impairment in perceptual decision-making performance in PD, e.g., impulsivity in the face of uncertainty, has been observed for human subjects (Frank et al. 2007). Several models have been suggested to investigate the roles that the BG play in perceptual decision-making for both normal and parkinsonian subjects, which will be discussed in the following.
Fig. 1

Scheme of the basal ganglia circuit and connections with the cortex, the thalamus, and the superior colliculus (SC). The direct, indirect, and hyper-direct pathways are indicated. GPe external segments of the globus pallidus (GP), GPi internal segments of the GP, STN subthalamic nucleus, SNr substantia nigra pars reticulate, LIP lateral intraparietal area, FEF frontal eye field, pSMA pre-supplementary motor area, ACC anterior cingulate cortex

Decision-Making Utilizing Go and No-Go Pathways

Network models including the BG, the cortex, and the thalamus can be constructed to investigate the role of the BG in perceptual decision-making under both normal and PD conditions (Frank 2006; Frank et al. 2004, 2007). In one such model (Frank et al. 2004), both the direct (“go”) and indirect (“no-go”) pathways of the BG (but without STN) were included, and the role of dopamine on reinforcement learning was checked. The striatal neurons projecting through the two pathways to the output nuclei are differentially modulated by positive and negative outcomes through reward-dependent dopamine signals: positive outcomes increase the excitability of direct pathway striatal projection neurons and decrease the excitability of indirect pathway projection neurons, while negative outcomes have the opposite effect (Gerfen and Surmeier 2011). It was shown in both simulation and experiments that learning from negative outcomes was impaired in patients with PD receiving dopamine medication. This model was later extended to include the STN to investigate effects of STN deep brain stimulation (DBS) on impulsivity in decision-making with high conflict (Frank 2006; Frank et al. 2007). These two later works emphasized the role of the cortico-STN projection in threshold adjustment during decision-making in high conflict situations. It was found that two forms of PD treatment impaired the performance in decision-making differently: dopamine medication impaired the ability to learn from wrong choices, while DBS treatment increased the impulsivity arising with high conflict. These two independent mechanisms that lead to impulsive choices for PD patients were then supported by experiments using a probabilistic selection task.

The above model has also been extended for investigating inhibitory control (Wiecki and Frank 2013). The stop-signal task is one common kind of perceptual decision-making paradigm that involves cancellation of an evidence accumulation process before decision initiation (Verbruggen and Logan 2008). The BG provides a natural substrate for implementing the cancellation process through the hyper-direct pathway. This has been evaluated in a cortical-BG-SC circuit model (Wiecki and Frank 2013), which extended the authors’ previous model by adding a frontal control circuit. The model has been shown to reproduce some key behavioral and physiological data.

Basal Ganglia Model for Multi-choice Decision-Making

Another hypothesis that has been considered computationally is the idea that the direct and hyper-direct pathways in the BG might provide a natural substrate for implementing an asymptotically optimal decision test for more than two alternatives, i.e., the multi-hypothesis sequential probability ratio test (MSPRT) (Baum and Veeravalli 1994). A model used to explore this concept (Bogacz and Gurney 2007) was based on the same BG functional structure as employed in a previous work (Gurney etal. 2001). In this model, each BG nucleus was represented by its average firing rate. The formula for detection probability in MSPRT was first log-transformed, and then each term in the expression for the log-transformed probability was connected with the firing activity of the STN and GP (GPe of primates). This mapping posed requirements for the input–output relation in these two nuclei. Specifically, the STN firing rate was required to be an exponential function of the inputs from the cortex and from the GP, while the GP firing rate was related to the summation of STN output for different channels corresponding to each alternative action. These requirements for the STN and GP input–output relationship were supported by experimental data from in vitro measurements of rat BG. The implementation of MSPRT could also be extended to include the indirect pathway. In this model, the interaction between the direct and hyper-direct pathways and the diffusive innervation from the STN to the GP and entopeduncular nucleus (EP), which is the homologue of GPi of primates, are essential for the BG in implementing the MSPRT.

Basal Ganglia Mechanism for Optimal Threshold Modulation

A biophysically based spiking network model involving the BG, the cortex, and the SC has been proposed to provide a mechanism for setting and adjusting the threshold in perceptual decision-making (Lo and Wang 2006). In this model, integrate-and-fire neuron model was used to describe individual neurons and tuned to specific properties in each area. The SC neurons fired a burst when the input was higher than a threshold value. The cortical circuit was modeled as an attractor network in which evidence accumulation occurred (Wang 2002). The cortical circuit projected to the SC through two routes: one was direct projection from the cortex to the SC and the other route through the BG to the SC. In the BG, only the direct pathway was included. The neurons in the striatum were designed to be silent and fire spikes only when the input was higher than a threshold level. When one of the competing populations in the cortex reached a value that drove the CD neurons to spike, the output nucleus SNr was inhibited, which in turn led to a disinhibition of the SC. When the SC was disinhibited, it could fire a burst in response to the excitatory input arriving directly from the cortex. This burst sent feedback to the cortex, terminated the evidence accumulation there, and set a threshold for perceptual decision-making. The SC burst was turned off by local recurrent connections within the SC. It was found that adjusting the strength of the corticostriatal connection could modulate the threshold within a wide range. On the contrary, adjusting the cortex to SC connection strength could only modestly modulate the threshold. A large range of threshold variability is beneficial when a subject needs to make a transition between response speed and accuracy (Gold and Shadlen 2002). In this model, modulation of the corticostriatal connection led to an inverted u shape of reward rate, reflecting a trade-off between speed and accuracy. The peak reward rate could be shifted according to the task difficulty. Physiologically, the modulation in the corticostriatal connection efficacy could be realized by a reward-dependent reinforcement learning process through dopaminergic synapses observed extensively in the striatum projection neurons (Gerfen and Surmeier 2011). Further experiments are needed to test this hypothesis.

Future Directions

Computational modeling represents an important approach in investigating possible roles of the BG in perceptual decision-making. Such work usually invokes some assumptions that require further experimental testing and also suggests new research directions. For example, in some circuit models, the STN and striatum received the same copy of input from the cortex, which usually represents evidence accumulated over time (e.g., Bogacz and Gurney 2007). But further experiments are needed to clarify the origin of cortical inputs to the direct and hyper-direct pathways of the BG (Mathai and Smith 2011). Ramping of striatal neuronal activity during evidence accumulation has been observed in primates (Ding and Gold 2013) and will curtail the efficiency of a corticostriatal mechanism for threshold modulation that includes only the direct pathway of the BG (as introduced in Lo and Wang 2006). Therefore, an extension of such a mechanism to incorporate the new experimental observation is called for, and experiments are needed to test the idea that the corticostriatal connection strength is modulated during tasks requiring a speed-accuracy trade-off.

Cross-References

References

  1. Baum CW, Veeravalli VV (1994) A sequential procedure for multihypothesis testing. IEEE Trans Inform Theory 40(6):1994–2007. doi:10.1109/18.340472CrossRefGoogle Scholar
  2. Bogacz R, Gurney K (2007) The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Comput 19(2):442–477. doi:10.1162/neco.2007.19.2.442PubMedCrossRefGoogle Scholar
  3. DeLong M, Wichmann T (2009) Update on models of basal ganglia function and dysfunction. Parkinsonism Relat Disord 15(Suppl 3):S237–240. doi:10.1016/S1353-8020(09)70822-3PubMedCrossRefGoogle Scholar
  4. Ding L, Gold JI (2010) Caudate encodes multiple computations for perceptual decisions. J Neurosci 30(47):15747–15759. doi:10.1523/JNEUROSCI.2894-10.2010PubMedCentralPubMedCrossRefGoogle Scholar
  5. Ding L, Gold JI (2013) The Basal Ganglia’s contributions to perceptual decision making. Neuron 79(4):640–649. doi:10.1016/j.neuron.2013.07.042PubMedCrossRefGoogle Scholar
  6. Frank MJ (2006) Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making. Neural Netw 19(8):1120–1136. doi:10.1016/j.neunet.2006.03.006PubMedCrossRefGoogle Scholar
  7. Frank MJ, Seeberger LC, O’Reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306(5703):1940–1943. doi:10.1126/science.1102941PubMedCrossRefGoogle Scholar
  8. Frank MJ, Samanta J, Moustafa AA, Sherman SJ (2007) Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science 318(5854):1309–1312. doi:10.1126/science.1146157PubMedCrossRefGoogle Scholar
  9. Gerfen CR, Surmeier DJ (2011) Modulation of striatal projection systems by dopamine. Annu Rev Neurosci 34:441–466. doi:10.1146/annurev-neuro-061010-113641PubMedCentralPubMedCrossRefGoogle Scholar
  10. Gold JI, Shadlen MN (2002) Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36(2):299–308PubMedCrossRefGoogle Scholar
  11. Gold JI, Shadlen MN (2007) The neural basis of decision making. Annu Rev Neurosci 30:535–574. doi:10.1146/annurev.neuro.29.051605.113038PubMedCrossRefGoogle Scholar
  12. Graybiel AM (1995) Building action repertoires: memory and learning functions of the basal ganglia. Curr Opin Neurobiol 5(6):733–741PubMedCrossRefGoogle Scholar
  13. Gurney K, Prescott TJ, Redgrave P (2001) A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biol Cybern 84(6):401–410PubMedCrossRefGoogle Scholar
  14. Hikosaka O, Takikawa Y, Kawagoe R (2000) Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol Rev 80(3):953–978PubMedGoogle Scholar
  15. Huk AC, Shadlen MN (2005) Neural activity in macaque parietal cortex reflects temporal integration of visual motion signals during perceptual decision making. J Neurosci 25(45):10420–10436. doi:10.1523/JNEUROSCI.4684-04.2005PubMedCrossRefGoogle Scholar
  16. Kim JN, Shadlen MN (1999) Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat Neurosci 2(2):176–185. doi:10.1038/5739PubMedCrossRefGoogle Scholar
  17. Lo CC, Wang XJ (2006) Cortico-basal ganglia circuit mechanism for a decision threshold in reaction time tasks. Nat Neurosci 9(7):956–963. doi:10.1038/nn1722PubMedCrossRefGoogle Scholar
  18. Mathai A, Smith Y (2011) The corticostriatal and corticosubthalamic pathways: two entries, one target. So what? Front Syst Neurosci 5:64. doi:10.3389/fnsys.2011.00064PubMedCentralPubMedCrossRefGoogle Scholar
  19. Mink JW (1996) The basal ganglia: focused selection and inhibition of competing motor programs. Prog Neurobiol 50(4):381–425PubMedCrossRefGoogle Scholar
  20. Obeso JA, Marin C, Rodriguez-Oroz C, Blesa J, Benitez-Temino B, Mena-Segovia J, Olanow CW (2008) The basal ganglia in Parkinson’s disease: current concepts and unexplained observations. Ann Neurol 64(Suppl 2):S30–S46. doi:10.1002/ana.21481PubMedGoogle Scholar
  21. Redgrave P, Rodriguez M, Smith Y, Rodriguez-Oroz MC, Lehericy S, Bergman H, Obeso JA (2010) Goal-directed and habitual control in the basal ganglia: implications for Parkinson’s disease. Nat Rev Neurosci 11(11):760–772. doi:10.1038/nrn2915PubMedCentralPubMedCrossRefGoogle Scholar
  22. Verbruggen F, Logan GD (2008) Response inhibition in the stop-signal paradigm. Trends Cogn Sci 12(11):418–424. doi:10.1016/j.tics.2008.07.005PubMedCentralPubMedCrossRefGoogle Scholar
  23. Wang XJ (2002) Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36(5):955–968PubMedCrossRefGoogle Scholar
  24. Wiecki TV, Frank MJ (2013) A computational model of inhibitory control in frontal cortex and basal ganglia. Psychol Rev 120(2):329–355. doi:10.1037/a0031542PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Center for Neural ScienceNew York UniversityNew YorkUSA
  2. 2.Department of Neurobiology and Kavli Institute for NeuroscienceYale University School of MedicineNew HavenUSA