Basal Ganglia: Mechanisms for Action Selection
KeywordsBasal Ganglion Superior Colliculus Projection Neuron Action Selection Parallel Loop
The basal ganglia are an interconnected circuit of fore- and midbrain nuclei common to all extant vertebrates, including the lamprey, which diverged from the main vertebrate line approximately 560 MYA. Moreover, there are analogous structures in birds and bony fish. The preservation of this circuit implies a crucial operation that solves a problem common to all vertebrates. While many functions for the basal ganglia have been proposed, our current best theory to unify these ideas is that this circuit solves the common problem of action selection. Informally put, this is the problem of how to decide what to do next.
Why Action Selection?
The necessity of a mechanism for action selection is imposed by the strong constraint that complex animals have a final common motor pathway: the connections of the spinal cord and the number of muscle groups limit the set of actions that can be expressed simultaneously. Some mechanism is required to reduce the repertoire of currently available actions to just those actions capable of simultaneous expression. Given the constraints of biological tissue, a central rather than distributed selection mechanism is likely the preferred solution implemented by a neural substrate (Prescott et al. 1999). Briefly, the argument runs as follows. Typically, distributed selection mechanisms, formed by reciprocal, inhibitory links between n behavior-representing nodes, contain n(n – 1) links and grow as 2n for every additional node. By contrast, a central selection device which is reciprocally connected with all n nodes (thus allowing control over the expression of each node’s represented behavior) requires just 2n links and grows by 2 for each additional node. Thus, a central selection device is more economical in both the number of connections required and the cost of adding nodes. Such economy of wiring appears to be a priority for the central nervous system (Cherniak 1994).
The basal ganglia appear to fulfill the criteria for such a central mechanism (Redgrave et al. 1999). The main input nucleus, the striatum, receives input from every region of cortex, from primary visual, auditory, and somatosensory cortex, through motor cortices, to the subregions of prefrontal cortex. It is thus in a position to integrate sensory information, current motor commands, planning, and goals into representations of the relative importance, or salience, of each action. The output of the basal ganglia targets regions in the thalamus and brain stem that underpin the execution of different actions. The basal ganglia are thus also in a position to directly control the activity in neural systems responsible for executing actions.
The idea that the basal ganglia perform action selection is also a potential solution to the long-standing paradox of their role in controlling movement (Marsden and Obeso 1994). Activity in the basal ganglia is strongly correlated with movement onset and parameters of that movement, and stimulation of regions of the striatum can evoke discrete, smooth limb movements (Alexander and DeLong 1985). Yet lesions of basal ganglia nuclei in animal studies or in Parkinson’s disease patients to alleviate their symptoms do not clearly impair the initiation of movement (Marsden and Obeso 1994). The theory that the basal ganglia are necessary to control the choice of actions explains how movement-related activity can arise yet not be needed to directly generate movement.
How Can the Basal Ganglia Circuit “Select”?
This disinhibition process has been worked out in detail for the control of voluntary eye movements, or saccades, in primates (see Hikosaka et al. 2000). Phasic activity in the caudate region of the striatum precedes saccade initiation and is time-locked to a decrease in activity in a subset of SNr neurons. The SNr projects to the intermediate layers of the superior colliculus, which contains a retinotopic map – neurons are laid out in a sheet that represents coordinates of gaze direction. A buildup of neural activity on this map drives the brain stem circuitry that controls the eye muscles and moves gaze to the location signaled by the map. The pause in SNr activity precedes and appears to be required for the buildup of activity in the retinotopic map of the superior colliculus. Conversely, continuously inhibiting SNr activity results in the inability to suppress saccades, as though every eye movement motor command was being executed (Hikosaka and Wurtz 1985). Thus, the direct striatum-SNr pathway likely controls the selection of saccade direction through a process of disinhibition.
Multiple Mechanisms Contributing to Action Selection
Computational models have provided quantitative tests of these ideas, revealed the further necessary mechanisms for successful selection, and shown how these mechanisms are implemented by components of the basal ganglia circuit. The box-and-arrow model accounts of “direct” pathway control of selection focussed on the flow of signals necessary to select one input and so omitted two crucial aspects of a selection process: how to resolve the competition between multiple action-representing inputs and how to switch from a currently selected action to another, more salient, action.
To be able to resolve a competition between actions first requires a substrate for separately representing each of those actions. A long-standing concept in basal ganglia research is the existence of parallel loops running through the basal ganglia (Alexander et al. 1986; Middleton and Strick 2000). At the macroscopic level, independent domains of the basal ganglia have been identified by the closed send-and-return loops originating in broad cortical regions (Alexander et al. 1986). Each macroscopic domain is thought to subdivide into a further large set of discrete parallel loops, termed channels here to distinguish them from the macroscopic-scale loops. For example, the somatotopic map found within the striatal motor domain is maintained throughout the basal ganglia’s intrinsic circuitry, such that there are separate channels for arm, leg, and face representations (Alexander and Crutcher 1990; Romanelli et al. 2005). Similar topographic maps have been proposed for the other macroscopic loops (Alexander and Crutcher 1990). Moreover, within these limb representations, there may be additional discrete channels corresponding to particular movements, demonstrated in striatum by microstimulation (Alexander and DeLong 1985) and markers for metabolic activity during behavior (Brown and Sharp 1995), and to repeated representations of body map locations (Brown et al. 1998). Computational models of the basal ganglia have proposed that it is these channels which form the basis for action representation, and so the resolution of competition is expressed by the selection of a particular channel in the output of the basal ganglia (Gurney et al. 2001a).
The basal ganglia circuit contains a number of mechanisms potentially suited to resolving competition between competing channels. One candidate mechanism is the inhibitory network formed by the local axon collaterals of the projection neurons in striatum, which could underpin a winner-takes-all competition between input signals (Alexander and Wickens 1993). A problem with this idea is that these feedback connections are weak and asymmetric so that a classical winner-takes-all competition is unlikely to be supported (Plenz 2003). Even if it is, because of the range of the local axon collaterals, this interaction can only extend over a short distance in the striatum, on the order of 200–300 μm (Wickens 1997; Humphries et al. 2010). Consequently, any dynamics within striatum are likely only to resolve very local competitions between cortical inputs (Gurney et al. 2001b).
A key insight from computational models is that the on-surround effect of the combined STN output can lead to SNr/GPi activity in many channels increasing above their tonic rate at the time of action selection – a phenomenon termed contrast enhancement. This makes functional sense: an enhanced tonic inhibitory signal being output from most channels indicates a definite nonselection of that action. However, when observed in experimental data, such increases in activity, time-locked to movement, have been interpreted as encoding that movement. By contrast, the models show how an increase in SNr/GPi neuron output time-locked to an action does not encode that action, but rather encodes the simultaneous nonselection of another potential action. Moreover, the models show that the number of neurons with such increased activity, time-locked to an action, will be in the majority and hence will appear more frequently during neural recordings (Humphries et al. 2006).
Once action selection has been achieved, then action switching is required: the sudden availability of a more salient action through changes in the environment or the animal’s needs should terminate the ongoing action and then switch to selecting the new, more salient alternative. Computational models have again shown how the same off-center, on-surround design naturally handles this: the new input further increases the STN output, canceling selection of the current action, while the corresponding new, strong inhibitory signal from striatum inhibits its target population in GPi/SNR (Gurney et al. 2001b; Humphries and Gurney 2002). Embedding these models in mobile robots has shown how smooth behavioral sequences naturally emerge from the changing saliences of each action, without the need for explicit storage of sequences (Khamassi et al. 2005; Prescott et al. 2006).
The “Indirect” Pathway: Prevention of Selection and Scaling of Selection
The contribution of the “indirect” pathway to selection has been more difficult to unravel. The original box-and-arrow models proposed that this pathway acts to counteract the selection of an action: increased inhibition of the GPe by its striatal inputs would lead to enhanced STN output to SNr/GPi, thereby counteracting inhibition they were receiving in the direct pathway (Alexander and Crutcher 1990). Consequently, it was proposed that any imbalance between the relative strengths of the direct and indirect pathways could lead to movement disorders (Albin et al. 1989). Recent experimental studies have shown how selectively activating the striatal D1 or D2 projection neuron populations does indeed lead to opposite effects on behavior: stimulation of D1 projection neurons facilitates behavior, whereas stimulation of D2 neurons suppresses ongoing behavior (Kravitz et al. 2010).
The computational models of Frank and colleagues (Frank et al. 2004; Frank 2005) have extended these ideas by proposing that recruitment of these “go” and “no-go” pathways by cortical input during learning would depend on the level of available dopamine. They showed these models could account for a range of learning biases exhibited by Parkinson’s disease patients, providing support for the go versus no-go competition between the direct and indirect pathways.
A further mystery was the role of STN feedback to the GPe (Fig. 1). An analysis by Gurney et al. (2001b) showed that this negative feedback loop provided capacity scaling for action selection. Without a loop between STN and GPe, the diffuse STN output means that the total excitatory input to the SNr/GPi would grow in proportion to the number of active inputs to the basal ganglia; with a sufficient number of inputs, the excitatory input would grow large enough to overwrite all focussed inhibition from the striatum and prevent any selection. However, the negative feedback loop ensures that the STN output is scaled to the number of active inputs: the more the inputs, the more the STN output tries to grow, but is held in check by the GPe output which is itself a function of the STN output. In this way, the basal ganglia’s capacity can scale and so can operate as a selection mechanism independently of the number of currently competing channels.