Classic theories suggest that central serotonergic neurons are involved in the behavioral inhibition that is associated with the prediction of negative rewards or punishment. Failed behavioral inhibition can cause impulsive behaviors. However, the behavioral inhibition that results from predicting punishment is not sufficient to explain some forms of impulsive behavior. In this article, we propose that the forebrain serotonergic system is involved in “waiting to avoid punishment” for future punishments and “waiting to obtain reward” for future rewards. Recently, we have found that serotonergic neurons increase their tonic firing rate when rats await food and water rewards and conditioned reinforcer tones. The rate of tonic firing during the delay period was significantly higher when rats were waiting for rewards than for tones, and rats were unable to wait as long for tones as for rewards. These results suggest that increased serotonergic neuronal firing facilitates waiting behavior when there is the prospect of a forthcoming reward and that serotonergic activation contributes to the patience that allows rats to wait longer. We propose a working hypothesis to explain how the serotonergic system regulates patience while waiting for future rewards.
Serotonin (5-hydroxytryptamine, 5-HT) has been implicated in a variety of motor, cognitive, and affective functions [1–3], such as locomotion, sleep–wake cycles, and mood disorders. A large number of studies have shown that reduced levels of 5-HT in the central nervous system promote impulsive behaviors [4–8], including impulsive action (i.e., the failure to suppress inappropriate actions) and impulsive choice (i.e., the choice of small, immediate rewards over larger, delayed rewards). However, recent studies of the effects of manipulating 5-HT levels on impulsivity have reported mixed results [9–27].
The aim of this article is to propose a new concept of the role of 5-HT system in waiting for delayed rewards, based on our recent microdialysis and unit recording studies [28, 29]. This article is organized as follows. First, we present an overview of the types of impulsive behaviors that are involved in the 5-HT system. The depletion of forebrain serotonin transmission induces impulsive action as assessed by the five-choice serial reaction time task (5-CSRTT), which is commonly used to measure impulsive action [13–16, 19–21, 25]. Modulating central 5-HT transmission also influences impulsive choice as assessed using the commonly used delay-discounting task. However, contradictory results have been reported [9–12, 17, 18, 22–24, 26, 27].
Second, we review recent microdialysis and unit recording studies that examined the 5-HT neural activity of behaving animals. We found previously that 5-HT efflux in the dorsal raphe nucleus (DRN), the primary origin of 5-HT projections to the forebrain , increases when rats perform a task that requires waiting for a delayed reward . We also found that 5-HT neurons in the DRN exhibit an increase in their tonic firing rate when rats await delayed rewards and that these neurons cease firing before rats stop waiting for a reward that has been delayed for too long . These results demonstrate an association between dorsal raphe 5-HT activation and the waiting behavior that is associated with a delayed reward.
Third, we propose the new concept “waiting to obtain reward”, which means that animals reduce their behavioral activity to obtain a forthcoming reward. We hypothesize that an increase in 5-HT neural activity during waiting for a delayed reward contributes to the regulation of the “waiting to obtain reward”. Classic theories suggest that central 5-HT neurons are involved in the behavioral inhibition that is associated with the prediction of negative rewards or punishment [30–33]. We propose that the 5-HT system is involved both in “waiting to avoid punishment” for future punishment and in “waiting to obtain reward” for future reward. Some forms of impulsive action that have been studied pharmacologically or by lesion studies can be explained by the concept of “waiting to obtain reward”.
Finally, we propose a neural mechanism of patience that is related to the “waiting to obtain reward”. Interactions among the orbitofrontal cortex (OFC), medial prefrontal cortex (mPFC), nucleus accumbens (NAcc), and 5-HT neurons in the DRN are closely related to the “waiting to obtain a reward”.
Serotonin and Impulsivity
An altered functionality of the serotonergic system has been implicated in impulsivity. Impulsivity can be divided broadly into impulsive action and impulsive choice  (Table 1). Impulsive action is the inability to inhibit undesired actions. One of the most frequently used and well-characterized tasks for rats is the five-choice serial reaction time task (5-CSRTT) [34–36] (Fig. 1). In the 5-CSRTT, to obtain a food pellet, the rat is required to perform a nose-poke response in one of five apertures in which a stimulus light located behind the aperture is briefly illuminated (Fig. 1a). The correct response yields a reward at a food magazine. A trial is initiated by the entry of the rat to the food magazine. Following the beginning of a trial and prior to the activation of a stimulus light, there is a 5-s inter-trial interval, during which the rat must refrain from responding at the five-aperture array. Any nose-poke responses to one of the five apertures before the presentation of the stimulus light are characterized as premature responses (Fig. 1b). Premature responses are used as an index of impulsive action. Incorrect responses (i.e., responses to the wrong location after the stimulus is turned on) and omissions [i.e., failure to respond within the limited hold (LH) period] do not indicate impulsive action (Fig. 1b). In the 5-CSRTT, the rat performs to obtain a food reward, and the light stimulus is presented in one of the five apertures as a conditioned reinforcer.
Impulsive choice is the tendency to choose an immediate reward over a delayed reward, even if the delayed reward is known to be larger. One of the most widely used measures of impulsive choice in laboratory animals is the delay-discounting task  (Fig. 2). In this task, rats choose between pressing one lever that always results in the immediate delivery of a single food pellet and pressing another lever that always results in four pellets, but only after a delay that is increased progressively across blocks of trials in each session. Impulsive choice, in this paradigm, is commonly attributed to subjects who show a greater choice of the lever giving the smaller, more immediate reward at some or all of the increased delays on the lever giving the larger reward.
The role of the 5-HT system in impulsivity has been studied primarily using forebrain 5-HT depletion and the pharmacological treatment of 5-HT receptors and transporters, yielding contradictory results, particularly in impulsive choice (Table 2). For example, the administration of serotonin selective reuptake inhibitors (SSRIs), which increase extracellular serotonin concentrations, increased the selection rate of a large, delayed reward over a small, immediate reward, meaning a decrease in impulsive choice [9, 10]; in contrast, a nonselective 5-HT antagonist promoted self-controlled choice . Forebrain 5-HT depletion leads to the choice of small, immediate rewards more frequently than large, delayed rewards [9, 11, 18, 27], and systemic treatment with the 5-HT1A agonist 8-hydroxy-2-(di-n-propylamino)-tetralin (8-OH-DPAT), which suppresses 5-HT neuronal firing through 5-HT1A autoreceptors, produces impulsive choice [17, 26]. A recent microdialysis study reported a significant increase in 5-HT efflux in the mPFC of rats that were performing the delay-discounting task compared with “yoked” rats, which exercised no choice . However, a recent rodent study demonstrated that the depletion of forebrain 5-HT via the intraventricular administration of the selective neurotoxin 5,7 dihydroxytryptamine (5,7-DHT) produced significant increases in premature responses in the 5-CSRTT but had no effect on the delay-discounting task [14, 15, 22, 23]. An increase in 5-HT release in the mPFC has been correlated with impulsive actions in a visual attention task . Finally, 5-HT2A and 5-HT2C receptor subtypes have been reported to have opposing effects on impulsive choice [13, 16, 19–21, 25]. A potential explanation for these contrasting data is the existence of multiple pre- and post-synaptic 5-HT receptor subtypes in the target areas of 5-HT projections  and dynamic compensation mechanisms that are dependent on how 5-HT was manipulated (e.g., by depletion and/or by pharmacological treatment).
In the 5-CSRTT, to decrease impulsive action, the rats have to wait before making action. In the delay-discounting task, before the rats have to be patient to wait for delayed reward, decision making process according to evaluation of relative value of immediate small reward and delayed large reward is needed. Previous studies have shown that 5-HT manipulations are more effective on impulsive action than on impulsive choice. These results suggest that 5-HT depletion may influence waiting behavior more effectively than decision making process.
Activation of Serotonin Neurons for Delayed Reward
Previous recording studies of the DRN revealed that the activation of putative 5-HT neurons was correlated with the level of behavioral arousal , salient sensory stimuli [40–42], and rhythmic motor outputs . In a recent study, DRN neurons were recorded in monkeys that were performing a reward-oriented saccade task; these neurons exhibited a tonic reward-related response for both small and large rewards . Some neurons showed increased activity during the prediction and receipt of the large reward, whereas other neurons showed increased activity during the prediction and receipt of the small reward. Further analyses revealed that neurons that are tonically excited or inhibited during the task predominantly carried positive reward signals and negative reward signals, respectively . In an odor-guided choice discrimination task, DRN neurons recorded from rats showed that the firing pattern of DRN neurons was correlated with diverse behavioral events, including rewards and conditioned cues . However, no studies have shown a functional link between 5-HT neural activity in the DRN and impulsive behaviors. We sought to provide direct evidence of the involvement of forebrain 5-HT activity in the regulation of impulsive behaviors. Thus, we recorded the firing of 5-HT neurons in the DRN while rats performed a task that required waiting for rewards and a conditioned reinforcer tone .
We proposed previously that 5-HT controls the time scale of reward prediction, with increased 5-HT activity promoting the consideration of further delayed rewards in action choice . Furthermore, in a human functional magnetic resonance imaging study, we demonstrated that the DRN was activated when subjects learned to obtain large future rewards . The manipulation of central 5-HT levels via dietary tryptophan depletion and loading has shown that low serotonin levels steepen delayed reward discounting in humans . These results support our serotonin hypothesis. However, there is little direct evidence that 5-HT neural firing and efflux are enhanced by the expectation of a delayed reward as opposed to an immediate reward.
In our pursuit of direct evidence that the DRN 5-HT neurons are activated when animals work for delayed rewards, we previously used in vivo microdialysis to compare 5-HT levels in the DRN of rats that were working for immediate or delayed rewards. Dialysates were collected while the rats performed a task that required waiting for a delayed reward . Serotonin efflux in the rat DRN increased when animals were required to continue poking their nose at the reward site for 4 s and then wait for a delayed reward compared with receiving a reward immediately following a nose poke . Although this result shows that DRN 5-HT neurons are specifically activated in relation to the waiting period for delayed rewards, it remains unclear which behavioral events trigger serotonin neurons to fire, as the temporal resolution of microdialysis measurements is of the order of minutes.
To examine how serotonergic neurons respond in real time, we recorded putative serotonergic neurons in the DRN while the rat performed a free operant task that we designated a sequential food–water navigation task . In this task, rats were individually trained and tested in a cylindrical apparatus 1.5 m in diameter with a 45-cm-high wall; three identical-looking cylinders that served as the tone, food, and water sites were fixed in an isosceles triangle (Fig. 3a). This task required the rats to make alternating visits and nose-pokes to the food and water sites via the tone site visit and nose-poke. The rats initiated a trial by maintaining nose-poking in a fixed posture to achieve a continuous interruption of the photo-beam at the tone site during a delay period until a tone (8 kHz, 0.4 s) was presented, thus signaling that a reward was available at one of the reward sites. After the presentation of the tone, the rat was required to continue to nose-poke at one of the reward sites during another delay period until the reward was delivered (Fig. 3b). To continue the task, the rats had to alternately visit two reward sites via the tone site. In the sequential food–water navigation task, the tone worked as a conditioned reinforcer that predicted future food or water rewards. We called the delay periods that preceded the tone and the rewards (food and water) the tone delay and reward delay, respectively.
We found that many 5-HT neurons exhibited an increase in tonic activity during the period in which the rat waited for forthcoming rewards  (Fig. 4a, b). These results revealed that the waiting behavior for delayed rewards was the crucial behavioral event for activating 5-HT neurons in the DRN. To investigate further how 5-HT neural activity is related to waiting behavior for delayed rewards, we compared the neural activity of rats that were waiting for delayed rewards with a conditioned reinforcer tone  (Fig. 4c). The sustained 5-HT neural activity during the reward delay period was significantly higher than the activity during the tone delay period, which suggests that this increased activity was not attributable simply to the nose-poking behavior, which was required for both the reward and tone sites. When the reward and tone delays were independently extended (an extended reward or tone delay test), tonic firing persisted until the delivery of the reward or tone, and the rats waited longer for primary rewards than for the conditioned reinforcer tone  (Fig. 5). When the reward delay was gradually prolonged during the extended reward delay test, the number of failures to wait for delayed rewards (rewards wait error) gradually increased, and 5-HT neural activity ceased before the rats ceased waiting for possible future rewards  (Fig. 6a, b). When an expected water reward was suddenly omitted for several continuous trials (i.e., a water omission test), 5-HT neural activity also dropped preceding the exit from the water site during adaptively truncated waiting in the water omission trials  (Fig. 6c, d). These results suggest that an increase in 5-HT neuronal firing facilitates a rat’s waiting behavior with the prospect of forthcoming rewards and that higher serotonin activation enables longer waiting periods.
Waiting to Obtain Reward and Waiting to Avoid Punishment
Classic theories suggest that central 5-HT neurons are involved in the behavioral inhibition that is associated with the prediction of negative rewards or punishment [30–33]. In both pharmacological treatment and lesion studies that decrease 5-HT transmission, animals exhibited a deficit in passive avoidance in which animals learned to suppress their natural tendency to enter a dark chamber from a light chamber after they experienced aversive stimuli such as a foot shock in the dark chamber . Dietary tryptophan depletion abolished the punishment-induced slowing of reaction times for the go responses in a go/no-go task in healthy volunteers. The go responses in control subjects became slower when incorrect go responses evoked a large punishment compared with conditions in which correct go responses earned a large reward. This punishment-induced inhibition of responding was absent following tryptophan depletion .
We found that 5-HT neural activity increased when rats waited for delayed rewards. These results suggest that the 5-HT system contributes to the modulation of patience for the attainment of rewards. In this article, we propose that the 5-HT system is involved in the decrease of behavioral activity both to avoid aversive events with a prediction of punishment as well as to achieve rewards with a prediction of reward. To clarify the decrease in these two behavioral activities, we defined the behavior as either “waiting to obtain reward” when they decreased their activity to obtain a reward or “waiting to avoid punishment” when the animals suppressed their activity to prevent future punishment.
In our task, maintaining nose-poking for a delayed reward is the “waiting to obtain reward”. When the expected water reward was suddenly omitted for several consecutive trials, the duration of nose-poking gradually shortened . This result suggests that rats maintained their nose-poking behavior at reward sites to receive rewards when they predict that a reward is forthcoming.
In the 5-CSRTT, the requirement that the rat withhold nose-poke responses in one of the five apertures until an internal stimulus light is briefly illuminated is the “waiting to obtain reward”. The increase in premature responses after forebrain 5-HT depletion might result from an inability to wait for the visual targets that act as a conditioned reinforcer and not from fear or the prediction of a 5-s time-out following a premature response, because if the rats are patient and wait for the visual targets, they receive the reward. If the intent of the rats is to avoid time-out and not wait for the visual targets, the rats might remain motionless after the presentation of the stimulus light. This waiting for visual targets in the 5-CSRTT resembles the waiting for tones that is observed in the sequential food–water navigation task in which 5-HT neural activity increases owing to both of the stimuli working as conditioned reinforcers. In the 5-CSRTT, 5-HT neurons might increase their firing rate while the rat is waiting for the visual target that is presented in one of the five apertures.
Similar to behavioral inhibition, the term “action inhibition” is used to explain the inhibitory control of animal behavior. Impulsive action occurs due to lack of action inhibition. Action inhibition can be divided into action restraint and action cancellation  (Table 1). Action restraint describes the inhibition of the motor response before the response has been initiated. Action restraint is studied using tasks such as the go/no-go task, and the main focus is the ability or failure to withhold responding . Action cancellation indicates the inhibition of a motor response that was already initiated during the execution of the motor response. Action cancellation is studied using the stop-signal reaction time task (SSRTT) in which the stop signal is presented to inhibit the ongoing go response following the presentation of go signal . Action restraint encompasses both “waiting to obtain reward” and “waiting to avoid punishment” as defined in this article (Table 1).
In the SSRTT, the manipulation of 5-HT levels in either rats or humans does not affect performance when the previously initiated go response is required to stop by the tone signal [51–55]. However, 5-HT depletion impairs task performance when the tone signal is presented without a delay at the start of the go response and when the time period during which the rat is required to withhold the go response is extended . The inhibition of the response in the SSRTT that is induced by the simultaneous presentation of the go signal and stop signal resembles the no-go trial of the go/no-go task. In rats, 5-HT depletion impairs waiting but not the stop-signal reaction time, which supports a role of 5-HT in the “waiting to obtain reward”, as in the SSRTT, the success to withhold response is rewarded .
Serotonin has also been implicated in inhibitory control in the go/no-go task [56, 57]. When a correct no-go response is rewarded, withholding the go response would be the “waiting to obtain reward”, as the animals execute a no-go trial while predicting a future reward. In contrast, if an incorrect no-go response is punished, withholding the go response would be the “waiting to avoid punishment”, as the animals inhibit a go response to avoid punishment. In a symmetrically reinforced go/no-go conditional visual discrimination task, global 5-HT depletion using 5,7-DHT fails to acquire visual discrimination due to an inability to withhold responding to a no-go signal and also impairs the ability of previously trained rats to subsequently inhibit correctly to the no-go signal . This inability to withhold the response in no-go trials can be explained by an impairment of the “waiting to obtain reward”. Rats that receive para-chloroamphetamine to induce 5-HT depletion within the brain show impaired acquisition of a go/no-go visual discrimination task in which the go responses during the light and dark phases are rewarded and non-rewarded, respectively . In this study, withholding the go response during the dark phase is the “waiting to avoid punishment”.
Depleting 5-HT by a median (but not dorsal) raphe injection of 5,7-DHT impairs the acquisition and performance of behaviors that are maintained under a differential reinforcement of a low-rate (DRL) schedule of reinforcement . During the DRL schedule of reinforcement, operant responses are reinforced only when they occur after a fixed interval (e.g., 20 s, as in a DRL 20-s schedule) following the previous rewarded response. Similarity between premature responses in the 5-CSRTT and non-rewarded operant responses in the DRL schedule has been suggested, as waiting for a defined temporal interval is required for reinforcement in both tasks . A primary difference between the two tasks is that the 5-CSRTT, uses explicit signals—the stimulus light at one of the five apertures—that predict future rewards. On the other hand, the DRL schedule has no explicit signal for future rewards. The lack of an effect on behavior following dorsal raphe 5,7-DHT lesions may be due to a lack of explicit goal expectations (rewards or conditioned reinforcers) that can be obtained after waiting, as waiting is not in itself sufficient to obtain rewards or conditioned reinforcers in the DRL schedule.
How is 5-HT neural activity related to the “waiting to obtain reward” and “waiting to avoid punishment”? Does the same DRN 5-HT neuron contribute to both the “waiting to obtain reward” and “waiting to avoid punishment”? Alternatively, do 5-HT neurons differently regulate the “waiting to obtain reward” and “waiting to avoid punishment”? Although no study has examined how 5-HT neurons respond during the “waiting to avoid punishment”, 5-HT neurons might increase their firing rate during the “waiting to avoid punishment”. If the same rat can learn that the same behavior (such as maintaining nose-poking causes reward gain or punishment avoidance, depending on the situation), we could examine how the activity of a single 5-HT neuron responds while “waiting to obtain reward” and “waiting to avoid punishment”.
We would like to propose a task for this purpose. In a tone discrimination task, tone 1 and tone 2 are associated with a reward and punishment (e.g., an electric shock), respectively. After the presentation of tone 1, the rat can receive a reward by maintaining nose-poking in a reward site for several seconds. This nose-poke behavior is the “waiting to obtain reward”. The rat can avoid punishment by maintaining nose-poking in a safe site for several seconds after the presentation of tone 2. In this case, the nose-poke behavior is the “waiting to avoid punishment”. During the task, the rat would wait for the reward in the positive reward prediction and inhibit its behavior to avoid aversive stimuli with a negative reward expectation. Unit recordings of 5-HT neurons from this rat would reveal whether the same 5-HT neurons are related to the “waiting to obtain reward” and the “waiting to avoid punishment” or whether the “waiting to obtain reward” and the “waiting to avoid punishment” are regulated separately by different 5-HT neurons. Furthermore, to examine which neural circuits regulate the “waiting to obtain reward” and “waiting to avoid punishment”, it is important to examine the projections of the 5-HT neurons that respond during the “waiting to obtain reward” and/or the “waiting to avoid punishment”. Electrical stimulation of these projection sites to produce antidromic activation may reveal the areas that are influenced by 5-HT.
Putative Role of Serotonin for the Regulation of Patience for Future Rewards
The neural circuitry that mediates the “waiting to obtain reward” might be related to patience with respect to future rewards. What are the neural substrates of the “waiting to obtain reward”, and how does 5-HT influence these neural circuits? First, the NAcc contributes to the “waiting to obtain reward”. Evidence from lesion studies suggests that the core region of the NAcc contributes to both DRL response inhibition and to premature responses in the 5-CSRTT [59, 60]. Recent studies have shown that systemic application of 5-HT2A receptor antagonists causes a reduction in impulsive action, whereas 5-HT2C receptor antagonists cause an increase in impulsivity in the 5-CSRTT [13, 16, 19, 21]. Intra-NAcc infusion of the 5-HT2A receptor antagonist M100907 and the 5-HT2C receptor antagonist SB242084 significantly decrease and increase, respectively, the premature responses in the 5-CSRTT . The integrity of the NAcc is necessary for the prevention of premature responses during the anticipation or waiting for reward presentation periods. Unit recording studies have revealed that neurons in the NAcc exhibit an anticipatory response to delayed rewards during waiting [62, 63].
Second, the mPFC and OFC also contribute to the waiting to obtain reward behavior. Excitotoxic lesions of the infralimbic PFC, the ventral part of the mPFC, and the OFC induce premature responses in the 5-CSRTT . Intra-mPFC infusion of the 5-HT2A antagonist M100907 decreases premature responses in the 5-CSRTT when the duration of the visual target is shortened . However, no effect of either M100907 or the 5-HT2C receptor antagonist SB242084 on premature responses with standard task parameters in the 5-CSRTT are observed with intra-mPFC infusions . Blocking NMDA receptors in the mPFC by intracortical infusion of 3-(R)-2-carboxypiperazin-4-propyl-1-phosphonic acid (CPP) markedly and reliably enhance extracellular glutamate [65, 66] and increase premature responses in the 5-CSRTT . The increase in premature responses that is induced by intra-mPFC CPP infusion is prevented by the systemic administration of the 5-HT2C receptor agonist Ro60-0175 . A recent study showed that 5-HT2C receptors are located in GABAergic interneurons within the mPFC, in particular, in neurons containing the calcium-binding protein palvalbumin . This result suggests that an increase in GABAergic tone mediated by 5-HT2C receptors in the mPFC contributes to the suppression of CPP-induced glutamate release and an increase in premature responses.
In the OFC and mPFC, a sustained increase in activity has been observed during waiting for delayed rewards [70–73]. These neural activities may interact with the activity of 5-HT neurons in the DRN. The role of the OFC is to signal expected outcomes to projection regions but not to contribute directly to response inhibition . Recently, the firing rates of many single neurons in the OFC were shown to represent the confidence of decision making when decision difficulty was manipulated by varying the distance between the stimuli and the category boundary . When tested in a delayed reward version of the task, the willingness of the rats to wait for rewards increased with confidence . An explicit representation of the goal and/or the value of the goal would be important in the learning of patience to receive future rewards. Confidence and/or reward expectation would be helpful in patience for delayed rewards, as explicit representations of goals would enable animals to be patient while waiting for future rewards. These confidence-related signals may influence 5-HT neural activity during the “waiting to obtain reward”.
It remains unclear how confidence and/or the explicit representation of goals modulate 5-HT neural activity and how 5-HT neural activity influences the neural activity of projection sites. Simultaneous recordings from the OFC/mPFC, NAcc, and DRN would help to examine the contribution of these regions to the “waiting to obtain reward”. For example, how is the neural activity of these regions correlated with an animal’s behavior when it stops waiting for the delayed reward? Moreover, how does neural activity change according to changes in the animal’s behavior during manipulations of confidence and the “waiting to obtain reward”?
It is well established that the 5-HT system contributes to “waiting to avoid punishment” when there is the prospect of future punishment. In this article, we propose that the 5-HT system also plays a role in “waiting to obtain reward”, which is a waiting behavior with the purpose of receiving future rewards. Interactions among the OFC, mPFC, NAcc, and 5-HT neurons in the DRN could be involved in the waiting to obtain reward behavior (Fig. 7). Neural circuits for the “waiting to obtain reward” might regulate patience while waiting for future rewards. Clarifying the neural mechanism in the “waiting to obtain reward” would be beneficial for the clinical treatment of patients who lack the patience to wait for delayed rewards: for example, individuals with attention deficit/hyperactivity disorder or drug addiction. Further study is needed to determine how 5-HT efferents modulate cellular and network properties to facilitate the “waiting to obtain reward” and how the afferents to the DRN regulate 5-HT neural activities.
Jacobs BL, Azmitia EC (1992) Structure and function of the brain serotonin system. Physiol Rev 72:165–229
Adell A, Celada P, Abellan MT, Artigas F (2002) Origin and functional role of the extracellular serotonin in the midbrain raphe nuclei. Brain Res Brain Res Rev 39:154–180
Hensler JG (2006) Serotonergic modulation of the limbic system. Neurosci Biobehav Rev 30:203–214
Cardinal RN (2006) Neural systems implicated in delayed and probabilistic reinforcement. Neural Netw 19:1277–1301
Evenden JL (1999) Varieties of impulsivity. Psychopharmacology (Berl) 146:348–361
Dalley JW, Everitt BJ, Robbins TW (2011) Impulsivity, compulsivity, and top-down cognitive control. Neuron 69:680–694
Pattij T, Vanderschuren LJ (2008) The neuropharmacology of impulsive behaviour. Trends Pharmacol Sci 29:192–199
Winstanley CA, Eagle DM, Robbins TW (2006) Behavioral models of impulsivity in relation to ADHD: translation between clinical and preclinical studies. Clin Psychol Rev 26:379–395
Bizot J, Le Bihan C, Puech AJ, Hamon M, Thiebot M (1999) Serotonin and tolerance to delay of reward in rats. Psychopharmacology (Berl) 146:400–412
Bizot JC, Thiebot MH, Le Bihan C, Soubrie P, Simon P (1988) Effects of imipramine-like drugs and serotonin uptake blockers on delay of reward in rats. Possible implication in the behavioral mechanism of action of antidepressants. J Pharmacol Exp Ther 246:1144–1151
Denk F, Walton ME, Jennings KA, Sharp T, Rushworth MF, Bannerman DM (2005) Differential involvement of serotonin and dopamine systems in cost–benefit decisions about delay or effort. Psychopharmacology (Berl) 179:587–596
Evenden JL, Ryan CN (1996) The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology (Berl) 128:161–170
Fletcher PJ, Tampakeras M, Sinyard J, Higgins GA (2007) Opposing effects of 5-HT(2A) and 5-HT(2C) receptor antagonists in the rat and mouse on premature responding in the five-choice serial reaction time test. Psychopharmacology (Berl) 195:223–234
Harrison AA, Everitt BJ, Robbins TW (1997) Doubly dissociable effects of median- and dorsal-raphe lesions on the performance of the five-choice serial reaction time test of attention in rats. Behav Brain Res 89:135–149
Harrison AA, Everitt BJ, Robbins TW (1997) Central 5-HT depletion enhances impulsive responding without affecting the accuracy of attentional performance: interactions with dopaminergic mechanisms. Psychopharmacology (Berl) 133:329–342
Higgins GA, Enderlin M, Haman M, Fletcher PJ (2003) The 5-HT2A receptor antagonist M100907 attenuates motor and 'impulsive-type' behaviours produced by NMDA receptor antagonism. Psychopharmacology (Berl) 170:309–319
Liu YP, Wilkinson LS, Robbins TW (2004) Effects of acute and chronic buspirone on impulsive choice and efflux of 5-HT and dopamine in hippocampus, nucleus accumbens and prefrontal cortex. Psychopharmacology (Berl) 173:175–185
Mobini S, Chiang TJ, Ho MY, Bradshaw CM, Szabadi E (2000) Effects of central 5-hydroxytryptamine depletion on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl) 152:390–397
Passetti F, Dalley JW, Robbins TW (2003) Double dissociation of serotonergic and dopaminergic mechanisms on attentional performance using a rodent five-choice reaction time task. Psychopharmacology (Berl) 165:136–145
Talpos JC, Wilkinson LS, Robbins TW (2006) A comparison of multiple 5-HT receptors in two tasks measuring impulsivity. J Psychopharmacol 20:47–58
Winstanley CA, Chudasama Y, Dalley JW, Theobald DE, Glennon JC, Robbins TW (2003) Intra-prefrontal 8-OH-DPAT and M100907 improve visuospatial attention and decrease impulsivity on the five-choice serial reaction time task in rats. Psychopharmacology (Berl) 167:304–314
Winstanley CA, Dalley JW, Theobald DE, Robbins TW (2003) Global 5-HT depletion attenuates the ability of amphetamine to decrease impulsive choice on a delay-discounting task in rats. Psychopharmacology (Berl) 170:320–331
Winstanley CA, Dalley JW, Theobald DE, Robbins TW (2004) Fractionating impulsivity: contrasting effects of central 5-HT depletion on different measures of impulsive behavior. Neuropsychopharmacology 29:1331–1343
Winstanley CA, Theobald DE, Dalley JW, Cardinal RN, Robbins TW (2006) Double dissociation between serotonergic and dopaminergic modulation of medial prefrontal and orbitofrontal cortex during a test of impulsive choice. Cereb Cortex 16:106–114
Winstanley CA, Theobald DE, Dalley JW, Glennon JC, Robbins TW (2004) 5-HT2A and 5-HT2C receptor antagonists have opposing effects on a measure of impulsivity: interactions with global 5-HT depletion. Psychopharmacology (Berl) 176:376–385
Winstanley CA, Theobald DE, Dalley JW, Robbins TW (2005) Interactions between serotonin and dopamine in the control of impulsive choice in rats: therapeutic implications for impulse control disorders. Neuropsychopharmacology 30:669–682
Wogar MA, Bradshaw CM, Szabadi E (1993) Effect of lesions of the ascending 5-hydroxytryptaminergic pathways on choice between delayed reinforcers. Psychopharmacology (Berl) 111:239–243
Miyazaki KW, Miyazaki K, Doya K (2011) Activation of central serotonergic system during work for delayed rewards. Eur J Neurosci 33:153–160
Miyazaki K, Miyazaki KW, Doya K (2011) Activation of dorsal raphe serotonin neurons underlies waiting for delayed rewards. J Neurosci 31:469–479
Boureau YL, Dayan P (2011) Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology 36:74–97
Cools R, Nakamura K, Daw ND (2011) Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology 36:98–113
Dayan P, Huys QJ (2009) Serotonin in affective control. Annu Rev Neurosci 32:95–126
Soubrié P (1986) Reconciling the role of central serotonin neurons in human and animal behavior. Behav Brain Sci 9:319–364
Bari A, Dalley JW, Robbins TW (2008) The application of the 5-choice serial reaction time task for the assessment of visual attentional processes and impulse control in rats. Nat Protoc 3:759–767
Carli M, Robbins TW, Evenden JL, Everitt BJ (1983) Effects of lesions to ascending noradrenergic neurones on performance of a 5-choice serial reaction task in rats; implications for theories of dorsal noradrenergic bundle function based on selective attention and arousal. Behav Brain Res 9:361–380
Robbins TW (2002) The 5-choice serial reaction time task: behavioural pharmacology and functional neurochemistry. Psychopharmacology (Berl) 163:362–380
Dalley JW, Theobald DE, Eagle DM, Passetti F, Robbins TW (2002) Deficits in impulse control associated with tonically-elevated serotonergic function in rat prefrontal cortex. Neuropsychopharmacology 26:716–728
Barnes NM, Sharp T (1999) A review of central 5-HT receptors and their function. Neuropharmacology 38:1083–1152
Jacobs BL, Fornal CA (1999) Activity of serotonergic neurons in behaving animals. Neuropsychopharmacology 21:9S–15S
Heym J, Trulson ME, Jacobs BL (1982) Raphe unit activity in freely moving cats: effects of phasic auditory and visual stimuli. Brain Res 232:29–39
Ranade SP, Mainen ZF (2009) Transient firing of dorsal raphe neurons encodes diverse and specific sensory, motor and reward events. J Neurophysiol 102:3026–3037
Waterhouse BD, Devilbiss D, Seiple S, Markowitz R (2004) Sensorimotor-related discharge of simultaneously recorded, single neurons in the dorsal raphe nucleus of the awake, unrestrained rat. Brain Res 1000:183–191
Fornal CA, Metzler CW, Marrosu F, Ribiero-do-Valle LE, Jacobs BL (1996) A subgroup of dorsal raphe serotonergic neurons in the cat is strongly activated during oral–buccal movements. Brain Res 716:123–133
Nakamura K, Matsumoto M, Hikosaka O (2008) Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus. J Neurosci 28:5331–5343
Bromberg-Martin ES, Hikosaka O, Nakamura K (2010) Coding of task reward value in the dorsal raphe nucleus. J Neurosci 30:6262–6272
Doya K (2002) Metalearning and neuromodulation. Neural Netw 15:495–506
Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, Yamawaki S (2004) Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci 7:887–893
Schweighofer N, Bertin M, Shishida K, Okamoto Y, Tanaka SC, Yamawaki S, Doya K (2008) Low-serotonin levels increase delayed reward discounting in humans. J Neurosci 28:4528–4532
Crockett MJ, Clark L, Robbins TW (2009) Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J Neurosci 29:11993–11999
Schachar R, Logan GD, Robaey P, Chen S, Ickowicz A, Barr C (2007) Restraint and cancellation: multiple inhibition deficits in attention deficit hyperactivity disorder. J Abnorm Child Psychol 35:229–238
Chamberlain SR, Muller U, Blackwell AD, Clark L, Robbins TW, Sahakian BJ (2006) Neurochemical modulation of response inhibition and probabilistic learning in humans. Science 311:861–863
Clark L, Roiser JP, Cools R, Rubinsztein DC, Sahakian BJ, Robbins TW (2005) Stop signal response inhibition is not modulated by tryptophan depletion or the serotonin transporter polymorphism in healthy volunteers: implications for the 5-HT theory of impulsivity. Psychopharmacology (Berl) 182:570–578
Eagle DM, Bari A, Robbins TW (2008) The neuropsychopharmacology of action inhibition: cross-species translation of the stop-signal and go/no-go tasks. Psychopharmacology (Berl) 199:439–456
Bari A, Eagle DM, Mar AC, Robinson ES, Robbins TW (2009) Dissociable effects of noradrenaline, dopamine, and serotonin uptake blockade on stop task performance in rats. Psychopharmacology (Berl) 205:273–283
Eagle DM, Lehmann O, Theobald DE, Pena Y, Zakaria R, Ghosh R, Dalley JW, Robbins TW (2009) Serotonin depletion impairs waiting but not stop-signal reaction time in rats: implications for theories of the role of 5-HT in behavioral inhibition. Neuropsychopharmacology 34:1311–1321
Harrison AA, Everitt BJ, Robbins TW (1999) Central serotonin depletion impairs both the acquisition and performance of a symmetrically reinforced go/no-go conditional visual discrimination. Behav Brain Res 100:99–112
Masaki D, Yokoyama C, Kinoshita S, Tsuchida H, Nakatomi Y, Yoshimoto K, Fukui K (2006) Relationship between limbic and cortical 5-HT neurotransmission and acquisition and reversal learning in a go/no-go task in rats. Psychopharmacology (Berl) 189:249–258
Fletcher PJ (1995) Effects of combined or separate 5,7-dihydroxytryptamine lesions of the dorsal and median raphe nuclei on responding maintained by a DRL 20s schedule of food reinforcement. Brain Res 675:45–54
Christakou A, Robbins TW, Everitt BJ (2004) Prefrontal cortical–ventral striatal interactions involved in affective modulation of attentional performance: implications for corticostriatal circuit function. J Neurosci 24:773–780
Pothuizen HH, Jongen-Relo AL, Feldon J, Yee BK (2005) Double dissociation of the effects of selective nucleus accumbens core and shell lesions on impulsive-choice behaviour and salience learning in rats. Eur J Neurosci 22:2605–2616
Robinson ES, Dalley JW, Theobald DE, Glennon JC, Pezze MA, Murphy ER, Robbins TW (2008) Opposing roles for 5-HT2A and 5-HT2C receptors in the nucleus accumbens on inhibitory response control in the 5-choice serial reaction time task. Neuropsychopharmacology 33:2398–2406
Khamassi M, Mulder AB, Tabuchi E, Douchamps V, Wiener SI (2008) Anticipatory reward signals in ventral striatal neurons of behaving rats. Eur J Neurosci 28:1849–1866
Miyazaki K, Mogi E, Araki N, Matsumoto G (1998) Reward-quality dependent anticipation in rat nucleus accumbens. Neuroreport 9:3943–3948
Chudasama Y, Passetti F, Rhodes SE, Lopian D, Desai A, Robbins TW (2003) Dissociable aspects of performance on the 5-choice serial reaction time task following lesions of the dorsal anterior cingulate, infralimbic and orbitofrontal cortex in the rat: differential effects on selectivity, impulsivity and compulsivity. Behav Brain Res 146:105–119
Carli M, Baviera M, Invernizzi RW, Balducci C (2006) Dissociable contribution of 5-HT1A and 5-HT2A receptors in the medial prefrontal cortex to different aspects of executive control such as impulsivity and compulsive perseveration in rats. Neuropsychopharmacology 31:757–767
Ceglia I, Carli M, Baviera M, Renoldi G, Calcagno E, Invernizzi RW (2004) The 5-HT receptor antagonist M100,907 prevents extracellular glutamate rising in response to NMDA receptor blockade in the mPFC. J Neurochem 91:189–199
Mirjana C, Baviera M, Invernizzi RW, Balducci C (2004) The serotonin 5-HT2A receptors antagonist M100907 prevents impairment in attentional performance by NMDA receptor blockade in the rat prefrontal cortex. Neuropsychopharmacology 29:1637–1647
Calcagno E, Carli M, Baviera M, Invernizzi RW (2009) Endogenous serotonin and serotonin2C receptors are involved in the ability of M100907 to suppress cortical glutamate release induced by NMDA receptor blockade. J Neurochem 108:521–532
Liu S, Bubar MJ, Lanfranco MF, Hillman GR, Cunningham KA (2007) Serotonin2C receptor localization in GABA neurons of the rat medial prefrontal cortex: implications for understanding the neurobiology of addiction. Neuroscience 146:1677–1688
Roesch MR, Taylor AR, Schoenbaum G (2006) Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51:509–520
Kalenscher T, Windmann S, Diekamp B, Rose J, Gunturkun O, Colombo M (2005) Single units in the pigeon brain integrate reward amount and time-to-reward in an impulsive choice task. Curr Biol 15:594–602
Schoenbaum G, Roesch MR, Stalnaker TA, Takahashi YK (2009) A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat Rev Neurosci 10:885–892
Miyazaki K, Miyazaki KW, Matsumoto G (2004) Different representation of forthcoming reward in nucleus accumbens and medial prefrontal cortex. Neuroreport 15:721–726
Kepecs A, Uchida N, Zariwala HA, Mainen ZF (2008) Neural correlates, computation and behavioural impact of decision confidence. Nature 455:227–231
Winstanley CA, Theobald DE, Cardinal RN, Robbins TW (2004) Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J Neurosci 24:4718–4722
We thank Dr. Makoto Ito and the members of the neural computation unit for their helpful comments and discussion regarding the role of the 5-HT system in the waiting to obtain reward behavior. A part of this study is the result of “Integrated research on neuropsychiatric disorders” carried out under the Strategic Research Program for Brain Sciences by the Ministry of Education, Culture, Sports, Science and Technology of Japan.
This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
About this article
Cite this article
Miyazaki, K., Miyazaki, K.W. & Doya, K. The Role of Serotonin in the Regulation of Patience and Impulsivity. Mol Neurobiol 45, 213–224 (2012). https://doi.org/10.1007/s12035-012-8232-6
- Delayed reward
- Dorsal raphe nucleus
- Waiting to obtain reward