The push and pull of dopamine in cue-reward learning

Summary

A recent study by Saunders, Richard, Margolis, and Janak (2018) shows that there is a great deal left to learn about what different mesotelencephalic dopamine circuits contribute to learning about the motivational significance of reward-related cues. Their findings suggest that nigrostriatal and mesolimbic dopamine pathways support distinct reinforcement processes that independently push and pull animals toward their goals.

Cues that signal reward acquire powerful motivational properties that allow them to demand attention and promote vigorous reward-seeking behavior. Rewards are known to trigger phasic dopamine neuron activity, which may serve as a reinforcement signal for Pavlovian motivational learning (Berridge, 2012). Research on this topic has typically involved manipulating or measuring dopamine system function in animals as they receive pairings between a cue and a natural reward stimulus, like palatable food. While such studies demonstrate the importance of dopamine in cue-reward learning, they do not address whether dopamine is the main driver of this process or whether it plays a supporting role, modulating the efficacy of other reward-elicited neural processes more directly responsible for assigning motivational value to cues.

A new study by Saunders, Richard, Margolis, and Janak (2018) approached this problem in an innovative way by asking if simply stimulating dopamine neurons—without reward delivery—is sufficient to reinforce Pavlovian learning. Rats were given repeated pairings between an audiovisual cue (7 s) and phasic, optogenetic stimulation of midbrain dopamine neurons (5 s, coterminating with cue). If reward-elicited dopamine neuron firing serves as the crucial reinforcement signal for cue-reward learning, then pairing a neutral cue with artificial dopamine neuron stimulation (without other reward-related neural activity) should be sufficient to assign that cue motivational properties.

In line with this prediction, Saunders et al. found that rats developed conditioned locomotor responses to cues paired with dopamine neuron activation, regardless of whether those neurons were located in the ventral tegmental area (VTA) or substantia nigra pars compacta (SNc). Moreover, fine-grained analyses revealed that these two dopamine circuits supported different forms of conditioned responding. Rats tended to approach cues associated with VTA dopamine neuron stimulation. This sign-tracking response is also elicited by food-paired cues and is used as a selective assay of Pavlovian incentive motivation (Robinson & Flagel, 2009). Rats also readily pressed a lever to produce cues that had been paired with VTA dopamine neuron stimulation (conditioned reinforcement), another common assay of Pavlovian incentive motivation (Robinson & Flagel, 2009). Therefore, phasic activation of VTA dopamine neurons was sufficient to attribute incentive value to associated cues, at least by these measures. In contrast, cues paired with SNc dopamine neuron stimulation did not support sign-tracking or conditioned reinforcement, but instead caused rats to turn in circles directed away from the stimulated hemisphere. Moreover, the authors found that conditioned responses elicited by cues paired with SNc dopamine neuron stimulation were considerably more vigorous (i.e., faster) that those elicited by cues that signaled VTA dopamine neuron stimulation, providing further evidence that these different dopamine circuits support distinct behavioral functions.

Saunders et al. also showed that the learning produced by stimulating dopamine neurons depends on both the location of those neurons and where they send their projections. Cues paired with stimulation of VTA dopamine neurons that innervate the nucleus accumbens core supported sign-tracking and conditioned reinforcement. This was not the case for cues paired with stimulation of a distinct group of VTA dopamine neurons that project to the nucleus accumbens shell or for cues paired with stimulation of SNc dopamine neurons projecting to the dorsal striatum. However, activating this projection-defined nigrostriatal dopamine circuit was sufficient to establish vigorous conditioned turning behavior, replicating the effects of less specific SNc dopamine neuron stimulation described above.

Based on such findings, the authors posit that phasic activity within the VTA ➔ core dopamine pathway is responsible for assigning incentive motivational properties to neutral cues—making them attractive and worthy of pursuit, whereas the SNc ➔ dorsal striatum dopamine pathway mediates a separate form of motivational learning, one that promotes vigorous movement without setting a specific direction. Put another way, these dopamine circuits may motivate behavior by separately pulling (mesolimbic) and pushing (nigrostriatal) animals toward their goals. This maps on nicely to a long-standing theoretical distinction between incentive motivation, on the one hand, and drive (or the related concept of general motivational arousal), on the other, which can be viewed as serving similar pull-and-push functions, respectively (Bolles, 1967).

The hypothesis that motivational learning is fractionated in this manner within the dopamine system certainly deserves further investigation. One way forward is to probe the ability for dopamine-paired cues to invigorate performance of an independently trained instrumental action. This Pavlovian-to-instrumental transfer effect provides an index of the nonspecific response-invigorating influence of reward-paired cues, which is known to depend on dopamine signaling (Ostlund & Maidment, 2011). The hypothesis stated above predicts that cues paired with SNc dopamine neuron stimulation should elicit more vigorous instrumental responding than cues paired with VTA dopamine neuron stimulation.

The notion that the nigrostriatal dopamine pathway modulates response vigor generally fits with the role this system plays in controlling the expression of voluntary movement and how dysfunction within this system produces motor slowing (bradykinesia) in Parkinson’s disease. However, Saunders et al. address a fundamentally different question. Rather than focusing on expression of movement, they show that activating the nigrostriatal pathway can trigger a learning process that allows previously neutral cues to arouse movement. The authors speculate that the resulting motivational state may play an important role in regulating expression of voluntary behaviors, establishing cues that facilitate responding and may therefore help overcome motor deficits. This is a ripe topic for future research.

Other findings in this study raise questions about dopamine's relative contributions to learning versus performance. Saunders et al. focused almost exclusively on learning, showing that phasic dopamine neuron activity is sufficient to establish new conditioned responding. They also applied fiber photometry to reveal that cues paired with dopamine neuron stimulation come to elicit their own conditioned dopamine neuron activity. Although the inference here is that these cue-elicited dopamine responses may contribute to the expression of cue-elicited behavior, this remains to be established. However, it is notable that the authors found that directly stimulating VTA or SNc dopamine neurons (in the absence of cues) was not sufficient to elicit unconditioned locomotor responses. This suggests that if cue-elicited dopamine neuron activity contributes to the expression of the conditioned responses described above, it likely plays a supporting role, acting in concert with other cue-elicited neural processes.

Interestingly, Saunders et al. found that dopamine neuron activity had characteristics resembling a reward-prediction error signal, in that omitting an expected train of optogenetic stimulation led to a dip in dopamine neuron activity, relative to baseline levels. This fits with the view that phasic dopamine signaling subserves a reinforcement learning process (cf. Berridge, 2012), though the nature of the learning supported by this process is not clear. Saunders et al. provide good evidence that the mesolimbic dopamine system supports Pavlovian incentive learning, but also show that rats will lever press for either VTA or SNc dopamine neuron stimulation. Thus, it is possible that this response-reinforcement process may have also contributed to the development of cue-elicited behavior. While cues were paired with dopamine neuron stimulation in a response-independent (Pavlovian) manner, this would not prevent that stimulation from reinforcing actions that just happened to be performed during cues. This may have helped strengthening or amplify response tendencies established through Pavlovian conditioning, which may otherwise have been relatively weak. Unconditioned orienting responses (approach) directed at the cue light may have also been unintentionally paired with dopamine neuron stimulation, contributing to the development of conditioned sign-tracking in rats receiving VTA dopamine neuron stimulation. However, this would not explain why cues paired with SNc dopamine neuron stimulation came to elicit conditioned turning behavior. Thus, while the degree to which response-reinforcement processes contributed to these conditioned responses remains unclear, it seems likely that these behaviors were predominantly established through a Pavlovian reinforcement process.

In summary, Saunders et al. provide compelling evidence that activating dopamine neurons is sufficient for assigning motivational significance to cues and that the nature of this learning is pathway specific. This study also demonstrates the unique insights that can be gained by combining cutting-edge neuroscience techniques with sophisticated behavioral analysis. And, like with any good study, it raises more questions than it answers.

References

  1. Berridge, K. C. (2012). From prediction error to incentive salience: Mesolimbic computation of reward motivation. European Journal of Neuroscience, 35(7), 1124–1143. https://doi.org/10.1111/j.1460-9568.2012.07990.x

    Article  PubMed  Google Scholar 

  2. Bolles, R. C. (1967). Theory of motivation. New York, NY: Harper & Row.

    Google Scholar 

  3. Ostlund, S. B., & Maidment, N. T. (2011). Dopamine receptor blockade attenuates the general incentive motivational effects of noncontingently delivered rewards and reward-paired cues without affecting their ability to bias action selection. Neuropsychopharmacology, 37, 508. https://doi.org/10.1038/npp.2011.217

    Article  PubMed  PubMed Central  Google Scholar 

  4. Robinson, T. E., & Flagel, S. B. (2009). Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences. Biological Psychiatry, 65(10), 869–873. https://doi.org/10.1016/j.biopsych.2008.09.006

    Article  PubMed  Google Scholar 

  5. Saunders, B. T., Richard, J. M., Margolis, E. B., & Janak, P. H. (2018). Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties. Nature Neuroscience, 21(8), 1072–1083. https://doi.org/10.1038/s41593-018-0191-4

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sean B. Ostlund.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ostlund, S.B. The push and pull of dopamine in cue-reward learning. Learn Behav 47, 273–274 (2019). https://doi.org/10.3758/s13420-018-0370-x

Download citation

Keywords

  • Incentive motivation
  • General arousal
  • Optogenetics
  • Reinforcement
  • Pavlovian