Abstract
The role of dopamine has been strongly implicated in reward processes, but recent work shows an additional role as a signal that promotes the stable incorporation of novel information into long-term hippocampal memory. Indeed, dopamine neurons, in addition to being activated by reward, can be activated by novelty in the absence of reward. The computation of novelty is thought to occur in the hippocampus and is carried to the dopamine cells of the VTA through a polysynaptic pathway. Although a picture of novelty-dependent processes in the VTA and hippocampus is beginning to emerge, many aspects of the process remain unclear. Here, we will consider several issues: (1) What is relationship of novelty signals coming to the VTA from the superior colliculus, as compared to those that come from the hippocampus? (2) Can dopamine released by a reward enhance the learning of novel information? (3) Is there an interaction between motivational signals and hippocampal novelty signals? (4) What are the properties of the axons that generate dopamine release in the hippocampus? We close with a discussion of some of the outstanding open issues in this field.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adcock, R., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., Gabrieli, J.: Reward-motivated learning: Mesolimbic activation precedes memory formation. Neuron 50, 507–517 (2006)
Arsenault, M., Parent, A., Seguela, P., Descarries, L.: Distribution and morphological characteristics of dopamine-immunoreactive neurons in the midbrain of the squirrel monkey (saimiri sciureus). J. Comp. Neurol. 267(4), 489–506 (1988)
Ballarini, F., Moncada, D., Martinez, M., Alen, N., Viola, H.: Behavioral tagging is a general mechanism of long-term memory formation. Proc. Natl. Acad. Sci. U. S. A. 106(34), 14599–604 (2009)
Bethus, I., Tse, D., Morris, R.: Dopamine and memory: Modulation of the persistence of memory for novel hippocampal nmda receptor-dependent paired associates. J. Neurosci. 30(5), 1610–1618 (2010)
Borgkvist, A., Malmlof, T., Feltmann, K., Lindskog, M., Schilstrom, B.: Dopamine in the hippocampus is cleared by the norepinephrine transporter. Int. J. Neuropsychopharmacol. 1–10 (2011)
Bromberg-Martin, E., Matsumoto, M., Hikosaka, O.: Distinct tonic and phasic anticipatory activity in lateral habenula and dopamine neurons. Neuron 67(1), 144–155 (2010a)
Bromberg-Martin, E., Matsumoto, M., Hikosaka, O.: Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron 68(5), 815–834 (2010b)
Bromberg-Martin, E., Matsumoto, M., Nakahara, H., Hikosaka, O.: Multiple timescales of memory in lateral habenula and dopamine neurons. Neuron 67(3), 499–510 (2010c)
Brown, M., Henny, P., Bolam, J., Magill, P.: Activity of neurochemically heterogeneous dopaminergic neurons in the substantia nigra during spontaneous and driven changes in brain state. J. Neurosci. 29(9), 2915–2925 (2009)
Bunzeck, N., Doeller, C., Fuentemilla, L., Dolan, R., Duzel, E.: Reward motivation accelerates the onset of neural novelty signals in humans to 85 milliseconds. Curr. Biol. 19(15), 1294–1300 (2009)
Bunzeck, N., Duzel, E.: Absolute coding of stimulus novelty in the human substantia nigra/vta. Neuron 51(3), 369–379 (2006)
Bunzeck, N., Schutze, H., Stallforth, S., Kaufmann, J., Duzel, S., Heinze, H., Duzel, E.: Mesolimbic novelty processing in older adults. Cereb. Cortex 17(12), 2940–2948 (2007)
Cheng, S., Frank, L.: New experiences enhance coordinated neural activity in the hippocampus. Neuron 57(2), 303–313 (2008)
Coulter, C., Happe, H., Bergman, D., Murrin, L.C.: Localization and quantification of the dopamine transporter: Comparison of [3h]win 35,428 and [125i]rti-55. Brain Res. 690(2), 217–224 (1995)
Cragg, S., Nicholson, C., Kume-Kick, J., Tao, L., Rice, M.: Dopamine-mediated volume transmission in midbrain is regulated by distinct extracellular geometry and uptake. J. Neurophysiol. 85(4), 1761–1771 (2001)
Csicsvari, J., O’Neill, J., Allen, K., Senior, T.: Place-selective firing contributes to the reverse-order reactivation of ca1 pyramidal cells during sharp waves in open-field exploration. Eur. J. Neurosci. 26(3), 704–716 (2007)
Curet, O., Dennis, T., Scatton, B.: The formation of deaminated metabolites of dopamine in the locus coeruleus depends upon noradrenergic neuronal activity. Brain Res. 335(2), 297–301 (1985)
Deadwyler, S., Hampson, R.: Differential but complementary mnemonic functions of the hippocampus and subiculum. Neuron 42(3), 465–476 (2004)
Deutch, A., Goldstein, M., Roth, R.: Activation of the locus coeruleus induced by selective stimulation of the ventral tegmental area. Brain Res. 363(2), 307–314 (1986)
Devoto, P., Flore, G., Pani, L., Gessa, G.: Evidence for co-release of noradrenaline and dopamine from noradrenergic neurons in the cerebral cortex. Mol. Psychiatry 6(6), 657–664 (2001)
Devoto, P., Flore, G., Saba, P., Fa, M., Gessa, G.: Co-release of noradrenaline and dopamine in the cerebral cortex elicited by single train and repeated train stimulation of the locus coeruleus. BMC Neurosci. 6(31) (2005)
Duzel, E., Bunzeck, N., Guitart-Masip, M., Duzel, S.: Novelty-related motivation of anticipation and exploration by dopamine (nomad): Implications for healthy aging. Neurosci. Biobehav. Rev. 34(5), 660–669 (2010)
Duzel, E., Guitart-Masip, M., Weiskopf, N., Kanowski, M.: The fMRI Book, chapter Midbrain fMRI: Applications, Limitations and Challenges (2012, in revision)
Fenker, D., Frey, J., Schuetze, H., Heipertz, D., Heinze, H., Duzel, E.: Novel scenes improve recollection and recall of words. J. Cogn. Neurosci. 20(7), 1250–1265 (2008)
Frey, U., Matthies, H., Reymann, K.: The effect of dopaminergic d1 receptor blockade during tetanization on the expression of long-term potentiation in the rat ca1 region in vitro. Neurosci. Lett. 129(1), 111–114 (1991)
Frey, U., Morris, R.: Synaptic tagging: Implications for late maintenance of hippocampal long-term potentiation. Trends Neurosci. 21(5), 181–188 (1998)
Frey, U., Schroeder, H., Matthies, H.: Dopaminergic antagonists prevent long-term maintenance of posttetanic ltp in the ca1 region of rat hippocampal slices. Brain Res. 522(1), 69–75 (1990)
Fuxe, K., Hokfelt, T., Johansson, O., Jonsson, G., Lidbrink, P., Ljungdahl, A.: The origin of the dopamine nerve terminals in limbic and frontal cortex. evidence for meso-cortico dopamine neurons. Brain Res. 82(2), 349–355 (1974)
Fyhn, M., Molden, S., Hollup, S., Moser, M., Moser, E.: Hippocampal neurons responding to first-time dislocation of a target object. Neuron 35(3), 555–566 (2002)
Gasbarri, A., Campana, E., Pacitti, C., Hajdu, F., Tombol, T.: Organization of the projections from the ventral tegmental area of tsai to the hippocampal formation in the rat. J. Hirnforsch. 32(4), 429–437 (1991)
Gasbarri, A., M.G., P., Campana, E., Pacitti, C.: Anterograde and retrograde tracing of projections from the ventral tegmental area to the hippocampal formation in the rat. Brain Res. Bull. 33(4), 445–452 (1994a)
Gasbarri, A., Packard, M., Sulli, A., Pacitti, C., Innocenzi, R., Perciavalle, V.: The projections of the retrorubral field a8 to the hippocampal formation in the rat. Exp. Brain Res. 112(2), 244–252 (1996)
Gasbarri, A., Sulli, A., Packard, M.: The dopaminergic mesencephalic projections to the hippocampal formation in the rat. Prog. Neuropsychopharmacol. Biol. Psychiatry 21(1), 1–22 (1997)
Gasbarri, A., Verney, C., Innocenzi, R., Campana, E., Pacitti, C.: Mesolimbic dopaminergic neurons innervating the hippocampal formation in the rat: A combined retrograde tracing and immunohistochemical study. Brain Res. 668(1–2), 71–79 (1994b)
Geffard, M., Buijs, R., Seguela, P., Pool, C., Le Moal, M.: First demonstration of highly specific and sensitive antibodies against dopamine. Brain Res. 294(1), 161–165 (1984a)
Geffard, M., Kah, O., Onteniente, B., Seguela, P., Le Moal, M., Delaage, M.: Antibodies to dopamine: Radioimmunological study of specificity in relation to immunocytochemistry. J. Neurochem. 42(6), 1593–1599 (1984b)
Geffard, M., Seguela, P., Heinrich-Rock, A.: Antisera against catecholamines: Specificity studies and physicochemical data for anti-dopamine and anti-p-tyramine antibodies. Mol. Immunol. 21(6), 515–522 (1984c)
Giese, K., Fedorov, N., Filipkowski, R., Silva, A.: Autophosphorylation at thr286 of the alpha calcium-calmodulin kinase ii in ltp and learning. Science 279(5352), 870–873 (1998)
Goldman-Rakic, P., Leranth, C., Williams, S., Mons, N., Geffard, M.: Dopamine synaptic complex with pyramidal neurons in primate cerebral cortex. Proc. Natl. Acad. Sci. U. S. A. 86(22), 9015–9019 (1989)
Guitart-Masip, M., Bunzeck, N., Stephan, K., Dolan, R., Duzel, E.: Contextual novelty changes reward representations in the striatum. J. Neurosci. 30(5), 1721–1726 (2010)
Guitart-Masip, M., Fuentemilla, L., Bach, D., Huys, Q., Dayan, P., Dolan, R., Duzel, E.: Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain. J. Neurosci. 31(21), 7867–7875 (2011)
Hokfelt, T., Fuxe, K., Johansson, O., Ljungdahl, A.: Pharmaco-histochemical evidence of the existence of dopamine nerve terminals in the limbic cortex. Eur. J. Pharmacol. 25(1), 108–112 (1974a)
Hokfelt, T., Ljungdahl, A., Fuxe, K., Johansson, O.: Dopamine nerve terminals in the rat limbic cortex: Aspects of the dopamine hypothesis of schizophrenia. Science 184(133), 177–179 (1974b)
Huang, Y., Kandel, E.: D1/d5 receptor agonists induce a protein synthesis-dependent late potentiation in the ca1 region of the hippocampus. Proc. Natl. Acad. Sci. U. S. A. 92(7), 2446–2450 (1995)
Ishikawa, K., Ott, T., McGaugh, J.: Evidence for dopamine as a transmitter in dorsal hippocampus. Brain Res. 232(1), 222–226 (1982)
Kakade, S., Dayan, P.: Dopamine: Generalization and bonuses. Neural Netw. 15(4–6), 549–559 (2002)
Karlsson, M., Frank, L.: Network dynamics underlying the formation of sparse, informative representations in the hippocampus. J. Neurosci. 28(52), 14271–14281 (2008)
Krebs, R., Heipertz, D., Schuetze, H., Duzel, E.: Novelty increases the mesolimbic functional connectivity of the substantia nigra/ventral tegmental area (sn/vta) during reward anticipation: Evidence from high-resolution fmri. NeuroImage 58(2), 647–655 (2011)
Krebs, R., Schott, B., Duzel, E.: Personality traits are differentially associated with patterns of reward and novelty processing in the human substantia nigra/ventral tegmental area. Biol. Psychiatry 65(2), 103–110 (2009)
Kwon, O., Paredes, D., Gonzalez, C., Neddens, J., Hernandez, L., Vullhorst, D., Buonanno, A.: Neuregulin-1 regulates ltp at ca1 hippocampal synapses through activation of dopamine d4 receptors. Proc. Natl. Acad. Sci. U. S. A. 105(40), 15587–1592 (2008)
Legault, M., Wise, R.: Novelty-evoked elevations of nucleus accumbens dopamine: Dependence on impulse flow from the ventral subiculum and glutamatergic neurotransmission in the ventral tegmental area. Eur. J. Neurosci. 13(4), 819–828 (2001)
Lemon, N., Manahan-Vaughan, D.: Dopamine d1/d5 receptors gate the acquisition of novel information through hippocampal long-term potentiation and long-term depression. J. Neurosci. 26(29), 7723–7729 (2006)
Li, S., Cullen, W., Anwyl, R., Rowan, M.: Dopamine-dependent facilitation of ltp induction in hippocampal ca1 by exposure to spatial novelty. Nat. Neurosci. 6(5), 526–531 (2003)
Lisman, J., Grace, A.: The hippocampal-vta loop: Controlling the entry of information into long-term memory. Neuron 46(5), 703–713 (2005)
Lisman, J., Grace, A., Duzel, E.: A neohebbian framework for episodic memory; role of dopamine-dependent late ltp. Trends Neurosci. 34(10), 536–547 (2011)
Ljungberg, T., Apicella, P., Schultz, W.: Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 67(1), 145–163 (1992)
Luo, A., Tahsili-Fahadan, P., Wise, R., Lupica, C., Aston-Jones, G.: Linking context with reward: A functional circuit from hippocampal ca3 to ventral tegmental area. Science 333(6040), 353–357 (2011)
Martig, A., Mizumori, S.: Ventral tegmental area and substantia nigra neural correlates of spatial learning. Learn. Mem. 18(4), 260–271 (2011)
Mirolli, M., Baldassarre, G.: Functions and mechanisms of intrinsic motivations: The knowledge versus competence distinction. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 49–72. Springer, Berlin (2012)
Navakkode, S., Sajikumar, S., Frey, J.: Synergistic requirements for the induction of dopaminergic d1/d5-receptor-mediated ltp in hippocampal slices of rat ca1 in vitro. Neuropharmacology 52(7), 1547–1554 (2007)
Nehmzow, U., Gatsoulis, Y., Kerr, E., Condell, J., Siddique, N.H., McGinnity, M.T.: Novelty detection as an intrinsic motivation for cumulative learning robots. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 185–207. Springer, Berlin (2012)
Neugebauer, F., Korz, V., Frey, J.: Modulation of extracellular monoamine transmitter concentrations in the hippocampus after weak and strong tetanization of the perforant path in freely moving rats. Brain Res. 1273, 29–38 (2009)
Nitz, D., McNaughton, B.: Differential modulation of ca1 and dentate gyrus interneurons during exploration of novel environments. J. Neurophysiol. 91(2), 863–872 (2004)
Oades, R., Halliday, G.: Ventral tegmental (a10) system: Neurobiology. 1. anatomy and connectivity. Brain Res. 434(2), 117–165 (1987)
Onteniente, B., Geffard, M., Calas, A.: Ultrastructural immunocytochemical study of the dopaminergic innervation of the rat lateral septum with anti-dopamine antibodies. Neuroscience 13(2), 385–393 (1984)
Otmakhova, N., Lisman, J.: D1/d5 dopamine receptor activation increases the magnitude of early long-term potentiation at ca1 hippocampal synapses. J. Neurosci. 16(23), 7478–7486 (1996)
Pacholczyk, T., Blakely, R., Amara, S.: Expression cloning of a cocaine- and antidepressant-sensitive human noradrenaline transporter. Nature 350(6316), 350–354 (1991)
Quintin, L., Hilaire, G., Pujol, J.: Variations in 3,4-dihydroxyphenylacetic acid concentration are correlated to single cell firing changes in the rat locus coeruleus. Neuroscience 18(4), 889–899 (1986)
Redgrave, P., Gurney, K.: The short-latency dopamine signal: A role in discovering novel actions? Nat. Rev. Neurosci. 7(12), 967–975 (2006)
Redondo, R., Morris, R.: Making memories last: The synaptic tagging and capture hypothesis. Nat. Rev. Neurosci. 12(1), 17–30 (2011)
Rossato, J., Bevilaqua, L., Izquierdo, I., Medina, J., Cammarota, M.: Dopamine controls persistence of long-term memory storage. Science 325(5943), 1017–1020 (2009)
Sawaguchi, T., Goldman-Rakic, P.: The role of d1-dopamine receptor in working memory: Local injections of dopamine antagonists into the prefrontal cortex of rhesus monkeys performing an oculomotor delayed-response task. J. Neurophysiol. 71(2), 515–528 (1994)
Scatton, B., Dennis, T., Curet, O.: Increase in dopamine and dopac levels in noradrenergic terminals after electrical stimulation of the ascending noradrenergic pathways. Brain Res. 298(1), 193–196 (1984)
Scatton, B., Javoy-Agid, F., Rouquier, L., Dubois, B., Agid, Y.: Reduction of cortical dopamine, noradrenaline, serotonin and their metabolites in parkinson’s disease. Brain Res. 275(2), 321–328 (1983)
Scatton, B., Rouquier, L., Javoy-Agid, F., Agid, Y.: Dopamine deficiency in the cerebral cortex in parkinson disease. Neurology 32(9), 1039–1040 (1982)
Scatton, B., Simon, H., Le Moal, M., Bischoff, S.: Origin of dopaminergic innervation of the rat hippocampal formation. Neurosci. Lett. 18(2), 125–131 (1980)
Schott, B., Seidenbecher, C., Fenker, D., Lauer, C., Bunzeck, N., Bernstein, H., Tischmeyer, W., Gundelfinger, E., Heinze, H., Duzel, E.: The dopaminergic midbrain participates in human episodic memory formation: Evidence from genetic imaging. J. Neurosci. 26(5), 1407–1417 (2006)
Schroeter, S., Apparsundaram, S., Wiley, R., Miner, L., Sesack, S., Blakely, R.: Immunolocalization of the cocaine- and antidepressant-sensitive l-norepinephrine transporter. J. Comp. Neurol. 420(2), 211–232 (2000)
Schultz, W.: Behavioral dopamine signals. Trends Neurosci. 30(5), 203–210 (2007a)
Schultz, W.: Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 30, 259–288 (2007b)
Schultz, W., Apicella, P., Ljungberg, T.: Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13(3), 900–913 (1993)
Schwab, M., Javoy-Agid, F., Agid, Y.: Labeled wheat germ agglutinin (wga) as a new, highly sensitive retrograde tracer in the rat brain hippocampal system. Brain Res. 152(1), 145–150 (1978)
Shohamy, D., Wagner, A.: Integrating memories in the human brain: Hippocampal-midbrain encoding of overlapping events. Neuron 60(2), 378–389 (2008)
Simon, H., Le Moal, M., Calas, A.: Efferents and afferents of the ventral tegmental-a10 region studied after local injection of [3h]leucine and horseradish peroxidase. Brain Res. 178(1), 17–40 (1979)
Smith, C.C, Greene, R.W.: CNS dopamine transmission mediated by noradrenergic innervation. J. Neurosci. 32(18), 6072–6080 (2012)
Sountsov, P., Santucci, D.M., Lisman, J.: A biologically plausible transform for visual recognition that is invariant to translation, scale, and rotation. Front. Comput. Neurosci. 5(53) (2011)
Swanson, L.: The projections of the ventral tegmental area and adjacent regions: A combined fluorescent retrograde tracer and immunofluorescence study in the rat. Brain Res. Bull. 9(1–6), 321–53 (1982)
Swanson, L., Hartman, B.: The central adrenergic system. an immunofluorescence study of the location of cell bodies and their efferent connections in the rat utilizing dopamine-beta-hydroxylase as a marker. J. Comp. Neurol. 163(4), 467–505 (1975)
Swanson, L., Kohler, C., Bjorklund, A.: The limbic region. I: The septohippocampal system. In: Bjorklund A, Hokfelt T.S.L. (eds.) Integrated Systems of the CNS, pp. 125–269. Elsevier, Amsterdam (1987)
Swant, J., Wagner, J.: Dopamine transporter blockade increases ltp in the ca1 region of the rat hippocampus via activation of the d3 dopamine receptor. Learn. Mem. 13(2), 161–167 (2006)
Van Eden, C., Hoorneman, E., Buijs, R., Matthijssen, M., Geffard, M., Uylings, H.: Immunocytochemical localization of dopamine in the prefrontal cortex of the rat at the light and electron microscopical level. Neuroscience 22(3), 849–862 (1987)
VanElzakker, M., Fevurly, R., Breindel, T., Spencer, R.: Environmental novelty is associated with a selective increase in fos expression in the output elements of the hippocampal formation and the perirhinal cortex. Learn. Mem. 15(12), 899–908 (2008)
Verney, C., Baulac, M., Berger, B., Alvarez, C., Vigny, A., Helle, K.: Morphological evidence for a dopaminergic terminal field in the hippocampal formation of young and adult rat. Neuroscience 14(4), 1039–1052 (1985)
Wang, S., Redondo, R., Morris, R.: Relevance of synaptic tagging and capture to the persistence of long-term potentiation and everyday spatial memory. Proc. Natl. Acad. Sci. U. S. A. 107, 19537–19542 (2010)
Whitlock, J., Heynen, A., Shuler, M., Bear, M.: Learning induces long-term potentiation in the hippocampus. Science 313(5790), 1093–1097 (2006)
Wittmann, B., Bunzeck, N., Dolan, R., Duzel, E.: Anticipation of novelty recruits reward system and hippocampus while promoting recollection. Neuroimage 38(1), 194–202 (2007)
Wittmann, B., Schott, B., Guderian, S., Frey, J., Heinze, H., Duzel, E.: Reward-related fmri activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation. Neuron 45(3), 459–467 (2005)
Wolosin, S., Liang, J., Zeithamova, D., Schmandt, N., Preston, A.: Reward modulation of hippocampal subregions during motivated associative encoding. Soc. Neurosci. (2009)
Wyss, J., Swanson, L., Cowan, W.: A study of subcortical afferents to the hippocampal formation in the rat. Neuroscience 4(4), 463–476 (1979)
Zoli, M., Torri, C., Ferrari, R., Jansson, A., Zini, I., Fuxe, K., Agnati, L.: The emergence of the volume transmission concept. Brain Res. Brain Res. Rev. 26(2–3), 136–47 (1998)
Acknowledgements
John Lisman gratefully acknowledges the support of NIH Grant N.2 P50 MH060450. Emrah Duzel has been supported by the Wellcome Trust (Project Grant to ED) and the DFG (KFO 163, SFB 776 TP A7).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Otmakhova, N., Duzel, E., Deutch, A.Y., Lisman, J. (2013). The Hippocampal-VTA Loop: The Role of Novelty and Motivation in Controlling the Entry of Information into Long-Term Memory. In: Baldassarre, G., Mirolli, M. (eds) Intrinsically Motivated Learning in Natural and Artificial Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32375-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-32375-1_10
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32374-4
Online ISBN: 978-3-642-32375-1
eBook Packages: Computer ScienceComputer Science (R0)