Neural Correlates of Anticipation in Cerebellum, Basal Ganglia, and Hippocampus

  • Jason G. Fleischer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4520)

Abstract

Animals anticipate the future in a variety of ways. For instance: (a) they make motor actions that are timed to a reference stimulus and motor actions that anticipate future movement dynamics; (b) they learn to make choices that will maximize reward they receive in the future; and (c) they form memories of behavioral episodes such that the animal’s future actions can be predicted by current neural activity associated with those memories. Although these effects are clearly observable at the behavioral level, research into the mechanisms of such anticipatory learning are still largely in the early stages. This review, intended for those who have a computational background and are less familiar with neuroscience, addresses neural mechanisms found in the mammalian cerebellum, basal ganglia, and the hippocampus that give rise to such adaptive anticipatory behavior.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aldridge, J.W., Berridge, K.C.: Coding of serial order by neostriatal neurons: a natural action approach to movement sequence. J Neurosci 18, 2777–2787 (1998)Google Scholar
  2. 2.
    Bastian, A.J.: Learning to predict the future: the cerebellum adapts feedforward movement control. Curr. Opin. Neurobiol 16, 645–649 (2006)CrossRefGoogle Scholar
  3. 3.
    Buhusi, C.V., Meck, W.H.: What makes us tick? functional and neural mechanisms of interval timing. Nat. Rev. Neurosci 6, 755–765 (2005)CrossRefGoogle Scholar
  4. 4.
    Butz, M.V., Siguad, O., Gerard, P.: Anticipatory behavior: Exploiting knowledge about the future to improve current behavior. In: Butz, M.V., Sigaud, O., Gérard, P. (eds.) Anticipatory Behavior in Adaptive Learning Systems. LNCS (LNAI), vol. 2684, pp. 1–10. Springer, Heidelberg (2003)Google Scholar
  5. 5.
    Clayton, N.S., Bussey, T.J., Dickinson, A.: Can animals recall the past and plan for the future? Nat. Rev. Neurosci 4, 685–691 (2003)CrossRefGoogle Scholar
  6. 6.
    Doya, K.: Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol 10, 732–739 (2000)CrossRefGoogle Scholar
  7. 7.
    Ferbinteanu, J., Shapiro, M.L.: Prospective and retrospective memory coding in the hippocampus. Neuron 40, 1227–1239 (2003)CrossRefGoogle Scholar
  8. 8.
    Fleischer, J., Marsland, S., Shapiro, J.: Sensory anticipation for autonomous selection of robot landmarks. In: Butz, M.V., Sigaud, O., Gérard, P. (eds.) Anticipatory Behavior in Adaptive Learning Systems.Foundations,Theories, and Systems. LNCS (LNAI), vol. 2684, Springer, Heidelberg (2003)Google Scholar
  9. 9.
    Fleischer, J.G., Gally, J.A., Edelman, G.M., Krichmar, J.L.: Retrospective and prospective responses arising in a modeled hippocampus during maze navigation by a brain-based device. Proc. Natl. Acad. Sci. USA 104, 3556–3561 (2007)Google Scholar
  10. 10.
    Frank, L.M., Brown, E.N., Wilson, M.A.: Trajectory encoding in the hippocampus and entorhinal cortex. Neuron 27, 169–178 (2000)CrossRefGoogle Scholar
  11. 11.
    Griffiths, D., Dickenson, A., Clayton, N.: Episodic memory: what can animals remember about their past? Trends in Cognitive Science 3, 74–80 (1999)CrossRefGoogle Scholar
  12. 12.
    Gurney, K., Prescott, T.J., Wickens, J.R., Redgrave, P.: Computational models of the basal ganglia: from robots to membranes. Trends in Neurosciences 27, 453–459 (2004)CrossRefGoogle Scholar
  13. 13.
    Houk, J., Adams, J., Barto, A.: A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of Information Processing in the Basal Ganglia, MIT Press, Cambridge (1995)Google Scholar
  14. 14.
    Ivry, R.B., Keele, S.W., Diener, H.C.: Dissociation of the lateral and medial cerebellum in movement timing and movement execution. Exp. Brain Res. 73, 167–180 (1988)CrossRefGoogle Scholar
  15. 15.
    Ivry, R.B., Spencer, R.M.C.: The neural representation of time. Curr. Opin. Neurobiol 14, 225–232 (2004)CrossRefGoogle Scholar
  16. 16.
    Joel, D., Niv, Y., Ruppin, E.: Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Networks 15, 535–547 (2002)CrossRefGoogle Scholar
  17. 17.
    Kandel, E., Schwartz, J., Jessell, T. (eds.): Principles of Neural Science, 4th edn. McGraw-Hill, New York (2000)Google Scholar
  18. 18.
    Kaplan, F., Oudeyer, P.Y.: Maximizing learning progress: An internal reward system for development. In: Iida, F., Pfeifer, R., Steels, L., Kuniyoshi, Y. (eds.) Embodied Artificial Intelligence. LNCS, vol. 3139, pp. 259–270. Springer, Heidelberg (2004)Google Scholar
  19. 19.
    Krichmar, J., Seth, A., Nitz, D., Fleischer, J., Edelman, G.: Spatial navigation and causal analysis in a brain-based device modeling cortical-hippocampal interactions. Neuroinformatics 3, 197–221 (2005)CrossRefGoogle Scholar
  20. 20.
    Lisman, J.: The theta/gamma discrete phase code occuring during the hippocampal phase precession may be a more general brain coding scheme. Hippocampus 15, 913–922 (2005)CrossRefGoogle Scholar
  21. 21.
    Maguire, E.A., Woollett, K., Spiers, H.J.: London taxi drivers and bus drivers: A structural MRI and neuropsychological analysis. Hippocampus 16, 1091–1101 (2006)CrossRefGoogle Scholar
  22. 22.
    Marr, D.: A theory of cerebellar cortex. J. Phsysiol. 202, 437–470 (1969)Google Scholar
  23. 23.
    Mauk, M.D., Buonomano, D.V.: The neural basis of temporal processing. Annu. Rev. Neurosci 27, 307–340 (2004)CrossRefGoogle Scholar
  24. 24.
    McCormick, D.A., Thompson, R.F.: Neuronal responses of the rabbit cerebellum during acquisition and performance of a classically conditioned nictitating membrane-eyelid response. J Neurosci 4, 2811–2822 (1984)Google Scholar
  25. 25.
    McEchron, M.D., Tseng, W., Disterhoft, J.F.: Single neurons in ca1 hippocampus encode trace interval duration during trace heart rate (fear) conditioning in rabbit. Journal of Neuroscience 23, 1535–1547 (2003)Google Scholar
  26. 26.
    McHaffie, J.G., Stanford, T.R., Stein, B.E., Coizet, V., Redgrave, P.: Subcortical loops through the basal ganglia. Trends Neurosci 28, 401–407 (2005)CrossRefGoogle Scholar
  27. 27.
    Miall, R.C., Reckess, G.Z.: The cerebellum and the timing of coordinated eye and hand tracking. Brain Cogn. 48, 212–226 (2002)CrossRefGoogle Scholar
  28. 28.
    Miall, R.C., Reckess, G.Z., Imamizu, H.: The cerebellum coordinates eye and hand tracking movements. Nat. Neurosci 4, 638–644 (2001)CrossRefGoogle Scholar
  29. 29.
    Montague, P.R., Berns, G.S.: Neural economics and the biological substrates of valuation. Neuron 36, 265–284 (2002)CrossRefGoogle Scholar
  30. 30.
    Moskovitch, M., Nadel, L., Winocur, G., Gilboa, A., Rosenbaum, R.S.: The cognitive neuroscience of remote episodic, semantic and spatial memory. Current Opinion in Neurobiology 16, 179–190 (2006)CrossRefGoogle Scholar
  31. 31.
    O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., Dolan, R.J.: Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004)CrossRefGoogle Scholar
  32. 32.
    O’Keefe, J., Dostrovsky, J.: The hippocampus as a spatial map. preliminary evidence from unit activity in the freely-moving rat. Brain Research 34, 171–175 (1971)Google Scholar
  33. 33.
    O’Keefe, J., Nadel, L.: The Hippocampus as a Cognitive Map. Clarendon Press, Oxford (1978)Google Scholar
  34. 34.
    O’Keefe, J., Recce, M.L.: Phase relationship between hippocampal place units and the eeg theta rhythm. Hippocampus 3, 317–330 (1993)CrossRefGoogle Scholar
  35. 35.
    O’Reilly, R.C., Frank, M.J.: Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Computation 18, 283–328 (2006)MATHCrossRefMathSciNetGoogle Scholar
  36. 36.
    Redgrave, P., Prescott, T.J., Gurney, K.: Is the short-latency dopamine response too short to signal reward error? Trends in Neurosciences 22, 146–151 (1999)CrossRefGoogle Scholar
  37. 37.
    Redish, A.D.: Beyond the cognitive map: From place cells to episodic memory. MIT Press, Cambridge (1999)Google Scholar
  38. 38.
    Rescorla, R., Wagner, A.: A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Classical Conditioning II. Appleton-Century-Crofts (1972)Google Scholar
  39. 39.
    Roberts, A., Robbins, T., Weiskrantz, L. (eds.): The Prefrontal Cortex: Executive and Cognitive Functions. Oxford University Press, Oxford, UK (1998)Google Scholar
  40. 40.
    Schmidhuber, J.: Curious model-building control systems. In: IEEE Intl Joint Conf. on Neural Networks. vol.2, pp. 1458–1463 (1991)Google Scholar
  41. 41.
    Schultz, W., Dayan, P., Montague, P.R.: A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)CrossRefGoogle Scholar
  42. 42.
    Schultz, W.: Getting formal with dopamine and reward. Neuron 36, 241–263 (2002)CrossRefGoogle Scholar
  43. 43.
    Schultz, W.: Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Curr. Opin. Neurobiol 14, 139–147 (2004)CrossRefGoogle Scholar
  44. 44.
    Scoville, W.B., Milner, B.: Loss of recent memory after bilateral hippocampal lesions. J. Neurochem. 20 (1957)Google Scholar
  45. 45.
    Shadmer, R., Wise, S.: The computational neurobiology of reaching and pointing: a foundation for motor learning. MIT Press, Cambridge (2005)Google Scholar
  46. 46.
    Squire, L.R., Stark, C.E.L., Clark, R.E.: The medial temporal lobe. Annu. Rev. Neurosci 27, 279–306 (2004)CrossRefGoogle Scholar
  47. 47.
    Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. MIT Press, Cambridge (1998)Google Scholar
  48. 48.
    Thompson, R.F.: In search of memory traces. Annual Review of Psychology 56, 1–23 (2005)CrossRefGoogle Scholar
  49. 49.
    Thorndike, E.: Animal Intelligence. Macmillan Press (1911)Google Scholar
  50. 50.
    Waelti, P., Dickinson, A., Schultz, W.: Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001)CrossRefGoogle Scholar
  51. 51.
    Wolpert, D.M., Ghahramani, Z.: Computational principles of movement neuroscience. Nat. Neurosci 3, 1212–1217 (2000)CrossRefGoogle Scholar
  52. 52.
    Wood, E.R., Dudchenko, P.A., Robitsek, R.J., Eichenbaum, H.: Hippocampal neurons encode information about different types of memory episodes occurring in the same location. Neuron 27, 623–633 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Jason G. Fleischer
    • 1
  1. 1.The Neurosciences Institute, 10640 John Jay Hopkins Drive, San Diego, CA 

Personalised recommendations