The Hierarchical Organisation of Cortical and Basal-Ganglia Systems: A Computationally-Informed Review and Integrated Hypothesis

  • Gianluca Baldassarre
  • Daniele Caligiore
  • Francesco Mannella
Chapter

Abstract

To suitably adapt to the challenges posed by reproduction and survival, animals need to learn to select when to perform different behaviours, to have internal criteria for guiding these learning processes, and to perform behaviours efficiently once selected. To implement these processes, their brains must be organised in a suitable hierarchical fashion. Here we briefly review two types of neural/behavioural/computational literatures, focussed, respectively, on cortex and on sub-cortical areas, and highlight their important limitations. Then we review two computational modelling works of the authors that exemplify the problems, brain areas, experiments, main concepts, and limitations of the two research threads. Finally we propose a theoretical integration of the two views, showing how this allows us to solve most of the problems found by the two accounts if taken in isolation. The overall picture that emerges is that the cortical and the basal ganglia systems form two highly-organised hierarchical systems working in close synergy and jointly solving all the challenges of choice, selection, and implementation needed to acquire and express adaptive behaviour.

Bibliography

  1. Abercrombie, E. D., Keefe, K. A., DiFrischia, D. S., Zigmond, M. J. (1989). Differential effect of stress on in vivo dopamine release in striatum, nucleus accumbens, and medial frontal cortex. Journal of Neurochemistry, 52(5), 1655–1658.Google Scholar
  2. Alcock, J. (1998). Animal behavior: an evolutionary approach, 6th edn. Sunderland: Sinauer Associated.Google Scholar
  3. Alexander, G. E., & Crutcher, M. D. (1990). Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends in Neurosciences, 13(7), 266–271.Google Scholar
  4. Alexander, G. E., DeLong, M. R., Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9, 357–381.Google Scholar
  5. Ashby, F. G., Ennis, J. M., Spiering, B. J. (2007). A neurobiological theory of automaticity in perceptual categorization. Psychological Review, 114(3), 632–656.Google Scholar
  6. Ashby, F. G., Turner, B. O., Horvitz, J. C. (2010). Cortical and basal ganglia contributions to habit learning and automaticity. Trends in Cognitive Sciences, 14(5), 208–215.Google Scholar
  7. Balleine, B. W., & Dickinson, A. (1998). Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology, 37(4–5), 407–419.Google Scholar
  8. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.Google Scholar
  9. Barto, A. G., Singh, S., Chentanez, N. (2004). Intrinsically motivated learning of hierarchical collections of skills. In J. Triesch, & T. Jebara (Eds.), International conference on developmental learning (ICDL2004) (pp. 112–119). Piscataway, NJ: IEEE. UCSD Institute for Neural Computation, LaJolla, CA.Google Scholar
  10. Barto, A. G., Sutton, R. S., Anderson, C. W. (1983). Neuronlike adaptive elements that can learn difficult control problems. IEEE Transactions on Systems Man and Cybernetics, 13, 835–846.Google Scholar
  11. Bast, T. (2007). Toward an integrative perspective on hippocampal function: from the rapid encoding of experience to adaptive behavior. Reviews in the Neurosciences, 18(3–4), 253–281.Google Scholar
  12. Berridge, K. C., & Robinson, T. E. (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Research Review, 28(3), 309–369.Google Scholar
  13. Botvinick, M. M., Niv, Y., Barto, A. (2008). Hierarchically organized behavior and its neural foundations: a reinforcement-learning perspective. Cognition, 113(3), 262–280.Google Scholar
  14. Caligiore, D., Borghi, A. M., Parisi, D., Baldassarre, G. (2010). TRoPICALS: a computational embodied neuroscience model of compatibility effects. Psychological Review, 117, 1188–1228.Google Scholar
  15. Caligiore, D., Borghi, A. M., Parisi, D., Ellis, R., Cangelosi, A., Baldassarre, G. (2012). How affordances associated with a distractor object affect compatibility effects: a study with the computational model tropicals. Psychological Research, 77, 7–19.Google Scholar
  16. Cardinal, R. N., Parkinson, J. A., Hall, J., Everitt, B. J. (2002). Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neuroscience & Biobehavioral Reviews, 26(3), 321–352.Google Scholar
  17. Carelli, R. M., Wolske, M., West, M. O. (1997). Loss of lever press-related firing of rat striatal forelimb neurons after repeated sessions in a lever pressing task. Journal of Neuroscience, 17(5), 1804–1814.Google Scholar
  18. Chevalier, G., & Deniau, J. M. (1990). Disinhibition as a basic process in the expression of striatal functions. Trends in Neurosciences, 13(7), 277–280.Google Scholar
  19. Cisek, P. (2007). Cortical mechanisms of action selection: the affordance competition hypothesis. Philosophical Transactions of the Royal Society B: Biological Sciences, 362, 1585–1599.Google Scholar
  20. Cisek, P., & Kalaska, J. F. (2010). Neural mechanisms for interacting with a world full of action choices. Annual Review of Neuroscience, 33, 269–298.Google Scholar
  21. Corbit, L. H., & Balleine, B. W. (2011). The general and outcome-specific forms of pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. Journal of Neuroscience, 31(33), 11786–11794.Google Scholar
  22. Corbit, L. H., Muir, J. L., Balleine, B. W. (2001). The role of the nucleus accumbens in instrumental conditioning: Evidence of a functional dissociation between accumbens core and shell. Journal of Neuroscience, 21(9), 3251–3260.Google Scholar
  23. Daw, N. D., Niv, Y., Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704–1711.Google Scholar
  24. Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience: computational and mathematical modeling of neural systems. Cambridge: MIT.Google Scholar
  25. Deco, G., & Rolls, E. T. (2003). Attention and working memory: a dynamical model of neuronal activity in the prefrontal cortex. European Journal of Neuroscience, 18, 2374–2390.Google Scholar
  26. Doya, K. (2000). Complementary roles of basal ganglia and cerebellum in learning and motor control. Current Opinion in Neurobiology, 10(6), 732–739.Google Scholar
  27. Ellis, R., & Tucker, M. (2001). The potentiation of grasp types during visual object categorization. Visual Cognition, 8, 769–800.Google Scholar
  28. Ellis, R., Tucker, M., Symes, E., Vainio, L. (2007). Does selecting one visual object from several require inhibition of the actions associated with nonselected objects? Journal of Experimental Psychology Human Perception and Performance, 33, 670–691.Google Scholar
  29. Erlhagen, W., & Schoner, G. (2002). Dynamic field theory of movement preparation. Psychological Review, 109, 545–571.Google Scholar
  30. Evangeliou, M. N., Raos, V., Galletti, C., Savaki, H. E. (2009). Functional imaging of the parietal cortex during action execution and observation. Cerebral Cortex, 19, 624–639.Google Scholar
  31. Fagg, A. H., & Arbib, M. A. (1998). Modeling parietal-premotor interactions in primate control of grasping. Neural Networks, 11(7–8), 1277–1303.Google Scholar
  32. Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1(1), 1–47.Google Scholar
  33. Fogassi, L., Ferrari, P. F., Gesierich, B., Rozzi, S., Chersi, F., Rizzolatti, G. (2005). Parietal lobe: from action organization to intention understanding. Science, 308, 662–667.Google Scholar
  34. Fuster, J. M. (2001). The prefrontal cortex—an update: time is of the essence. Neuron, 30, 319–333.Google Scholar
  35. Gazzaniga, M. (2004). The cognitive neurosciences III. Cambridge: MIT.Google Scholar
  36. Grace, A. A., Floresco, S. B., Goto, Y., Lodge, D. J. (2007). Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends in Neurosciences, 30(5), 220–227.Google Scholar
  37. Graybiel, A., & Kimura, M. (1995). Adaptive neural networks in the basal ganglia. In J. C. Houk, J. Davis, D. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 103–116). Cambridge: MIT.Google Scholar
  38. Graziano, M. S. A. (2011). New insights into motor cortex. Neuron, 71(3), 387–388.MathSciNetGoogle Scholar
  39. Gurney, K., Prescott, T., Redgrave, P. (2001). A computational model of action selection in the basal ganglia. i. a new functional anatomy. Biological Cybernetics, 84, 401–410.Google Scholar
  40. Haber, S. N. (2003). The primate basal ganglia: parallel and integrative networks. Journal of Chemical Neuroanatomy, 26(4), 317–330.MathSciNetGoogle Scholar
  41. Haber, S. N., Fudge, J. L., McFarland, N. R. (2000). Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. Journal of Neuroscience, 20, 2369–2382.Google Scholar
  42. Hall, J., Parkinson, J. A., Connor, T. M., Dickinson, A., Everitt, B. J. (2001). Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behaviour. European Journal of Neuroscience, 13(10), 1984–1992.Google Scholar
  43. Hamilton, A. F., & Grafton, S. (2007). The motor hierarchy: from kinematics to goals and intentions (vol. 22, pp. 381–408). Oxford: Oxford University Press.Google Scholar
  44. Hebb, D. O. (1949). The organization of behaviour. New York: Wiley.Google Scholar
  45. Heimer, L., Switzer, R. D., Hoesen, V. G. W. (1982). Ventral striatum and ventral pallidum: components of the motor system? Trends in Neurosciences, 5(0), 83–87.Google Scholar
  46. Houk, J. C., Davis, J., Beiser, D. (Eds.), (1995). Models of information processing in the basal ganglia. Cambridge: MIT.Google Scholar
  47. Houk, J. C., & Wise, S. P. (1995). Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: Their role in planning and controlling action. Cerebral Cortex, 5(2), 95–110.Google Scholar
  48. Hua, S. E., & Houk, J. C. (1997). Cerebellar guidance of premotor network development and sensorimotor learning. Learning & Memory, 4(1), 63–76.Google Scholar
  49. Humphries, M. D., & Prescott, T. J. (2010). The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Progress in Neurobiology, 90(4), 385–417.Google Scholar
  50. Iriki, A., Pavlides, C., Keller, A., Asanuma, H. (1989). Long-term potentiation in the motor cortex. Science, 245(4924), 1385–1387.Google Scholar
  51. Ito, M. (2008). Control of mental activities by internal models in the cerebellum. Nature Reviews Neuroscience, 9(4), 304–313.Google Scholar
  52. Jeannerod, M. (1999). Visuomotor channels: their integration in goal-directed prehension. Human Movement Science, 18(2–3), 201–218.Google Scholar
  53. Joel, D., Niv, Y., Ruppin, E. (2002). Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Networks, 15(4–6), 535–547.Google Scholar
  54. Kawato, M. (1999). Internal models for motor control and trajectory planning. Current Opinion in Neurobiology, 9(6), 718–727.Google Scholar
  55. Kilner, J. M. (2011). More than one pathway to action understanding. Trends in Cognitive Sciences, 15, 352–357.Google Scholar
  56. Kimchi, E. Y., & Laubach, M. (2009). Dynamic encoding of action selection by the medial striatum. Journal of Neuroscience, 29(10), 3148–3159.Google Scholar
  57. Kirkwood, A., Rioult, M. C., Bear, M. F. (1996). Experience-dependent modification of synaptic plasticity in visual cortex. Nature, 381(6582), 526–528.Google Scholar
  58. Koechlin, E., & Summerfield, C. (2007). An information theoretical approach to prefrontal executive function. Trends Cognitive Sciences, 11(6), 229–235.Google Scholar
  59. Kohonen, T. (2003). Self-organized maps of sensory events. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 361, 1177–1186.MathSciNetGoogle Scholar
  60. Lestou, V., Pollick, F. E., Kourtzi, Z. (2008). Neural substrates for action understanding at different description levels in the human brain. Journal of Cognitive Neuroscience, 20, 324–341.Google Scholar
  61. Levy, R., Friedman, H. R., Davachi, L., Goldman-Rakic, P. S. (1997). Differential activation of the caudate nucleus in primates performing spatial and nonspatial working memory tasks. Journal of Neuroscience, 17(10), 3870–3882.Google Scholar
  62. Lewis, S. J. G., Dove, A., Robbins, T. W., Barker, R. A., Owen, A. M. (2004). Striatal contributions to working memory: a functional magnetic resonance imaging study in humans. European Journal of Neuroscience, 19(3), 755–760.Google Scholar
  63. Lisman, J. E., & Grace, A. A. (2005). The hippocampal-vta loop: controlling the entry of information into long-term memory. Neuron, 46(5), 703–713.Google Scholar
  64. MacFarland, D. (1993). Animal behavior, 2nd edn. Harlow: Longman Group.Google Scholar
  65. Mannella, F., Mirolli, M., Baldassarre, G. (2010). The interplay of Pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat. In C. Tosh, & G. Ruxton (Eds.), Modelling perception with artificial neural networks (pp. 93–113). Cambridge: Cambridge University Press.Google Scholar
  66. Mannella, F., Mirolli, M., Baldassarre, G. (2011). A system-level neural model of the brain mechanisms underlying instrumental devaluation in rats. In COSYNE—Computational and Systems Neuroscience (2011), Salt Lake City, 24 February 2011. Available from Nature Precedings: http://precedings.nature.com/documents/5849/version/1.
  67. Meunier, D., Lambiotte, R., Bullmore, E. T. (2010). Modular and hierarchically modular organization of brain networks. Frontiers in Neuroscience, 4, 200.Google Scholar
  68. Middleton, F. A., & Strick, P. L. (1996). The temporal lobe is a target of output from the basal ganglia. Proceedings of the National Academy of Sciences USA, 93(16), 8683–8687.Google Scholar
  69. Middleton, F. A., & Strick, P. L. (2000). Basal ganglia and cerebellar loops: motor and cognitive circuits. Brain Research Reviews, 31(2–3), 236–250.Google Scholar
  70. Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167–202.Google Scholar
  71. Milner, A. D., & Goodale, M. A. (2008). Two visual systems re-viewed. Neuropsychologia, 46, 774–785.Google Scholar
  72. Mink, J. W. (1996). The basal ganglia: focused selection and inhibition of competing motor programs. Progress in Neurobiology, 50(4), 381–425.Google Scholar
  73. Mirolli, M., Mannella, F., Baldassarre, G. (2010). The roles of the amygdala in the affective regulation of body, brain, and behaviour. Connection Science, 22(3), 215–245.Google Scholar
  74. Mirolli, M., Santucci, V. G., Baldassarre, G. (2013). Phasic dopamine as a prediction error of intrinsic and extrinsic reinforcements driving both action acquisition and reward maximization: a simulated robotic study. Neural Networks, 39, 40–51.Google Scholar
  75. Munakata, Y., Herd, S. A., Chatham, C. H., Depue, B. E., Banich, M. T., O’Reilly, R. C. (2011). A unified framework for inhibitory control. Trends in Cognitive Sciences, 15(10), 453–459.Google Scholar
  76. Nachev, P., Kennard, C., Husain, M. (2008). Functional role of the supplementary and pre-supplementary motor areas. Nature Reviews Neuroscience, 9, 856–869.Google Scholar
  77. Oztop, E., & Arbib, M. A. (2002). Schema design and implementation of the grasp-related mirror neuron system. Biological Cybernetics, 87(2), 116–40.MATHGoogle Scholar
  78. Pavlides, C., Miyashita, E., Asanuma, H. (1993). Projection from the sensory to the motor cortex is important in learning motor skills in the monkey. Journal of Neurophysiology, 70(2), 733–741.Google Scholar
  79. Pitkänen, A., Savander, V., LeDoux, J. E. (1997). Organization of intra-amygdaloid circuitries in the rat: an emerging framework for understanding functions of the amygdala. Trends in Neurosciences, 20(11), 517–523.Google Scholar
  80. Pouget, A., Dayan, P., Zemel, R. (2000). Information processing with population codes. Nature Reviews Neuroscience, 1, 125–132.Google Scholar
  81. Redgrave, P., Prescott, T. J., Gurney, K. (1999). The basal ganglia: a vertebrate solution to the selection problem? Neuroscience, 89, 1009–1024.Google Scholar
  82. Redgrave, P., Rodriguez, M., Smith, Y., Rodriguez-Oroz, M. C., Lehericy, S., Bergman, H., et al. (2010). Goal-directed and habitual control in the basal ganglia: implications for Parkinson’s disease. Nature Reviews Neuroscience, 11(11), 760–772.Google Scholar
  83. Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27, 169–192.Google Scholar
  84. Rizzolatti, G., Fadiga, L., Gallese, V., Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Brain Research, 3, 131–141.Google Scholar
  85. Rolls, E. T., & Treves, A. (1998). Neural networks and brain function. Oxford: Oxford Unversity Press.Google Scholar
  86. Romanelli, P., Esposito, V., Schaal, D. W., Heit, G. (2005). Somatotopy in the basal ganglia: experimental and clinical evidence for segregated sensorimotor channels. Brain Research Reviews, 48(1), 112–128.Google Scholar
  87. Schrimsher, G. W., Billingsley, R. L., Jackson, E. F., Moore, B. D. (2002). Caudate nucleus volume asymmetry predicts attention-deficit hyperactivity disorder (ADHD) symptomatology in children. Journal of Child Neurology, 17(12), 877–884.Google Scholar
  88. Sobel, I., & Feldman, G. (1968). A 3x3 isotropic gradient operator for image processing. Presentation for Stanford Artificial Project.Google Scholar
  89. Solway, A., & Botvinick, M. M. (2012). Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. Psychological Review, 119(1), 120–154.Google Scholar
  90. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: an introduction. Cambridge: MIT.Google Scholar
  91. Tang, C., Pawlak, A. P., Prokopenko, V., West, M. O. (2007). Changes in activity of the striatum during formation of a motor habit. European Journal of Neuroscience, 25(4), 1212–1227.Google Scholar
  92. Thill, S., Caligiore, D., Borghi, A. M., Ziemke, T., Baldassarre, G. (2013). Theories and computational models of affordance and mirror systems: An integrative review. Neuroscience and Biobehavioral Reviews, 37, 491–521.Google Scholar
  93. Todorov, E., & Jordan, M. I. (2002). Optimal feedback control as a theory of motor coordination. Nature Neuroscience, 5(11), 1226–1235.Google Scholar
  94. Tokimura, H., Di Lazzaro, V., Tokimura, Y., Oliviero, A., Profice, P., Insola, A., et al. (2000). Short latency inhibition of human hand motor cortex by somatosensory input from the hand. Journal of Physiology, 523(Pt 2), 503–513.Google Scholar
  95. Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In D. J. Ingle, M. A. Goodale, W. R. J. Mansfield (Eds.), Analysis of visual behavior (vol. 549, pp. 549–586). Cambridge: MIT.Google Scholar
  96. Venditti, A., Mirolli, M., Parisi, D., Baldassarre, G. (2009). A neural-network model of the dynamics of hunger, learning and action vigor in mice. In R. Serra, M. Villani, I. Poli (Eds.), Artificial life and evolutionary computation: Proceedings of Wivace 2008 (pp. 131–142). Singapore: World Scientific. Venice, Italy, 8–10 September 2008.Google Scholar
  97. Volkow, N. D., Wang, G.-J., Newcorn, J., Telang, F., Solanto, M. V., Fowler, J. S., et al. (2007). Depressed dopamine activity in caudate and preliminary evidence of limbic involvement in adults with attention-deficit/hyperactivity disorder. Archives of General Psychiatry, 64(8), 932–940.Google Scholar
  98. Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., Pennartz, C. M. A. (2004). Putting a spin on the dorsal-ventral divide of the striatum. Trends in Neurosciences, 27(8), 468–474.Google Scholar
  99. Wallis, J. D., Anderson, K. C., Miller, E. K. (2001). Single neurons in prefrontal cortex encode abstract rules. Nature, 411, 953–956.Google Scholar
  100. Walsh, V., & Cowey, A. (2000). Transcranial magnetic stimulation and cognitive neuroscience. Nature Reviews Neuroscience, 1(1), 73–79.Google Scholar
  101. Wilson, C. J. (1995). The contribution of cortical neurons to the firing pattern of striatal spiny neurons. In J. C. Houk, J. L. Davids, D. G. Beiser, (Eds.), Models of information processing in the basal ganglia (pp. 29–50). Cambridge: MIT.Google Scholar
  102. Yeterian, E. H., & Pandya, D. N. (1995). Corticostriatal connections of extrastriate visual areas in rhesus monkeys. Journal of Comparative Neurology, 352(3), 436–457.Google Scholar
  103. Yin, H. H., & Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nature Reviews Neuroscience, 7, 464–476.Google Scholar
  104. Yin, H. H., Ostlund, S. B., Balleine, B. W. (2008). Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. European Journal of Neuroscience, 28(8), 1437–1448.Google Scholar
  105. Zahm, D. S. (2000). An integrative neuroanatomical perspective on some subcortical substrates of adaptive responding with emphasis on the nucleus accumbens. Neuroscience & Biobehavioral Reviews, 24(1), 85–105.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Gianluca Baldassarre
    • 1
  • Daniele Caligiore
    • 1
  • Francesco Mannella
    • 1
  1. 1.Laboratory of Computational Embodied NeuroscienceInstitute of Cognitive Sciences and Technologies, National Research CouncilRomeItaly

Personalised recommendations