Advertisement

The Basal Ganglia System as an Engine for Exploration

  • V. Srinivasa Chakravarthy
  • Pragathi Priyadharsini Balasubramani
Chapter
Part of the Cognitive Science and Technology book series (CSAT)

Abstract

One of the earliest attempts at building a theory of the basal ganglia (BG) is based on the clinical findings that lesions to the direct and indirect pathways of the BG produce quite opposite motor manifestations (Albin et al., in Trends Neurosci 12(10):366–375, 1989). While lesions of the direct pathway (DP), affecting particularly the projections from the striatum to GPi, are associated with hypokinetic disorders (distinguished by a paucity of movement), lesions of the indirect pathway (IP) produce hyperkinetic disorders, such as chorea and tremor. In this chapter, we argue that describing the two BG pathways as having mutually opponent actions has limitations. We argue that the BG indirect pathway also plays a role in exploration. We should evidence from various motor learning and decision-making tasks that exploration is a necessary process in various behavioral processes. Importantly, we use the exploration mechanism explained here to simulate various processes of the basal ganglia which we discuss in the following chapters.

References

  1. Albin, R. L., Young, A. B., & Penney, J. B. (1989). The functional anatomy of basal ganglia disorders. Trends in Neurosciences, 12(10), 366–375.CrossRefGoogle Scholar
  2. Baunez, C., Humby, T., Eagle, D. M., Ryan, L. J., Dunnett, S. B., & Robbins, T. W. (2001). Effects of STN lesions on simple vs choice reaction time tasks in the rat: Preserved motor readiness, but impaired response selection. European Journal of Neuroscience, 13(8), 1609–1616.CrossRefGoogle Scholar
  3. Bergman, H., Wichmann, T., Karmon, B., & DeLong, M. (1994). The primate subthalamic nucleus. II. Neuronal activity in the MPTP model of Parkinsonism. Journal of Neurophysiology, 72(2), 507–520.CrossRefGoogle Scholar
  4. Brown, P., Oliviero, A., Mazzone, P., Insola, A., Tonali, P., & Di Lazzaro, V. (2001). Dopamine dependency of oscillations between subthalamic nucleus and pallidum in Parkinson’s disease. The Journal of Neuroscience, 21(3), 1033–1038.Google Scholar
  5. Brunel, N. (2000). Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons. Journal of Computational Neuroscience 8, 183–208.Google Scholar
  6. Chakravarthy, V. S., Joseph, D., & Bapi, R. S. (2010). What do the basal ganglia do? A modeling perspective. Biological Cybernetics, 103(3), 237–253.  https://doi.org/10.1007/s00422-010-0401-y.MathSciNetCrossRefzbMATHGoogle Scholar
  7. Contreras-Vidal, J., & Stelmach, G. E. (1995). Effects of Parkinsonism on motor control. Life Sciences, 58(3), 165–176.CrossRefGoogle Scholar
  8. Crick, F. (1984). Function of the thalamic reticular complex: The searchlight hypothesis. Proceedings of the National Academy of Sciences, 81(14), 4586–4590.CrossRefGoogle Scholar
  9. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095), 876–879.CrossRefGoogle Scholar
  10. Flores-Hernandez, J., Cepeda, C., Hernandez-Echeagaray, E., Calvert, C. R., Jokel, E. S., Fienberg, A. A., … Levine, M. S. (2002). Dopamine enhancement of NMDA currents in dissociated medium-sized striatal neurons: Role of D1 receptors and DARPP-32. Journal of Neurophysiol, 88(6), 3010–3020.Google Scholar
  11. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of Cognitive Neuroscience, 17(1), 51–72.  https://doi.org/10.1162/0898929052880093.CrossRefGoogle Scholar
  12. Gangadhar, G., Joseph, D., & Chakravarthy, V. S. (2008). Understanding Parkinsonian handwriting through a computational model of basal ganglia. Neural Computation, 20(10), 2491–2525.CrossRefGoogle Scholar
  13. Gillies, A., Willshaw, D., Atherton, J., & Arbuthnott, G. (2002). Functional interactions within the subthalamic nucleus. In The basal ganglia VII (pp. 359–368). Boston: Springer.Google Scholar
  14. Grabli, D., McCairn, K., Hirsch, E. C., Agid, Y., Féger, J., François, C., et al. (2004). Behavioural disorders induced by external globus pallidus dysfunction in primates: I. Behavioural study. Brain, 127(9), 2039–2054.CrossRefGoogle Scholar
  15. Gurney, K., Prescott, T. J., & Redgrave, P. (2001). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biological Cybernetics, 84(6), 401–410.CrossRefzbMATHGoogle Scholar
  16. Hollerman, J. R., & Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuroscience, 1(4), 304–309.CrossRefGoogle Scholar
  17. Houk, J. C., Davis, J. L., & Beiser, D. G. (1995). Models of information processing in the basal ganglia. Cambridge: The MIT press.Google Scholar
  18. Humphries, M., & Gurney, K. (2002). The role of intra-thalamic and thalamocortical circuits in action selection. Network: Computation in Neural Systems, 13(1), 131–156.CrossRefzbMATHGoogle Scholar
  19. Humphries, M. D., & Prescott, T. J. (2010). The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Progress in Neurobiology, 90(4), 385–417.  https://doi.org/10.1016/j.pneurobio.2009.11.003.CrossRefGoogle Scholar
  20. Hurtado, J. M., Gray, C. M., Tamas, L. B., & Sigvardt, K. A. (1999). Dynamics of tremor-related oscillations in the human globus pallidus: a single case study. Proceedings of the National Academy of Sciences, 96(4), 1674–1679.CrossRefGoogle Scholar
  21. Joel, D., Niv, Y., & Ruppin, E. (2002). Actor-critic models of the basal ganglia: New anatomical and computational perspectives. Neural Networks, 15(4–6), 535–547.CrossRefGoogle Scholar
  22. Kalva, S. K., Rengaswamy, M., Chakravarthy, V. S., & Gupte, N. (2012). On the neural substrates for exploratory dynamics in basal ganglia: A model. Neural Networks, 32, 65–73.  https://doi.org/10.1016/j.neunet.2012.02.031.CrossRefGoogle Scholar
  23. Kandel, E. R., Schwartz, J. H., & Jessell, T. M. (2000). Principles of neural science (Vol. 4). New York: McGraw-Hill.Google Scholar
  24. Kirkpatrick, S., Gelatt, C. D., Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671–680.MathSciNetCrossRefzbMATHGoogle Scholar
  25. Kliem, M. A., Maidment, N. T., Ackerson, L. C., Chen, S., Smith, Y., & Wichmann, T. (2007). Activation of nigral and pallidal dopamine D1-like receptors modulates basal ganglia outflow in monkeys. Journal of Neurophysiology, 98(3), 1489–1500.CrossRefGoogle Scholar
  26. Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001). Anticipation of increasing monetary reward selectively recruits nucleus accumbens. Journal of Neuroscience, 21(16), 159.Google Scholar
  27. Kravitz, A. V., Freeze, B. S., Parker, P. R., Kay, K., Thwin, M. T., Deisseroth, K., et al. (2010). Regulation of Parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature, 466(7306), 622–626.CrossRefGoogle Scholar
  28. Krishnan, R., Ratnadurai, S., Subramanian, D., Chakravarthy, V. S., & Rengaswamy, M. (2011). Modeling the role of basal ganglia in saccade generation: is the indirect pathway the explorer? Neural Networks, 24(8), 801–813.  https://doi.org/10.1016/j.neunet.2011.06.002.CrossRefGoogle Scholar
  29. Magdoom, K. N., Subramanian, D., Chakravarthy, V. S., Ravindran, B., Amari, S., & Meenakshisundaram, N. (2011). Modeling basal ganglia for understanding Parkinsonian reaching movements. Neural Computation, 23(2), 477–516. https://doi.org/10.1162/NECO_a_00073.CrossRefzbMATHGoogle Scholar
  30. Magill, P., Bolam, J., & Bevan, M. (2001). Dopamine regulates the impact of the cerebral cortex on the subthalamic nucleus–globus pallidus network. Neuroscience, 106(2), 313–330.CrossRefGoogle Scholar
  31. Mandali, A., & Chakravarthy, V. S. (2016). Probing the role of medication, DBS electrode position, and antidromic activation on impulsivity using a computational model of basal ganglia. Frontiers in Human Neuroscience, 10, 450.CrossRefGoogle Scholar
  32. Mandali, A., Rengaswamy, M., Chakravarthy, S., & Moustafa, A. A. (2015). A spiking Basal Ganglia model of synchrony, exploration and decision making. Frontiers in Neuroscience, 9, 191.CrossRefGoogle Scholar
  33. Michmizos, K. P., & Nikita, K. S. (2011). Local field potential driven Izhikevich model predicts a subthalamic nucleus neuron activity. Paper presented at the Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE.Google Scholar
  34. Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16(5), 1936–1947.Google Scholar
  35. Muralidharan, V., Balasubramani, P. P., Chakravarthy, V. S., Gilat, M., Lewis, S. J., & Moustafa, A. A. (2017). A neurocomputational model of the effect of cognitive load on freezing of gait in Parkinson’s disease. Frontiers in Human Neuroscience, 10, 649.  https://doi.org/10.3389/fnhum.2016.00649.CrossRefGoogle Scholar
  36. Muralidharan, V., Balasubramani, P. P., Chakravarthy, V. S., Lewis, S. J., & Moustafa, A. A. (2014). A computational model of altered gait patterns in Parkinson’s disease patients negotiating narrow doorways. Frontiers in Computational Neuroscience, 7, 190.  https://doi.org/10.3389/fncom.2013.00190.CrossRefGoogle Scholar
  37. O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304(5669), 452–454.CrossRefGoogle Scholar
  38. Plenz, D., & Kital, S. T. (1999). A basal ganglia pacemaker formed by the subthalamic nucleus and external globus pallidus. Nature, 400(6745), 677–682.CrossRefGoogle Scholar
  39. Pragathi Priyadharsini, B., Ravindran, B., & Srinivasa Chakravarthy, V. (2012). Understanding the role of serotonin in basal ganglia through a unified model. In A. P. Villa, W. Duch, P. Érdi, F. Masulli, & G. Palm (Eds.), Artificial Neural Networks and Machine Learning—ICANN 2012 (Vol. 7552, pp. 467–473). Berlin: Springer.Google Scholar
  40. Redgrave, P., Prescott, T. J., & Gurney, K. (1999). The basal ganglia: A vertebrate solution to the selection problem? Neuroscience, 89(4), 1009–1023.CrossRefGoogle Scholar
  41. Rushworth, M. F., & Behrens, T. E. (2008). Choice, uncertainty and value in prefrontal and cingulate cortex. Nature Neuroscience, 11(4), 389–397.CrossRefGoogle Scholar
  42. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599.CrossRefGoogle Scholar
  43. Sridharan, D., Prashanth, P. S., & Chakravarthy, V. S. (2006). The role of the basal ganglia in exploration in a neural model based on reinforcement learning. International Journal of Neural Systems, 16(2), 111–124.CrossRefGoogle Scholar
  44. Stein, P. S., Grillner, S., Selverston, A. I., & Stuart, D. G. (1997). Neurons, networks, and behavior. Cambridge, MA: MIT Press.Google Scholar
  45. Steiner, H., & Tseng, K. Y. (2010). Handbook of basal ganglia structure and function: A decade of progress (Vol. 20). Access Online via Elsevier.Google Scholar
  46. Sukumar, D., Rengaswamy, M., & Chakravarthy, V. S. (2012). Modeling the contributions of Basal ganglia and Hippocampus to spatial navigation using reinforcement learning. PLoS ONE, 7(10), e47467.  https://doi.org/10.1371/journal.pone.0047467.CrossRefGoogle Scholar
  47. Sutton, R., & Barto, A. (1998). Reinforcement learning: An introduction. Adaptive computations and machine learning. MIT Press/Bradford.Google Scholar
  48. Terman, D., Rubin, J., Yew, A., & Wilson, C. (2002). Activity patterns in a model for the subthalamopallidal network of the basal ganglia. The Journal of Neuroscience, 22(7), 2963–2976.Google Scholar
  49. Willshaw, D., & Li, Z. (2002). Subthalamic–pallidal interactions are critical in determining normal and abnormal functioning of the basal ganglia. Proceedings of the Royal Society of London, Series B: Biological Sciences, 269(1491), 545–551.CrossRefGoogle Scholar
  50. Yoshida, W., & Ishii, S. (2006). Resolution of uncertainty in prefrontal cortex. Neuron, 50(5), 781–789.CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • V. Srinivasa Chakravarthy
    • 1
  • Pragathi Priyadharsini Balasubramani
    • 2
  1. 1.Department of Biotechnology, Bhupat and Jyoti Mehta School of BiosciencesIndian Institute of Technology, MadrasChennaiIndia
  2. 2.Department of NeuroscienceUniversity of Rochester Medical CenterRochesterUSA

Personalised recommendations