Advertisement

Synchronization and Exploration in Basal Ganglia—A Spiking Network Model

  • Alekhya Mandali
  • V. Srinivasa Chakravarthy
Chapter
Part of the Cognitive Science and Technology book series (CSAT)

Abstract

Making an optimal decision could be to either ‘Explore’ or ‘exploit’ or ‘not to take any action,’ and basal ganglia (BG) are considered to be a key neural substrate in decision making. In earlier chapters, we had hypothesized earlier that the indirect pathway (IP) of the BG could be the subcortical substrate for exploration. Here, we build a spiking network model to relate exploration to synchrony levels in the BG (which are a neural marker for tremor in Parkinson’s disease). Key BG nuclei such as the subthalamic nucleus (STN), Globus Pallidus externus (GPe), and Globus Pallidus internus (GPi) were modeled as Izhikevich spiking neurons, whereas the striatal output was modeled as Poisson spikes. We have applied reinforcement learning framework with the dopamine signal representing the reward prediction error used for cortico-striatal weight update. We apply the model to two decision-making tasks: a binary action selection task and an n-armed bandit task. The model shows that exploration levels could be controlled by STN’s lateral connection strength which also influenced the synchrony levels in the STN–GPe circuit. An increase in STN’s lateral strength led to a decrease in exploration which can be thought as the possible explanation for reduced exploratory levels in Parkinson’s patients.

References

  1. Bergman, H., Wichmann, T., Karmon, B., & DeLong, M. (1994). The primate subthalamic nucleus. II. Neuronal activity in the MPTP model of parkinsonism. Journal of Neurophysiology, 72(2), 507–520.CrossRefGoogle Scholar
  2. Bevan, M. D., Magill, P. J., Terman, D., Bolam, J. P., & Wilson, C. J. (2002). Move to the rhythm: Oscillations in the subthalamic nucleus–external globus pallidus network. Trends in Neurosciences, 25(10), 525–531.CrossRefGoogle Scholar
  3. Borisyuk, G. N., Borisyuk, R. M., Khibnik, A. I., & Roose, D. (1995). Dynamics and bifurcations of two coupled neural oscillators with different connection types. Bulletin of Mathematical Biology, 57(6), 809–840.CrossRefzbMATHGoogle Scholar
  4. Bourdaud, N., Chavarriaga, R., Galán, F., & del R Millan, J. (2008). Characterizing the EEG correlates of exploratory behavior. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 6(6), 549–556.Google Scholar
  5. Brown, P. (2003). Oscillatory nature of human basal ganglia activity: Relationship to the pathophysiology of Parkinson’s disease. Movement Disorders, 18(4), 357–363.CrossRefGoogle Scholar
  6. Chakravarthy, V., Joseph, D., & Bapi, R. S. (2010). What do the basal ganglia do? A modeling perspective. Biological Cybernetics, 103(3), 237–253.MathSciNetCrossRefzbMATHGoogle Scholar
  7. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095), 876–879.CrossRefGoogle Scholar
  8. Fan, K. Y., Baufreton, J., Surmeier, D. J., Chan, C. S., & Bevan, M. D. (2012). Proliferation of external globus pallidus-subthalamic nucleus synapses following degeneration of midbrain dopamine neurons. The Journal of Neuroscience, 32(40), 13718–13728.CrossRefGoogle Scholar
  9. Frank, M. J. (2006). Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making. Neural Networks, 19(8), 1120–1136.CrossRefzbMATHGoogle Scholar
  10. Frank, M. J., Samanta, J., Moustafa, A. A., & Sherman, S. J. (2007). Hold your horses: Impulsivity, deep brain stimulation, and medication in parkinsonism. Science, 318(5854), 1309–1312.CrossRefGoogle Scholar
  11. Gillies, A., Willshaw, D., Gillies, A., & Willshaw, D. (1998). A massively connected subthalamic nucleus leads to the generation of widespread pulses. Proceedings of the Royal Society of London, Series B: Biological Sciences, 265(1410), 2101–2109.CrossRefGoogle Scholar
  12. Gupta, A., Balasubramani, P. P., & Chakravarthy, V. S. (2013). Computational model of precision grip in Parkinson’s disease: A utility based approach. Frontiers in Computational Neuroscience, 7.Google Scholar
  13. Hadipour-Niktarash, A., Rommelfanger, K. S., Masilamoni, G. J., Smith, Y., & Wichmann, T. (2012). Extrastriatal D2-like receptors modulate basal ganglia pathways in normal and parkinsonian monkeys. Journal of Neurophysiology, 107(5), 1500–1512.Google Scholar
  14. Hammond, C., Bergman, H., & Brown, P. (2007). Pathological synchronization in Parkinson’s disease: Networks, models and treatments. Trends in Neurosciences, 30(7), 357–364.CrossRefGoogle Scholar
  15. Hauptmann, C., & Tass, P. A. (2007). Therapeutic rewiring by means of desynchronizing brain stimulation. Biosystems, 89(1), 173–181.CrossRefGoogle Scholar
  16. Humphries, M. D., Khamassi, M., & Gurney, K. (2012). Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. Frontiers in Neuroscience, 6.Google Scholar
  17. Humphries, M. D., Stewart, R. D., & Gurney, K. N. (2006). A physiologically plausible model of action selection and oscillatory activity in the basal ganglia. The Journal of Neuroscience, 26(50), 12921–12942.CrossRefGoogle Scholar
  18. Izhikevich, E. M. (2007). Dynamical systems in neuroscience. Cambridge: The MIT press.Google Scholar
  19. Kalva, S. K., Rengaswamy, M., Chakravarthy, V. S., & Gupte, N. (2012). On the neural substrates for exploratory dynamics in basal ganglia: A model. Neural Networks, 32, 65–73.  https://doi.org/10.1016/j.neunet.2012.02.031.CrossRefGoogle Scholar
  20. Kang, G., & Lowery, M. M. (2013). Interaction of oscillations, and their suppression via deep brain stimulation, in a model of the cortico-basal ganglia network. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 21(2), 244–253.CrossRefGoogle Scholar
  21. Kliem, M. A., Maidment, N. T., Ackerson, L. C., Chen, S., Smith, Y., & Wichmann, T. (2007). Activation of nigral and pallidal dopamine D1-like receptors modulates basal ganglia outflow in monkeys. Journal of Neurophysiology, 98(3), 1489–1500.CrossRefGoogle Scholar
  22. Kreiss, D. S., Mastropietro, C. W., Rawji, S. S., & Walters, J. R. (1997). The response of subthalamic nucleus neurons to dopamine receptor stimulation in a rodent model of Parkinson’s disease. The Journal of Neuroscience, 17(17), 6807–6819.Google Scholar
  23. Krishnan, R., Ratnadurai, S., Subramanian, D., Chakravarthy, V. S., & Rengaswamy, M. (2011). Modeling the role of basal ganglia in saccade generation: Is the indirect pathway the explorer? Neural Networks, 24(8), 801–813.CrossRefGoogle Scholar
  24. Kumar, A., Cardanobile, S., Rotter, S., & Aertsen, A. (2011). The role of inhibition in generating and controlling Parkinson’s disease oscillations in the basal ganglia. Frontiers in Systems Neuroscience, 5.Google Scholar
  25. Magdoom, K., Subramanian, D., Chakravarthy, V. S., Ravindran, B., Amari, S.-I., & Meenakshisundaram, N. (2011). Modeling basal ganglia for understanding Parkinsonian reaching movements. Neural Computation, 23(2), 477–516.CrossRefzbMATHGoogle Scholar
  26. Mandali, A., Rengaswamy, M., Chakravarthy, S., & Moustafa, A. A. (2015). A spiking Basal Ganglia model of synchrony, exploration and decision making. Frontiers in Neuroscience, 9, 191.CrossRefGoogle Scholar
  27. Muralidharan, V., Balasubramani, P. P., Chakravarthy, V. S., Lewis, S. J., & Moustafa, A. A. (2013). A computational model of altered gait patterns in parkinson’s disease patients negotiating narrow doorways. Frontiers in Computational Neuroscience, 7.Google Scholar
  28. Niv, Y. (2009). Reinforcement learning in the brain. Journal of Mathematical Psychology, 53(3), 139–154.MathSciNetCrossRefzbMATHGoogle Scholar
  29. Park, C., Worth, R. M., & Rubchinsky, L. L. (2010). Fine temporal structure of beta oscillations synchronization in subthalamic nucleus in Parkinson’s disease. Journal of Neurophysiology, 103(5), 2707–2716.CrossRefGoogle Scholar
  30. Park, C., Worth, R. M., & Rubchinsky, L. L. (2011). Neural dynamics in parkinsonian brain: the boundary between synchronized and nonsynchronized dynamics. Physical Review E, 83(4), 042901.CrossRefGoogle Scholar
  31. Pinsky, P. F., & Rinzel, J. (1995). Synchrony measures for biological neural networks. Biological Cybernetics, 73(2), 129–137.CrossRefzbMATHGoogle Scholar
  32. Plenz, D., & Kitai, S. T. (1998). Up and down states in striatal medium spiny neurons simultaneously recorded with spontaneous activity in fast-spiking interneurons studied in cortex–striatum–substantia nigra organotypic cultures. The Journal of Neuroscience, 18(1), 266–283.Google Scholar
  33. Plenz, D., & Kital, S. T. (1999). A basal ganglia pacemaker formed by the subthalamic nucleus and external globus pallidus. Nature, 400(6745), 677–682.CrossRefGoogle Scholar
  34. Reynolds, J. N. J., & Wickens, J. R. (2002). Dopamine-dependent plasticity of corticostriatal synapses. Neural Networks, 15(4), 507–521.CrossRefGoogle Scholar
  35. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1–27.MathSciNetCrossRefGoogle Scholar
  36. Sharott, A., Doig, N. M., Mallet, N., & Magill, P. J. (2012). Relationships between the firing of identified striatal interneurons and spontaneous and driven cortical activities in vivo. The Journal of Neuroscience, 32(38), 13221–13236.CrossRefGoogle Scholar
  37. Sinha, S. (1999). Noise-free stochastic resonance in simple chaotic systems. Physica A: Statistical Mechanics and its Applications, 270(1), 204–214.CrossRefGoogle Scholar
  38. Sukumar, D., Rengaswamy, M., & Chakravarthy, V. S. (2012). Modeling the contributions of Basal ganglia and Hippocampus to spatial navigation using reinforcement learning. PLoS ONE, 7(10), e47467.CrossRefGoogle Scholar
  39. Surmeier, D. J., Ding, J., Day, M., Wang, Z., & Shen, W. (2007). D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends in Neurosciences, 30(5), 228–235.CrossRefGoogle Scholar
  40. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1). UK: Cambridge University Press.Google Scholar
  41. Tachibana, Y., Iwamuro, H., Kita, H., Takada, M., & Nambu, A. (2011). Subthalamo-pallidal interactions underlying parkinsonian neuronal oscillations in the primate basal ganglia. European Journal of Neuroscience, 34(9), 1470–1484.CrossRefGoogle Scholar
  42. Vickers, D. (1970). Evidence for an accumulator model of psychophysical discrimination. Ergonomics, 13(1), 37–58.CrossRefGoogle Scholar
  43. Weinberger, M., & Dostrovsky, J. O. (2011). A basis for the pathological oscillations in basal ganglia: The crucial role of dopamine. NeuroReport, 22(4), 151.CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Department of Psychiatry, School of Clinical SciencesUniversity of CambridgeCambridgeUK
  2. 2.Computational Neuroscience Laboratory, Department of BiotechnologyBhupat and Jyoti Mehta School of Biosciences, Indian Institute of Technology MadrasChennaiIndia

Personalised recommendations