Advertisement

Go-Explore-NoGo (GEN) Paradigm in Decision Making—A Multimodel Approach

  • Alekhya Mandali
  • S. Akila Parvathy Dharshini
  • V. Srinivasa Chakravarthy
Chapter
Part of the Cognitive Science and Technology book series (CSAT)

Abstract

In this chapter, we built a hybrid model using the combination of biophysical and Izhikevich neurons and validated our earlier hypothesis about Go-Explore-NoGo (GEN) mechanism in BG. The hybrid model consists of Hodgkin–Huxley type model for STN, GPe, and GPi and spiking model for striatum. To capture the effect of dopamine (DA) on the BG nuclei dynamics, the synaptic weights between STN–GPe and the T-type cs in STN known to induce bursting behavior were modulated by DA. We compared the results from hybrid model with spiking Izhikevich model and rate-coded model for binary action selection task. The results from the hybrid model further reinforced the theory of GEN showing exploration levels are dependent on the level of DA. The results from n-arm bandit task also show that by decreasing the striatum (D1) to GPi weight in the spiking model, we can increase the exploration level in the system reflected as the decreased average reward obtained by the model. The n-arm bandit results were compared with the results from rate-coded and lumped softmax model.

References

  1. Bergman, H., Wichmann, T., Karmon, B., & DeLong, M. (1994). The primate subthalamic nucleus. II. Neuronal activity in the MPTP model of parkinsonism. Journal of Neurophysiology, 72(2), 507–520.CrossRefGoogle Scholar
  2. Bevan, M. D., Magill, P. J., Terman, D., Bolam, J. P., & Wilson, C. J. (2002). Move to the rhythm: oscillations in the subthalamic nucleus–external globus pallidus network. Trends in Neurosciences, 25(10), 525–531.CrossRefGoogle Scholar
  3. Borisyuk, G. N., Borisyuk, R. M., Khibnik, A. I., & Roose, D. (1995). Dynamics and bifurcations of two coupled neural oscillators with different connection types. Bulletin of Mathematical Biology, 57(6), 809–840.CrossRefzbMATHGoogle Scholar
  4. Chakravarthy, V., Joseph, D., & Bapi, R. S. (2010). What do the basal ganglia do? A modeling perspective. Biological cybernetics, 103(3), 237–253.MathSciNetCrossRefzbMATHGoogle Scholar
  5. Dovzhenok, A., & Rubchinsky, L. L. (2012). On the Origin of Tremor in Parkinson’s Disease. PLoS ONE, 7(7), e41598.CrossRefGoogle Scholar
  6. Gupta, A., Balasubramani, P. P., & Chakravarthy, V. S. (2013). Computational model of precision grip in Parkinson’s disease: a utility based approach. Frontiers in computational neuroscience, 7.Google Scholar
  7. Hammond, C., Bergman, H., & Brown, P. (2007). Pathological synchronization in Parkinson’s disease: Networks, models and treatments. Trends in Neurosciences, 30(7), 357–364.CrossRefGoogle Scholar
  8. Heida, T., Marani, E., & Usunoff, K. G. (2008). The Basal Ganglia: Springer.Google Scholar
  9. Humphries, M. D., Stewart, R. D., & Gurney, K. N. (2006). A physiologically plausible model of action selection and oscillatory activity in the basal ganglia. The Journal of Neuroscience, 26(50), 12921–12942.CrossRefGoogle Scholar
  10. Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14(6), 1569–1572.MathSciNetCrossRefGoogle Scholar
  11. Kalva, S. K., Rengaswamy, M., Chakravarthy, V. S., & Gupte, N. (2012). On the neural substrates for exploratory dynamics in basal ganglia: a model. Neural Networks, 32, 65–73.  https://doi.org/10.1016/j.neunet.2012.02.031.CrossRefGoogle Scholar
  12. Krishnan, R., Ratnadurai, S., Subramanian, D., Chakravarthy, V. S., & Rengaswamy, M. (2011). Modeling the role of basal ganglia in saccade generation: Is the indirect pathway the explorer? Neural Networks, 24(8), 801–813.CrossRefGoogle Scholar
  13. Loucif, A. J., Woodhall, G. L., Sehirli, U. S., & Stanford, I. M. (2008). Depolarisation and suppression of burst firing activity in the mouse subthalamic nucleus by dopamine D1/D5 receptor activation of a cyclic-nucleotide gated non-specific cation conductance. Neuropharmacology, 55(1), 94–105.CrossRefGoogle Scholar
  14. Magdoom, K., Subramanian, D., Chakravarthy, V. S., Ravindran, B., Amari, S.-I., & Meenakshisundaram, N. (2011). Modeling basal ganglia for understanding parkinsonian reaching movements. Neural Computation, 23(2), 477–516.CrossRefzbMATHGoogle Scholar
  15. Mandali, A., & Chakravarthy, V. S. (2015). A computational basal ganglia model to assess the role of STN-DBS on Impulsivity in Parkinson’s disease. Paper presented at the Neural Networks (IJCNN), 2015 International Joint Conference on.Google Scholar
  16. Mandali, A., Rengaswamy, M., Chakravarthy, V. S., & Moustafa, A. A. (2015). A spiking Basal Ganglia model of synchrony, exploration and decision making. Frontiers in Neuroscience, 9, 191.CrossRefGoogle Scholar
  17. Muralidharan, V., Balasubramani, P. P., Chakravarthy, V. S., Lewis, S. J., & Moustafa, A. A. (2013). A computational model of altered gait patterns in parkinson’s disease patients negotiating narrow doorways. Frontiers in computational neuroscience, 7.Google Scholar
  18. Park, C., Worth, R. M., & Rubchinsky, L. L. (2010). Fine temporal structure of beta oscillations synchronization in subthalamic nucleus in Parkinson’s disease. Journal of Neurophysiology, 103(5), 2707–2716.CrossRefGoogle Scholar
  19. Park, C., Worth, R. M., & Rubchinsky, L. L. (2011). Neural dynamics in parkinsonian brain: the boundary between synchronized and nonsynchronized dynamics. Physical Review E, 83(4), 042901.CrossRefGoogle Scholar
  20. Ramanathan, S., Tkatch, T., Atherton, J. F., Wilson, C. J., & Bevan, M. D. (2008). D2-like dopamine receptors modulate SKCa channel function in subthalamic nucleus neurons through inhibition of Cav2. 2 channels. Journal of Neurophysiology, 99(2), 442–459.CrossRefGoogle Scholar
  21. Rubin, J. E., & Terman, D. (2004). High frequency stimulation of the subthalamic nucleus eliminates pathological thalamic rhythmicity in a computational model. Journal of Computational Neuroscience, 16(3), 211–235.CrossRefGoogle Scholar
  22. Sinha, S. (1999). Noise-free stochastic resonance in simple chaotic systems. Physica A: Statistical Mechanics and its Applications, 270(1), 204–214.CrossRefGoogle Scholar
  23. Sukumar, D., Rengaswamy, M., & Chakravarthy, V. S. (2012). Modeling the contributions of Basal ganglia and Hippocampus to spatial navigation using reinforcement learning. PLoS ONE, 7(10), e47467.CrossRefGoogle Scholar
  24. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1): Cambridge Univ Press.Google Scholar
  25. Tachibana, Y., Iwamuro, H., Kita, H., Takada, M., & Nambu, A. (2011). Subthalamo-pallidal interactions underlying parkinsonian neuronal oscillations in the primate basal ganglia. European Journal of Neuroscience, 34(9), 1470–1484.CrossRefGoogle Scholar
  26. Tai, C.-H., Yang, Y.-C., Pan, M.-K., Huang, C.-S., & Kuo, C.-C. (2011). Modulation of subthalamic T-type Ca 2+ channels remedies locomotor deficits in a rat model of Parkinson disease. The Journal of Clinical Investigation, 121(8), 3289–3305.CrossRefGoogle Scholar
  27. Terman, D., Rubin, J., Yew, A., & Wilson, C. (2002). Activity patterns in a model for the subthalamopallidal network of the basal ganglia. The Journal of neuroscience, 22(7), 2963–2976.Google Scholar
  28. Weinberger, M., & Dostrovsky, J. O. (2011). A basis for the pathological oscillations in basal ganglia: the crucial role of dopamine. NeuroReport, 22(4), 151.CrossRefGoogle Scholar
  29. Yang, Y.-C., Tai, C.-H., Pan, M.-K., & Kuo, C.-C. (2014). The T-type calcium channel as a new therapeutic target for Parkinson’s disease. Pflügers Archiv-European Journal of Physiology, 466(4), 747–755.CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Alekhya Mandali
    • 1
  • S. Akila Parvathy Dharshini
    • 2
  • V. Srinivasa Chakravarthy
    • 3
  1. 1.Department of Psychiatry, School of Clinical SciencesUniversity of CambridgeCambridgeUK
  2. 2.Protein Bioinformatics LaboratoryChennaiIndia
  3. 3.Computational Neuroscience Laboratory, Bhupat and Jyoti Mehta School of Biosciences, Department of BiotechnologyIndian Institute of Technology MadrasChennaiIndia

Personalised recommendations