Classical Computational Approaches to Modeling the Basal Ganglia
There have been several modeling approaches to simulate BG structure and function. In this chapter, we discuss major modeling frameworks that have been proposed to simulate many functions of the BG. Many of such modeling studies are classical approaches in the field of BG modeling, which have been repeatedly to simulate many BG functions. In short, here we discuss the following model approaches: dimensionality reduction models, action section selection models, Go/NoGo models, reinforcement learning (RL) models of the basal ganglia, and Actor–Critic models. Importantly, this chapter mainly provides an overview of main architectures used to simulate the BG structure and function. In addition, we discuss many other models, such as those of gait, reaching, and other in the following chapters.
- Baldassarre, G., & Parisi, D. (2000). Classical and instrumental conditioning: From laboratory phenomena to integrated mechanisms for adaptation.Google Scholar
- Barto, A. G. (1995). Adaptive critics and the basal ganglia. In J. C. Houk, J. L. Davis & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. xii, 382p). Cambridge, MA: MIT Press.Google Scholar
- Barto, A. G. (2003). Reinforcement learning. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks (pp. 963–968). Cambridge, MA: MIT Press.Google Scholar
- Benazzouz, A., & Hallett, M. (2000). Mechanism of action of deep brain stimulation. Neurology, 55(12 Suppl 6), S13–S16.Google Scholar
- Berns, G. S., & Sejnowski, T. J. (1996). How the basal ganglia make decisions. In A. Damasio, H. Damasio, & Y. Christen (Eds.), The neurobiology of decision making. Berlin: Springer.Google Scholar
- Braver, T. S., Barch, D. M., Keys, B. A., Carter, C. S., Cohen, J. D., Kaye, J. A., … Reed, B. R. (2001). Context processing in older adults: Evidence for a theory relating cognitive control to neurobiology in healthy aging. Journal of Experimental Psychology: General, 130(4), 746–763.Google Scholar
- Funkiewiez, A., Ardouin, C., Krack, P., Fraix, V., Van Blercom, N., Xie, J., … Pollak, P. (2003). Acute psychotropic effects of bilateral subthalamic nucleus stimulation and levodopa in Parkinson’s disease. Movement Disorders, 18(5), 524–530.Google Scholar
- Houk, J. C. (1995a). Information processing in modular circuits linking basal ganglia and cerebral Cortex. In J. C. Houk, J. L. Davis & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. xii, 382p). Cambridge, MA: MIT Press.Google Scholar
- Houk, J. C. (1995b). A model of how the basal ganglia generate and use neural signals that predict reinforcement. In J. C. Houk, J. L. Davis & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. xii, 382p). Cambridge, MA: MIT Press.Google Scholar
- Houk, J. C., Adams, J. L., & Barto, A. G. (1995). A model of how the basal ganglia generate and use neural signals that predict reinforcement. Models of Information Processing in the Basal Ganglia, 249–270.Google Scholar
- Humphries, M. D., Khamassi, M., & Gurney, K. (2012). Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. Frontiers in Neuroscience, 6.Google Scholar
- Khamassi, M., Girard, B., Berthoz, A., & Guillot, A. (2004). Comparing three Critic models of reinforcement learning in the basal ganglia connected to a detailed actor part in a S-R task. Paper presented at the Proceedings of the Eighth International Conference on Intelligent Autonomous Systems IAS-8, Amsterdam, The Netherlands.Google Scholar
- Kim, T., Hamade, K. C., Todorov, D., Barnett, W. H., Capps, R. A., Latash, E. M., … Molkov, Y. I. (2017). Reward based motor adaptation mediated by basal ganglia. Frontiers in Computational Neuroscience, 11, 19. https://doi.org/10.3389/fncom.2017.00019.
- Krack, P., Kumar, R., Ardouin, C., Dowsey, P. L., McVicker, J. M., Benabid, A. L., & Pollak, P. (2001). Mirthful laughter induced by subthalamic nucleus stimulation. Movement Disorders, 16(5), 867–875.Google Scholar
- Meissner, W., Leblois, A., Hansel, D., Bioulac, B., Gross, C. E., Benazzouz, A., & Boraud, T. (2005). Subthalamic high frequency stimulation resets subthalamic firing and reduces abnormal oscillations. Brain, 128(10), 2372–2382.Google Scholar
- O’Reilly, R. C. (2003). Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia. ICS Technical Report, (pp. 1–23).Google Scholar
- Prescott, T. J. (2002). Basal ganglia. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks (pp. xvii, 1290p). Cambridge, MA: MIT Press.Google Scholar
- Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. Journal of Neuroscience, 13(3), 900–913.Google Scholar
- Shivkumar, S., Muralidharan, V., & Chakravarthy, V. S. (2017). A biologically plausible architecture of the striatum to solve context-dependent reinforcement learning tasks. Frontiers in Neural Circuits, 11(45). https://doi.org/10.3389/fncir.2017.00045.
- Stocco, A. (2017). A biologically plausible action selection system for cognitive architectures: Implications of basal ganglia anatomy for learning and decision-making models. Cognitive Science https://doi.org/10.1111/cogs.12506.
- Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1). Cambridge: Cambridge University Press.Google Scholar
- Tanaka, S. C., Schweighofer, N., Asahi, S., Shishida, K., Okamoto, Y., Yamawaki, S., & Doya, K. (2007). Serotonin differentially regulates short- and long-term prediction of rewards in the ventral and dorsal striatum. PLoS One, 2(12), e1333. https://doi.org/10.1371/journal.pone.0001333.
- Tian, L., Arnold, M., Sejnowski, T., & Jabri, M. (2003). A biologically inspired computational model of the block copying task. Paper presented at the Proceedings of the third international workshop on Epigenetic robotics, Lund University Cognitive Studies.Google Scholar
- Wickens, J., & Kötter, R. (1995). Cellular models of reinforcement.Google Scholar
- Wilson, C. J. (2004). Basal ganglia. In G. M. Shepherd (Ed.), The synaptic organization of the 136 brain (pp. 361–413). New York: Oxford University Press.Google Scholar