Abstract
We present a model of basal ganglia as a key player in exploratory behavior. The model describes exploration of a virtual rat in a simulated “water pool” experiment. The virtual rat is trained using a reward-based or reinforcement learning paradigm which requires units with stochastic behavior for exploration of the system’s state space. We model the STN-GPe system as a pair of neuronal layers with oscillatory dynamics, exhibiting a variety of dynamic regimes like chaos, traveling waves and clustering. Invoking the property of chaotic systems to explore a state space, we suggest that the complex “exploratory” dynamics of STN-GPe system in conjunction with dopamine-based reward signaling present the two key ingredients of a reinforcement learning system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Redgrave, P., Prescott, T.J., Gurney, K.: The basal ganglia: a vertebrate solution to the selection problem? Neuroscience 89, 1009–1023 (1999)
Berns, G.S., Sejnowski, T.J.: A computational model of how the Basal Ganglia produce sequences. Journal of Cognitive Neuroscience 10(1), 108–121 (1998)
Houk, J.C., Davis, J.L., Beiser, D.G.: Models of Information Processing in the Basal Ganglia. MIT Press, Cambridge (1995)
Bevan, M.D., Magill, P.J., Terman, D., Bolam, J.P., Wilson, C.J.: Move To The Rhythm: Oscillations In The Subthalamic Nucleus-External Globus Pallidus Network. In: Trends in Neuroscience (2003) (In press)
Chirikov, B.: A universal instability of many-dimensional oscillator systems. Phys. Rev. 52, 263–379 (1979)
Chakravarthy, V.S., Thomas, S.T., Nair, N.: A model for scheduling motor unit recruitment in skeletal muscle. In: International Conference on Theoretical Neurobiology, National Brain Research Center, Gurgoan, February, 24-26 (2003)
Gillies, A., Willshaw, D., Li, Z.: Subthalamic-pallidal interactions are critical in determining normal and abnormal functioning of the basal ganglia. In: Proc R Soc Lond B Biol Sci., March 22, vol. 269(1491), pp. 545–551 (2002)
Terman, D., Rubin, J.E., Yew, A.C., Wilson, C.J.: Activity Patterns in a Model for the Subthalamopallidal Network of the Basal Ganglia. In: J Neurosci., April 1, vol. 22(7), pp. 2963–2976 (2002)
Harner, A.M.: An Introduction to Basal Ganglia Function. Boston University, Boston (1997)
Obeso, J.A., Rodriguez-Oroz, M.C., Rodriguez, M., Arbizu, J., Gimenez-Amaya, J.M.: The Basal Ganglia and Disorders of Movement: Pathophysiological Mechanisms. News Physiol Sci. 17, 51–55 (2002)
Montague, Dayan, Sejnowski: A Framework for Mesencephalic Dopamine Systems Based on Predictive Hebbian Learning. The Journal of Neuroscience 16(5), 1936–1947 (1996)
Sridharan, D.: Human Factors in Aviation: Willed action and its disorders, MTech Thesis, Department of Aerospace Engineering, IIT, Madras, India (2004)
Barto, A.G.: Reinforcement Learning. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks (1st Edition), MIT Press, Cambridge (1999)
Morris, R.G.M., Garrud, P., Rawlins, J.N.P.: Place navigation impaired in rats with hippocampal lesions. Nature 297, 681–683 (1982)
Skarda, C.A., Freeman, W.J.: How brain makes chaos in order to make sense of the world. Behavioral and Brain Sciences 10, 161–195 (1987)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Devarajan, S., Prashanth, P.S., Chakravarthy, V.S. (2004). The Role of the Basal Ganglia in Exploratory Behavior in a Model Based on Reinforcement Learning. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds) Neural Information Processing. ICONIP 2004. Lecture Notes in Computer Science, vol 3316. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30499-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-30499-9_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23931-4
Online ISBN: 978-3-540-30499-9
eBook Packages: Springer Book Archive