Abstract
The paper presents a hierarchical spike timing neural network model developed in NEST simulator that aims to reproduce the eye movements’ behavior of humans during decision making. It includes the main visual information processing structures starting from early level of the visual system up to the areas responsible for decision making based on accumulated sensory evidence. In addition, subcortical nuclei like the basal ganglia modulating the eyes’ motor activity and thus, the voluntary saccade control are included. The model mimics learning process in human brain regarded as Reinforcement Learning (RL) based on received reward information from the environment and modulating functions of the sensory-motor cortex via neurotransmitter dopamine influencing synapses plasticity. The model connectivity is designed according to available literature sources. Adaptation of dopamine-energetic synapses in dependence on external binary reinforcement signal is investigated. Our aim for future work is to compare the model behavior with experimental data from planned reinforcement learning experiments with humans observing visual stimuli and receiving feedback about their response correctness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Barto, A.G.: Adaptive critics and the basal ganglia. In: Houk, J.C., Davis, J.L., Beiser, D.G., (eds.) Models of Information Processing in the Basal Ganglia, pp. 215–232. MIT Press, Cambridge, MA (1995)
Bogacz, R., Larsen, T.: Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. Neural Comput. 23(4), 817–851 (2011)
Casti, A., Hayot, F., Xiao, Y., Kaplan, E.: A simple model of retina-LGN transmission. J. Comput. Neurosci. 24, 235–252 (2008)
Dunovan, K., Lynch, B., Molesworth, T., Verstynen, T.: Competing basal-ganglia pathways determine the difference between stopping and deciding not to go. eLife, 1–24 (2015). https://doi.org/10.7554/eLife.08723
Dunovan, K., Verstynen, T.: Believer-skeptic meets actor-critic : rethinking the role of basal ganglia pathways during decision-making and reinforcement learning. Front. Neurosci. 10(March), 1–15 (2016). https://doi.org/10.1101/037085
Escobar, M.-J., Masson, G.S., Vieville, T., Kornprobst, P.: Action recognition using a bio-inspired feedforward spiking network. Int. J. Comput. Vis. 82, 284–301 (2009)
Frank, M.J., Seeberger, L.C., O’Reilly, R.C.: By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306, 5703, 1940–1943 (2004). https://doi.org/10.1126/science.1102941
Fregnac, Y., Bathellier, B.: Cortical correlates of low-level perception: from neural circuits to percepts. Neuron 88, 110–126 (2015)
Ghodratia, M., Khaligh-Razavic, S.-M., Lehky, S.R.: Towards building a more complex view of the lateral geniculate nucleus: recent advances in understanding its role. Prog. Neurobiol. 156, 214–255 (2017)
Gleeson, P., Martinez, R., Davison, A.: Network models of V1. http://www.opensourcebrain.org/projects/111 (2016)
Herz, D.M., Zavala, B.A., Bogacz, R., Brown, P.: Neural correlates of decision thresholds in the human subthalamic nucleus. Current Biol. 1–5 (2016). https://doi.org/10.1016/j.cub.2016.01.051
Igarashi, J., Shounob, O., Fukai, T., Tsujino, H.: Real-time simulation of a spiking neural network model of the basal ganglia circuitry using general purpose computing on graphics processing units. Neural Netw. 24, 950–960 (2011)
Joel, D., Niv, Y., Ruppin, E.: Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw. 15, 535–547 (2002)
Koprinkova-Hristova, P., Bocheva, N., Nedelcheva, S.: Investigation of feedback connections effect of a spike timing neural network model of early visual system. In: Innovations in Intelligent Systems and Applications (INISTA), 3-5 July 2018, Thessaloniki, Greece (2018). https://doi.org/10.1109/INISTA.2018.8466292
Koprinkova-Hristova, P., Bocheva, N., Nedelcheva, S., Stefanova, M.: A model of self-motion perception developed in NEST. Front. Comput. Neurosci. (2019). https://doi.org/10.3389/fncom.2019.00020
Kremkow, J., Perrinet, L.U., Monier, C., Alonso, J.-M., Aertsen, A., Frégnac, Y., Masson, G.S.: Push-pull receptive field organization and synaptic depression: mechanisms for reliably encoding naturalistic stimuli in V1. Front. Neural Circ. (2016). https://doi.org/10.3389/fncir.2016.00037
Krishnan, R., Ratnadurai, S., Subramanian, D., Chakravarthy, V.S., Rengaswamyd, M.: Modeling the role of basal ganglia in saccade generation: is the indirect pathway the explorer? Neural Netw. 24, 801–813 (2011)
Kunkel, S., et al.: NEST 2.12.0. Zenodo. (2017). https://doi.org/10.5281/zenodo.259534
Layton, O.W., Fajen, B.R.: Possible role for recurrent interactions between expansion and contraction cells in MSTd during self-motion perception in dynamic environments. J. Vis. 17(5), 1–21 (2017)
Nedelcheva, S., Koprinkova-Hristova, P.: orientation selectivity tuning of a spike timing neural network model of the first layer of the human visual cortex. In: Georgiev, K., Todorov, M., Georgiev, I. (eds.)Advanced Computing in Industrial Mathematics. BGSIAM 2017. Studies in Computational Intelligence, vol. 793, pp. 291–303. Springer, Cham (2019)
Potjans, W., Morrison, A., Diesmann, M.: Enabling functional neural circuit simulations with distributed computing of neuromodulated plasticity. Front. in Comp. Neurosci. 4 (2010). https://doi.org/10.3389/fncom.2010.00141many
Sadeh, S., Rotter, S.: Statistics and geometry of orientation selectivity in primary visual cortex. Biol. Cybern. 108, 631–653 (2014)
Shadlen, M.N., Newsome, W.T.: Motion perception: seeing and deciding. Proc. Natl. Acad. Sci. U.S.A. 93(2), 628–633 (1996)
Troyer, T.W., Krukowski, A.E., Priebe, N.J., Miller, K.D.: Contrast invariant orientation tuning in cat visual cortex: thalamocortical input tuning and correlation-based intracortical connectivity. J. Neurosci. 18, 5908–5927 (1998)
Tsodyks, M., Uziel, A., Markram, H.: Synchrony generation in recurrent networks with frequency-dependent synapses. J. Neurosci. 20RC50, 1–5 (2000)
Webb, B.S., Ledgeway, T.Y., McGraw, P.V.: Relating spatial and temporal orientation pooling to population decoding solutions in human vision. Vision. Res. 50, 2274–2283 (2010)
Acknowledgements
The reported work is a part of and was supported by the project DN02/3/2016 “Modelling of voluntary saccadic eye movements during decision making” funded by the Bulgarian Science Fund.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Koprinkova-Hristova, P., Bocheva, N. (2021). Spike Timing Neural Model of Eye Movement Motor Response with Reinforcement Learning. In: Georgiev, I., Kostadinov, H., Lilkova, E. (eds) Advanced Computing in Industrial Mathematics. BGSIAM 2018. Studies in Computational Intelligence, vol 961. Springer, Cham. https://doi.org/10.1007/978-3-030-71616-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-71616-5_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-71615-8
Online ISBN: 978-3-030-71616-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)