A Spiking Neural Network Based Autonomous Reinforcement Learning Model and Its Application in Decision Making

Conference paper

DOI: 10.1007/978-3-319-49685-6_12

Volume 10023 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Wang G., Zeng Y., Xu B. (2016) A Spiking Neural Network Based Autonomous Reinforcement Learning Model and Its Application in Decision Making. In: Liu CL., Hussain A., Luo B., Tan K., Zeng Y., Zhang Z. (eds) Advances in Brain Inspired Cognitive Systems. BICS 2016. Lecture Notes in Computer Science, vol 10023. Springer, Cham


Inspired by biological spike information processing and the multiple brain region coordination mechanism, we propose an autonomous spiking neural network model for decision making. The proposed model is an expansion of the basal ganglia circuitry with automatic environment perception. It automatically constructs environmental states from image inputs. Contributions of this investigation can be summarized as the following: (1) In our model, the simplified Hodgkin-Huxley computing model is developed to achieve calculation efficiency closed to the LIF model and is used to obtain and test the ionic level properties in cognition. (2) A spike based motion perception mechanism is proposed to extract key elements for learning process from raw pixels without large amount of training. We apply our model in the “flappy bird” game and after dozens of training times, it can automatically generate rules to play well in the game. Besides, our model simulates cognitive defects when blocking some of sodium or potassium ion channels in the Hodgkin-Huxley model and this can be considered as a computational exploration on the mechanisms of cognition deep into ionic level.


Spiking neural network Hodgkin-Huxley model Autonomous reinforcement learning Decision making Basal Ganglia 

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Institute of Automation, Chinese Academy of ScienceBeijingChina
  2. 2.Center for Excellence in Brain Science and Intelligence TechnologyChinese Academy of SciencesShanghaiChina
  3. 3.University of Chinese Academy of SciencesBeijingChina