Chaos-Based Reinforcement Learning When Introducing Refractoriness in Each Neuron
Aiming for the emergence of “thinking”, we have proposed new reinforcement learning using a chaotic neural network. Then we have set up a hypothesis that the internal chaotic dynamics would grow up into “thinking” through learning. In our previous works, strong recurrent connection weights generate internal chaotic dynamics. On the other hand, chaotic dynamics are often generated by introducing refractoriness in each neuron. Refractoriness is the property that a firing neuron becomes insensitive for a while and observed in biological neurons. In this paper, in the chaos-based reinforcement learning, refractoriness is introduced in each neuron. It is shown that the network can learn a simple goal-reaching task through our new chaos-based reinforcement learning. It can learn with smaller recurrent connection weights than the case without refractoriness. By introducing refractoriness, the agent behavior becomes more exploratory and Lyapunov exponent becomes larger with the same recurrent weight range.
KeywordsReinforcement learning Chaotic neural network Goal reaching Refractoriness
This work was supported by JSPS KAKENHI Grant Number 15K00360.
- 1.Shibata, K., Goto, Y.: New reinforcement learning using a chaotic neural network for emergence of “thinking” - “exploration” grows into “thinking” through learning. arXiv:1705.05551 (2017)
- 2.Volodymyr, M., et al.: Playing atari with deep reinforcement learning. In: NIPS Deep Learning Workshop 2013 (2013)Google Scholar
- 3.Shibata, K., Utsunomiya, H.: Discovery of pattern meaning from delayed rewards by reinforcement learning with a recurrent neural network. In: Proceedings of IJCNN 2011, pp. 1445–1452 (2011)Google Scholar
- 4.Shibata, K., Goto, K.: Emergence of flexible prediction-based discrete decision making and continuous motion generation through actor-Q-learning. In: Proceedings of ICDL-Epirob, ID 15 (2013)Google Scholar
- 5.Shibata, K., Sakashita, Y.: Reinforcement learning with internal-dynamics-based exploration using a chaotic neural network. In: Proceedings of IJCNN (2015)Google Scholar
- 7.Osana, Y., Hagiwara, M.: Successive learning in chaotic neural network. In: Proceedings of IJCNN 1998, vol. 2, pp. 1510–1515 (1998)Google Scholar