Abstract
Aiming for the emergence of higher functions such as “logical thinking”, our group has proposed completely novel reinforcement learning where exploration is performed based on the internal dynamics of a chaotic neural network. In this paper, in the learning of an obstacle avoidance task, it was examined that in the process of growing the dynamics through learning, the level of exploration changes from “lower” to “higher”, in other words, from “motor level” to “more abstract level”. It was shown that the agent learned to reach the goal while avoiding the obstacle and there is an area where the agent looks to pass through the right side or left side of the obstacle randomly. The result shows the possibility of the “higher exploration” though the agent sometimes collided with the obstacle and was trapped for a while as learning progressed.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Shibata, K., Okabe, Y.: Reinforcement learning when visual signals are directly given as inputs. In: Proceedings of ICNN 1997, vol. 3, pp. 1716–1720 (1997)
Shibata, K.: Emergence of intelligence through reinforcement learning with a neural network. In: Mellouk, A. (ed.) Advances in Reinforcement Learning, pp. 99–120. InTech (2011)
Krizhevsky, A., et al.: ImageNet classification with deep convolutional neural networks. Adv. NIPS 25, 1097–1105 (2012)
Mnih, V., et al.: Playing Atari with deep reinforcement learning. In: NIPS Deep Learning Workshop 2013 (2013)
Shibata, K., Utsunomiya, H.: Discovery of pattern meaning from delayed rewards by reinforcement learning with a recurrent neural network. In: Proceedings of IJCNN 2011, pp. 1445–1452 (2011)
Shibata, K., Goto, K.: Emergence of flexible prediction-based discrete decision making and continuous motion generation through actor-Q-learning. In: Proceedings of ICDL-Epirob 2013, ID 15 (2013)
Sawatsubashi, Y., et al.: Emergence of discrete and abstract state representation through reinforcement learning in a continuous input task. In: Kim, J.-H., Matson, E.T., Myung, H., Xu, P. (eds.) Robot Intelligence Technology and Applications 2012. AISC, vol. 208, pp. 13–22. Springer, Heidelberg (2012)
Shibata, K., Sakashita, Y.: Reinforcement learning with internal-dynamics-based exploration using a chaotic neural network. In: Proceedings of IJCNN 2015, #15231 (2015)
Sussillo, D.C.: Learning in Chaotic Recurrent Neural Networks. Columbia University, Ph.D. thesis (2009)
Hoerzer, G.M., et al.: Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. Cereb. Cortex 24(3), 677–690 (2014)
Shibata, K., et al.: Direct-vision-based reinforcement learning in “Going a Target” task with an obstacle and with a variety of target sizes. In: Proceedings of NEURAP 1998, pp. 95–102 (1998)
Acknowledgement
This work was supported by JSPS KAKENHI Grant Number 15K00360.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Goto, Y., Shibata, K. (2016). Emergence of Higher Exploration in Reinforcement Learning Using a Chaotic Neural Network. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science(), vol 9947. Springer, Cham. https://doi.org/10.1007/978-3-319-46687-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-46687-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46686-6
Online ISBN: 978-3-319-46687-3
eBook Packages: Computer ScienceComputer Science (R0)