Accelerating Deep Q Network by Weighting Experiences
Deep Q Network (DQN) is a reinforcement learning methodlogy that uses deep neural networks to approximate the Q-function. Literature reveals that DQN can select better responses than humans. However, DQN requires a lengthy period of time to learn the appropriate actions by using tuples of state, action, reward and next state, called “experience”, sampled from its memory. DQN samples them uniformly and randomly, but the experiences are skewed resulting in slow learning because frequent experiences are redundantly sampled but infrequent ones are not. This work mitigates the problem by weighting experiences based on their frequency and manipulating their sampling probability. In a video game environment, the proposed method learned the appropriate responses faster than DQN.
KeywordsReinforcement learning Deep learning
- 2.Mnih, V., et al.: Playing Atari with deep reinforcement learning. arXiv:1312.5602v1 (2013)
- 4.Brockman, G., et al.: OpenAI Gym. arXiv:1606.01540 (2016)
- 6.Moore, A.W., Atkeson, C.G.: Prioritized sweeping: reinforcement learning with less data and less time. Mach. Learn. 13, 103–130 (1993)Google Scholar
- 7.Schaul, T., Quan, J., Antonoglou, I., Silver, D.: Prioritized experience replay. In: International Conference on Learning Representations (2016)Google Scholar
- 9.Miyazaki, K., Yamamura, M., Kobayashi, S.: A theory of profit sharing in reinforcement learning. J. Jpn. Soc. Artif. Intell. 9(4), 580–587 (1994). (in Japanese)Google Scholar