Autonomous Robots

, Volume 34, Issue 4, pp 327–346

DCOB: Action space for reinforcement learning of high DoF robots

  • Akihiko Yamaguchi
  • Jun Takamatsu
  • Tsukasa Ogasawara
Article

DOI: 10.1007/s10514-013-9328-1

Cite this article as:
Yamaguchi, A., Takamatsu, J. & Ogasawara, T. Auton Robot (2013) 34: 327. doi:10.1007/s10514-013-9328-1

Abstract

Reinforcement learning (RL) for robot control is an important technology for future robots since it enables us to design a robot’s behavior using the reward function. However, RL for high degree-of-freedom robot control is still an open issue. This paper proposes a discrete action space DCOB which is generated from the basis functions (BFs) given to approximate a value function. The remarkable feature is that, by reducing the number of BFs to enable the robot to learn quickly the value function, the size of DCOB is also reduced, which improves the learning speed. In addition, a method WF-DCOB is proposed to enhance the performance, where wire-fitting is utilized to search for continuous actions around each discrete action of DCOB. We apply the proposed methods to motion learning tasks of a simulated humanoid robot and a real spider robot. The experimental results demonstrate outstanding performance.

Keywords

Reinforcement learning Action space Motion learning Humanoid robot Crawling 

Supplementary material

10514_2013_9328_MOESM1_ESM.mpg (22.5 mb)
Supplementary material 1 (mpg 23060 KB)

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Akihiko Yamaguchi
    • 1
  • Jun Takamatsu
    • 1
  • Tsukasa Ogasawara
    • 1
  1. 1.Graduate School of Information ScienceNara Institute of Science and TechnologyNaraJapan

Personalised recommendations