BioADIT 2004: Biologically Inspired Approaches to Advanced Information Technology pp 244-257 | Cite as
A Study on Designing Robot Controllers by Using Reinforcement Learning with Evolutionary State Recruitment Strategy
Conference paper
Abstract
Recently, much attention has been focused on utilizing reinforcement learning (RL) for designing robot controllers. However, as the state spaces of these robots become continuous and high dimensional, it results in time-consuming process. In order to adopt the RL for designing the controllers of such complicated systems, not only adaptability but also computational efficiencies should be taken into account. In this paper, we introduce an adaptive state recruitment strategy which enables a learning robot to rearrange its state space conveniently according to the task complexity and the progress of the learning.
Keywords
Radial Basis Function Mobile Robot Reinforcement Learning Radial Basis Function Network Real Robot
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Preview
Unable to display preview. Download preview PDF.
References
- 1.Asada, M., Noda, S., Tawaratsumida, S., Hosoda, K.: Purposive Behavior Acquisition for a Real Robot by Vision-Based Reinforcement Learning. Machine Learning 23, 279–303 (1996)Google Scholar
- 2.Barto, A.G., Sutton, R.S., Anderson, C.W.: Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans. Systems, Man, and Cybernetics 13, 834–846 (1983)Google Scholar
- 3.Brooks, R.A.: Artificial life and real robots. In: Proc. of the First European Conference on Artificial Life, MIT Press, Cambridge (1992)Google Scholar
- 4.Digney, B.L.: Learning Hierarchical Control Structure for Multiple Tasks and Changing Environments. In: Proc. of the Fifth International Conf. on Simulation of Adaptive Behavior, pp. 321/330. A Bradford Book, MIT Press, Cambridge, MA (1998)Google Scholar
- 5.Eggenberger, P., Ishiguro, A., Tokura, S., Kondo, T., Uchikawa, Y.: Toward Seamless Transfer from Simulated to Real Worlds: A Dynamically-Rearranging Neural Network Approach. In: Demiris, J., Wyatt, J.C. (eds.) EWLR 1999. LNCS (LNAI), vol. 1812, pp. 44–60. Springer, Heidelberg (2000)CrossRefGoogle Scholar
- 6.Fritzke, B.: Incremental learning of locally linear mappings. In: Proc. of the International Conference on Artificial Neural Networks (1995)Google Scholar
- 7.Holland, J.: Adaptation in Natural and Artificial Systems. University of Michigan Press, MIT Press (1975)Google Scholar
- 8.Jockusch, S., Ritter, H.: Self Organizing Maps: Local Competition and Evolutionary Optimization Neural Networks 7(8), 1229–1239 (1994)Google Scholar
- 9.Kohonen, T.: Self-Organizing Maps. Springer Series in Information Sciences (1995)Google Scholar
- 10.K-Team SA: Khepera User Manual ver 5.0 (1998)Google Scholar
- 11.Miglino, O., Lund, H.H., Nolfi, S.: Evolving Mobile Robots in Simulated and Real Environments. Artificial Life 2, 417–434 (1995)CrossRefGoogle Scholar
- 12.Moody, J., Darken, C.J.: Fast learning in networks of locally-tuned processing units. Neural Computation 1, 281–294 (1989)CrossRefGoogle Scholar
- 13.Moriarty, D.E.: Efficient reinforcement learning through symbiotic evolution. Machine Learning 32(22), 11 (1996)Google Scholar
- 14.Morimoto, J., Doya, K.: Acquisition of Stand-up Behavior by Real Robot using Reinforcement Learning. In: Proc. of International Conference on Machine Learning, pp. 623–630 (2000)Google Scholar
- 15.Nolfi, S., Floreano, D.: “Evolutionary Robotics” —The Biology, Intelligence, and Technology of Self-Organizing Machines–. A Bradford Book, MIT Press, Cambridge, MA (2000)Google Scholar
- 16.Platt, J.: A Resource-Allocating Network for Function Interpolation Neural Networks 3(2), 213–225 (1991)Google Scholar
- 17.Samejima, K., Omori, T.: Adaptive internal state space formation by reinforcement learning for real-world agent. Neural Networks 12(7-8), 1143–1156 (1999)CrossRefGoogle Scholar
- 18.Sato, M., Ishii, S.: On-line EM Algorithms for the Normalized Gaussian Network. Neural Computation 12(2) (1999)Google Scholar
- 19.Stewart, W.W.: Classifier that Approximate Functions IlliGAL Report No. 2001027, Illinois Genetic Algorithms Laboratory (2001)Google Scholar
- 20.Vijayakumar, S., Schaal, S.: Fast and Efficient Incremental Learning for Highdimensional Movement Systems. In: Proc. of International Conf. on Robotics and Automation, ICRA 2000 (2000)Google Scholar
- 21.Yoshimoto, J., Ishii, S., Sato, M.: Application of reinforcement learning based on on-line EM algorithm to balancing of acrobot. Systems and Computers in Japan 32(5), 12–20 (2001)CrossRefGoogle Scholar
Copyright information
© Springer-Verlag Berlin Heidelberg 2004