Q Learning Based on Self-organizing Fuzzy Radial Basis Function Network
A fuzzy Q learning based on a self-organizing fuzzy radial basis function (FRBF) network is proposed to solve the ‘curse of dimensionality’ problem caused by state space generalization in the paper. A FRBF network is used to represent continuous action and the corresponding Q value. The interpolation technique is adopted to represent the appropriate utility value for the wining local action of every fuzzy rule. Neurons can be organized by the FRBF network itself. The methods of the structure and parameter learning, based on new adding and merging neurons techniques and a gradient descent algorithm, are simple and effective, with a high accuracy and a compact structure. Simulation results on balancing control of inverted pendulum illustrate the performance and applicability of the proposed fuzzy Q learning scheme to real-world problems with continuous states and continuous actions.
KeywordsMembership Function Fuzzy Rule Fuzzy Inference System Inverted Pendulum Continuous Action
Unable to display preview. Download preview PDF.
- 3.Smith, A.J.: Applications of the Self-Organizing Map to Reinforcement Learning. Neural Network (15), 1107–1124 (2002)Google Scholar
- 4.Gross, H.M., Stephan, V., Krabbes, M.: A Neural Field Approach to Topological Reinforcement Learning in Continuous Action Spaces. In: Proceedings of the IEEE World Congress on Computational Intelligence, San Diego, vol. 3, pp. 3460–3465 (1998)Google Scholar
- 6.Kim, M.S., Hong, S.G., Lee, J.J.: On-line Fuzzy Q-Learning with Extended Rule Interpolation Technique. In: Proceedings of the 1999/RSJ International Conference on Intelligent Robots and Systems, Kyongiu, vol. 2, pp. 757–762 (1999)Google Scholar
- 7.Samejima, K., Omori, T.: Adaptive Internal State Space Construction Method for Reinforcement Learning of a Real-World Agent. Neural Networks (12), 1143–1155 (1999)Google Scholar