Abstract
We consider the problem of approximating the cost-to-go functions in reinforcement learning. By mapping the state implicitly into a feature space, we perform a simple algorithm in the feature space, which corresponds to a complex algorithm in the original state space. Two kernel-based reinforcement learning algorithms, the ε -insensitive kernel based reinforcement learning (ε – KRL) and the least squares kernel based reinforcement learning (LS-KRL) are proposed. An example shows that the proposed methods can deal effectively with the reinforcement learning problem without having to explore many states.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sutton, R.S.: Learning to Predict by the Methods of Temporal Differences. Machine Learning 3, 9–44 (1988)
Watkins, C.J.C.H.: Q-Learning. Machine Learning 8, 279–292 (1992)
Santharam, G., Sastry, P.S.: A Reinforcement Learning Neural Network for Adaptive Control Markov Chains. IEEE Transactions on System, Man and Cybernetics-Part A 27, 588–600 (1997)
Tsitsiklis, J.N., Roy, B.V.: An Analysis of Temporal-Difference Learning with Function Approximation. IEEE Transactions on Automatic Control 42, 674–690 (1997)
Tsitsiklis, J.N., Roy, B.V.: Feature-Based Methods for Large Scale Dynamic Programming. Machine Learning 22, 59–94 (1996)
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Camgbridge (2004)
Bertsekas, D.P.: Dynamic Programming: Deterministic and Stochastic Methods. Prentice-Hall, Englewood Cliffs (1987)
Vapnik, V.N.: Statistical Learning Theory. John Wiley & Sons, New York (1998)
Cawley, G.C., Talbot, N.L.C.: Improved Sparse Least-Squares Support Vector Machines. Neurocomputing 48, 1025–1031 (2002)
Flake, G.W., Lawrence, S.: Efficient SVM Regression Training with SMO. Machine Learning 46, 271–290 (2002)
Suykens, J.A.K., Brabanter, J.D., Lukas, L., Vandewalle, J.: Weighted Least Squares Support Machines: Robustness and Sparse Approximation. Neurocomputing 48, 85–105 (2002)
Platt, J.: Fast Training of Support Vector Machines Using Sequential Minimal Optimization. In: Scholkopf, B., Burges, C.J., Smola, A.J. (eds.) Advances in Kernel Methods –Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hu, G., Qiu, Y., Xiang, L. (2006). Kernel-Based Reinforcement Learning. In: Huang, DS., Li, K., Irwin, G.W. (eds) Intelligent Computing. ICIC 2006. Lecture Notes in Computer Science, vol 4113. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11816157_92
Download citation
DOI: https://doi.org/10.1007/11816157_92
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-37271-4
Online ISBN: 978-3-540-37273-8
eBook Packages: Computer ScienceComputer Science (R0)