Abstract
This paper presents a novel hybrid learning method and performance evaluation methodology for adaptive autonomous agents. Measuring the performance of a learning agent is not a trivial task and generally requires long simulations as well as knowledge about the domain. A generic evaluation methodology has been developed to precisely evaluate the performance of policy estimation techniques. This methodology has been integrated into a hybrid learning algorithm which aim is to decrease the learning time and the amount of errors of an adaptive agent. The hybrid learning method namely K-learning, integrates the Q-learning and K Nearest-Neighbors algorithm. Experiments show that the K-learning algorithm surpasses the Q-learning algorithm in terms of convergence speed to a good policy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aha, D.W., Kibler, D., Albert, M.K.: Instance-based Learning Algorithms. Machine Learning 6(1), 37–66 (1991)
Almeida, A., Ramalho, G.L., Santana, H.P., Tedesco, P., Menezes, T.R., Corruble, V., Chevaleyre, Y.: Recent Advances on Multi-Agent Patrolling. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS (LNAI), vol. 3171, pp. 474–483. Springer, Heidelberg (2004)
Bianchi, R.A.C., Ribeiro, C.H.C., Costa, A.H.R.: Heuristically Accelerated Q-learning: A New Approach to Speed Up Reinforcement Learning. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS, vol. 3171, pp. 245–254. Springer, Heidelberg (2004)
Downing, K.L.: Reinforced Genetic Programming. Genetic Programming and Evolvable Machines 2(3), 259–288 (2001)
Ernst, D., Geurts, P., Wehenke, L.: Tree-Based Batch Mode Reinforcement learning. Journal of Machine Learning Research 6, 503–556 (2005)
Figueiredo, K., Vellasco, M., Pacheco, M., Souza, M.: Reinforcement Learning Hierarchical Neuro-Fuzzy Politree Model for Control of Autonomous Agents. In: Fourth Int. Conference on Hybrid Intelligent Systems (HIS 2004), pp. 130–135 (2004)
Henderson, J., Lemon, O., Georgila, K.: Hybrid reinforcement/supervised learning for dialogue policies from COMMUNICATOR data. In: Proc. IJCAI workshop on Knowledge and Reasoning in Practical Dialogue Systems, Edinburgh (2005)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement Learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Levner, I., Bulitko, V., Madani, O., Greiner, R.: Performance of lookahead control policies in the face of abstractions and approximations. In: Koenig, S., Holte, R.C. (eds.) SARA 2002. LNCS, vol. 2371, pp. 299–307. Springer, Heidelberg (2002); Maes, P.: Artificial Life Meets Entertainment: Lifelike Autonomous Agents. Communications of ACM 38(11), 108-114 (1995)
Mitchell, T.: Machine Learning. McGraw-Hill, Boston (1997)
Ramon, J.: On the convergence of reinforcement learning using a decision tree learner. In: Proceedings of ICML 2005 Workshop on Rich Representation for Reinforcement Learning, Bonn, Germany (2005)
Ribeiro, C.H.C.: A Tutorial on Reinforcement Learning Techniques. In: Int. Joint Conference on Neuronal Networks. INNS Press, Washington (1999)
Russel, S., Norvig, P.: Inteligência Artificial, 2nd edn. Editora Elsevier, Rio de Janeiro (2004)
Ryan, M.R.K.: Hierarchical Reinforcement Learning: A Hybrid Approach. PhD Thesis, University of New South Wales, School of Computer Science and Engineering (2004)
Santana, H., Ramalho, G., Corruble, V., Ratitch, B.: Multi-Agent Patrolling with Reinforcement Learning. In: Proc. 3rd International Joint Conference on Autonomous Agents and Multi-Agents Systems (AAMAS 2004), pp. 1122–1129. ACM, New York (2004)
Siedlecki, W., Sklansky, J.: A note on Genetic Algorithms for Large-Scale Selection. Pattern Recognition Letters 10, 335–347 (1989)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Tesauro, G.: Temporal Difference Learning and TD-Gammon. Communications of the ACM 38(3), 58–68 (1995)
Watkins, C.J.C.H., Dayan, P.: Q-learning, Machine Learning, 8th edn., pp. 279–292 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ribeiro, R., Enembreck, F., Koerich, A.L. (2006). A Hybrid Learning Strategy for Discovery of Policies of Action. In: Sichman, J.S., Coelho, H., Rezende, S.O. (eds) Advances in Artificial Intelligence - IBERAMIA-SBIA 2006. IBERAMIA SBIA 2006 2006. Lecture Notes in Computer Science(), vol 4140. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11874850_31
Download citation
DOI: https://doi.org/10.1007/11874850_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45462-5
Online ISBN: 978-3-540-45464-9
eBook Packages: Computer ScienceComputer Science (R0)