Abstract
The main objective of transfer learning is to reuse knowledge acquired in a previous learned task, in order to enhance the learning procedure in a new and more complex task. Transfer learning comprises a suitable solution for speeding up the learning procedure in Reinforcement Learning tasks. In this work, we propose a novel method for transferring models to a hybrid reinforcement learning agent. The models of the transition and reward functions of a source task, will be transferred to a relevant but different target task. The learning algorithm of the target task’s agent takes a hybrid approach, implementing both model-free and model-based learning, in order to fully exploit the presence of a model. The empirical evaluation, of the proposed approach, demonstrated significant results and performance improvements in the 3D Mountain Car task, by successfully using the models generated from the standard 2D Mountain Car.
Chapter PDF
Similar content being viewed by others
References
Albus, J.S.: A new approach to manipulator control: The cerebellar model articulation controller (cmac). Journal of Dynamic Systems, Measurement, and Control 97, 220 (1975)
Fernández, F., Veloso, M.: Probabilistic policy reuse in a reinforcement learning agent. In: 5th International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 720–727 (2006)
Gullapalli, V., Gullapalli, V., Barto, A.G.: Shaping as a method for accelerating reinforcement learning. In: Proceedings of the 1992 IEEE International Symposium on Intelligent Control, pp. 554–559. IEEE, Los Alamitos (1992)
Kalyanakrishnan, S., Stone, P., Liu, Y.: Model-based reinforcement learning in a complex domain (2008)
Lazaric, A., Restelli, M., Bonarini, A.: Transfer of samples in batch reinforcement learning. In: ICML 2008: Proceedings of the 25th International Conference on Machine Learning, pp. 544–551 (2008)
Moore, A.: Variable resolution dynamic programming: Efficiently learning action maps in multivariate real-valued state-spaces. In: Proceedings of the Eighth International Conference on Machine Learning (1991)
Rivest, F., Precup, D.: Combining td-learning with cascade-correlation networks (2003)
Singh, S.P., Sutton, R.S.: Reinforcement learning with replacing eligibility traces. Machine Learning 22(1-3), 123–158 (1996)
Tanner, B., White, A.: Rl-glue: Language-independent software for reinforcement-learning experiments. Journal of Machine Learning Research 10, 2133–2136 (2010)
Taylor, M., Stone, P.: Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research 10, 1633–1685 (2009)
Taylor, M.E., Jong, N.K., Stone, P.: Transferring instances for model-based reinforcement learning. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part II. LNCS (LNAI), vol. 5212, pp. 488–505. Springer, Heidelberg (2008)
Taylor, M.E., Kuhlmann, G., Stone, P.: Autonomous transfer for reinforcement learning. In: AAMAS 2008: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 283–290 (2008)
Taylor, M.E., Stone, P.: Cross-domain transfer for reinforcement learning. In: ICML 2007: Proceedings of the 24th International Conference on Machine Learning, pp. 879–886 (2007)
Taylor, M.E., Stone, P., Liu, Y.: Transfer learning via inter-task mappings for temporal difference learning. Journal of Machine Learning Research 8, 2125–2167 (2007)
Torrey, L., Shavlik, J., Walker, T., Maclin, R.: Skill acquisition via transfer learning and advice taking. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 425–436. Springer, Heidelberg (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 International Federation for Information Processing
About this paper
Cite this paper
Fachantidis, A., Partalas, I., Tsoumakas, G., Vlahavas, I. (2011). Transferring Models in Hybrid Reinforcement Learning Agents. In: Iliadis, L., Jayne, C. (eds) Engineering Applications of Neural Networks. EANN AIAI 2011 2011. IFIP Advances in Information and Communication Technology, vol 363. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23957-1_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-23957-1_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23956-4
Online ISBN: 978-3-642-23957-1
eBook Packages: Computer ScienceComputer Science (R0)