An Autonomous Transfer Learning Algorithm for TD-Learners

  • Anestis Fachantidis
  • Ioannis Partalas
  • Matthew E. Taylor
  • Ioannis Vlahavas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8445)


The main objective of transfer learning is to use the knowledge acquired from a source task in order to boost the learning procedure in a target task. Transfer learning comprises a suitable solution for reinforcement learning algorithms, which often require a considerable amount of training time, especially when dealing with complex tasks. This work proposes an autonomous method for transfer learning in reinforcement learning agents. The proposed method is empirically evaluated in the keepaway and the mountain car domains. The results demonstrate that the proposed method can improve the learning procedure in the target task.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Taylor, M.E., Stone, P., Liu, Y.: Transfer learning via inter-task mappings for temporal difference learning. Journal of Machine Learning Research 8, 2125–2167 (2007)zbMATHMathSciNetGoogle Scholar
  2. 2.
    Fachantidis, A., Partalas, I., Taylor, M.E., Vlahavas, I.: Transfer learning via multiple inter-task mappings. In: Sanner, S., Hutter, M. (eds.) EWRL 2011. LNCS, vol. 7188, pp. 225–236. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  3. 3.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning, An Introduction. MIT Press (1998)Google Scholar
  4. 4.
    Stone, P., Sutton, R.S., Kuhlmann, G.: Reinforcement learning for RoboCup-soccer keepaway. Adaptive Behavior 13(3), 165–188 (2005)CrossRefGoogle Scholar
  5. 5.
    Singh, S.P., Sutton, R.S.: Reinforcement learning with replacing eligibility traces. Machine Learning 22(1-3), 123–158 (1996)CrossRefzbMATHGoogle Scholar
  6. 6.
    Taylor, M.E., Jong, N.K., Stone, P.: Transferring instances for model-based reinforcement learning. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part II. LNCS (LNAI), vol. 5212, pp. 488–505. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Taylor, M.E., Kuhlmann, G., Stone, P.: Autonomous transfer for reinforcement learning. In: 7th AAMAS, pp. 283–290 (2008)Google Scholar
  8. 8.
    Stone, P., Kuhlmann, G., Taylor, M.E., Liu, Y.: Keepaway soccer: From machine learning testbed to benchmark. In: Bredenfeld, A., Jacoff, A., Noda, I., Takahashi, Y. (eds.) RoboCup 2005. LNCS (LNAI), vol. 4020, pp. 93–105. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Fernández, F., Veloso, M.: Probabilistic policy reuse in a reinforcement learning agent. In: 5th International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 720–727 (2006)Google Scholar
  10. 10.
    Taylor, M.E., Stone, P.: Cross-domain transfer for reinforcement learning. In: 24th International Conference on Machine Learning, pp. 879–886 (2007)Google Scholar
  11. 11.
    Torrey, L., Shavlik, J., Walker, T., Maclin, R.: Skill acquisition via transfer learning and advice taking. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 425–436. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Talvitie, E., Singh, S.: An experts algorithm for transfer learning. In: 20th IJCAI, pp. 1065–1070 (2007)Google Scholar
  13. 13.
    Sorg, J., Singh, S.: Transfer via soft homomorphisms. In: Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2009, vol. 2, pp. 741–748 (2009)Google Scholar
  14. 14.
    Ammar, H.B., Tuyls, K., Taylor, M.E., Driessen, K., Weiss, G.: Reinforcement learning transfer via sparse coding. In: International Conference on Autonomous Agents and Multiagent Systems, AAMAS (June 2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Anestis Fachantidis
    • 1
  • Ioannis Partalas
    • 2
  • Matthew E. Taylor
    • 3
  • Ioannis Vlahavas
    • 1
  1. 1.Department of InformaticsAristotle University of ThessalonikiGreece
  2. 2.Laboratoire LIGUniversité Joseph FourierFrance
  3. 3.School of Electrical Engineering and Computer ScienceWashington State UniversityUSA

Personalised recommendations