Advertisement

Case-Based Reasoning in Transfer Learning

  • David W. Aha
  • Matthew Molineaux
  • Gita Sukthankar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5650)

Abstract

Positive transfer learning (TL) occurs when, after gaining experience from learning how to solve a (source) task, the same learner can exploit this experience to improve performance and/or learning on a different (target) task. TL methods are typically complex, and case-based reasoning can support them in multiple ways. We introduce a method for recognizing intent in a source task, and then applying that knowledge to improve the performance of a case-based reinforcement learner in a target task. We report on its ability to significantly outperform baseline approaches for a control task in a simulated game of American football. We also compare our approach to an alternative approach where source and target task learning occur concurrently, and discuss the tradeoffs between them.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Blaylock, N., Allen, J.: Fast hierarchical goal schema recognition. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence, pp. 796–801. AAAI Press, Boston (2006)Google Scholar
  2. Bui, H.: A general model for online probabilistic plan recognition. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp. 1309–1318. Morgan Kaufmann, Acapulco (2002)Google Scholar
  3. Bransford, J.D., Brown, A.L., Cocking, R.R. (eds.): How people learn: Brain, mind, experience, and school. National Academy Press, Washington (2000)Google Scholar
  4. Chang, C.-C., Lin, C.-J.: LIBSVM: A library for support vector machines (2001), http://www.csie.ntu.edu.tw/~cjlin/libsvm
  5. Fagan, M., Cunningham, P.: Case-based plan recognition in computer games. In: Ashley, K.D., Bridge, D.G. (eds.) ICCBR 2003. LNCS, vol. 2689, pp. 161–170. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  6. Falkenhainer, B., Forbus, K.D., Gentner, D.: The structure-mapping engine: Algorithm and examples. Artificial Intelligence 41(1), 1–63 (1989)CrossRefzbMATHGoogle Scholar
  7. Gentner, D.: The mechanisms of analogical learning. In: Buchanan, B.G., Wilkins, D.C. (eds.) Readings in knowledge acquisition and learning: Automating the construction and improvement of expert systems. Morgan Kaufmann, San Francisco (1993)Google Scholar
  8. Goel, A., Bhatta, S.: Design patterns: A unit of analogical transfer in creative design. Advanced Engineering Informatics 18(2), 85–94 (2004)CrossRefGoogle Scholar
  9. von Hessling, A., Goel, A.: Abstracting reusable cases from reinforcement learning. In: Aha, D.W., Wilson, D.C. (eds.) Computer gaming and simulation environments: Proceedings of the ICCBR Workshop (2005); S. Bruninghaus (ed.) Workshop Proceedings of the Sixth ICCBR. DePaul University, ChicagoGoogle Scholar
  10. Hinrichs, T., Forbus, K.: Analogical learning in a turn-based strategy game. In: Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, pp. 853–858. Professional Book Center, Hyderabad (2007)Google Scholar
  11. Kautz, H.: A formal theory of plan recognition. Doctoral dissertation, University of Rochester, Rochester, NY (1987)Google Scholar
  12. Kerkez, B., Cox, M.T.: Incremental case-based plan recognition with local predictions. International Journal on Artificial Intelligence Tools 12(4), 413–463 (2003)CrossRefGoogle Scholar
  13. Klenk, M., Forbus, K.D.: Measuring the level of transfer learning by an AP physics problem-solver. In: Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, pp. 446–451. AAAI Press, Vancouver (2007)Google Scholar
  14. Kuhlmann, G., Stone, P.: Graph-based domain mapping for transfer learning in general games. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS, vol. 4701, pp. 188–200. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  15. Liao, L., Fox, D., Kautz, H.A.: Location-based activity recognition using relational Markov networks. In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, pp. 773–778. Professional Book Center, Edinburgh (2005)Google Scholar
  16. Liu, Y., Stone, P.: Value-function-based transfer for reinforcement learning using structure mapping. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence, pp. 415–420. AAAI Press, Boston (2006)Google Scholar
  17. López de Mantaras, R., McSherry, D., Bridge, D.G., Leake, D.B., Smyth, B., Craw, S., Faltings, B., Maher, M.L., Cox, M.T., Forbus, K.D., Keane, M., Aamodt, A., Watson, I.D.: Retrieval, reuse, revision and retention in case-based reasoning. Knowledge Engineering Review 20(3), 215–240 (2005)CrossRefGoogle Scholar
  18. Marx, Z., Rosenstein, M.T., Kaelbling, L.P., Dietterich, T.G.: Transfer learning with an ensemble of background tasks. In: Silver, D., Bakir, G., Bennett, K., Caruana, R., Pontil, M., Russell, S., Tadepalli, P. (eds.) Inductive Transfer: 10 Years Later: Papers from the NIPS Workshop, Whistler, BC, Canada (2005), http://iitrl.acadiau.ca/itws05/
  19. Molineaux, M., Aha, D.W.: TIELT: A testbed for gaming environments. In: Proceedings of the Twentieth National Conference on Artificial Intelligence, pp. 1690–1691. AAAI Press, Pittsburgh (2005)Google Scholar
  20. Molineaux, M., Aha, D.W., Moore, P.: Learning continuous action models in a real-time strategy environment. In: Proceedings of the Twenty-First International FLAIRS Conference, pp. 257–262. AAAI Press, Coconut Grove (2008)Google Scholar
  21. Molineaux, M., Aha, D.W., Sukthankar, G.: Beating the defense: Using plan recognition to inform learning agents. To appear in Proceedings of the Twenty-Second International FLAIRS Conference. AAAI Press, Sanibel Island (2009)Google Scholar
  22. Natarajan, S., Bui, H.H., Tadepalli, P., Kersting, K., Wong, W.-K.: Logical hierarchical hidden Markov models for modeling user activities. In: Železný, F., Lavrač, N. (eds.) ILP 2008. LNCS, vol. 5194, pp. 192–209. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  23. Natarajan, S., Tadepalli, P., Fern, A.: A relational hierarchical model for decision-theoretic assistance. In: Blockeel, H., Ramon, J., Shavlik, J., Tadepalli, P. (eds.) ILP 2007. LNCS, vol. 4894, pp. 175–190. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  24. Perkins, D.N., Salomon, G.: Transfer of learning. In: Husen, T., Postelwhite, T.N. (eds.) International Handbook of Educational Research, pp. 6452–6457. Pergamon Press, Oxford (1994)Google Scholar
  25. Raina, R., Ng, A.Y., Koller, D.: Constructing informative priors using transfer learning. In: Proceedings of the Twenty-Third International Conference on Machine Learning, pp. 713–720. ACM, Pittsburgh (2006)Google Scholar
  26. Rosenstein, M.T., Marx, Z., Kaelbling, L.P., Dietterich, T.G.: To transfer or not to transfer. In: Silver, D., Bakir, G., Bennett, K., Caruana, R., Pontil, M., Russell, S., Tadepalli, P. (eds.) Inductive Transfer: 10 Years Later: Papers from the NIPS Workshop, Whistler, BC, Canada (2005), http://iitrl.acadiau.ca/itws05/Papers/ITWS10-RosensteinM05_ITWS.pdf
  27. Shapiro, D., Könik, T., O’Rorke, P.: Achieving far transfer in an integrated cognitive architecture. In: Proceedings of the Twenty-Third Conference on Artificial Intelligence, pp. 1325–1330. AAAI Press, Chicago (2008)Google Scholar
  28. Sharma, M., Holmes, M., Santamaria, J.C., Irani, A., Isbell Jr., C.L., Ram, A.: Transfer learning in real-time strategy games using hybrid CBR/RL. In: Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, Hyderabad, India, pp. 1041–1046 (2007), http://www.aaai.org/Papers/IJCAI/2007/IJCAI07-168.pdf
  29. Simon, H.A.: Search and reasoning in problem solving. Artificial Intelligence 21, 7–29 (1983)CrossRefGoogle Scholar
  30. Stracuzzi, D.: Memory organization and knowledge transfer. In: Banerjee, B., Liu, Y., Youngblood, G.M. (eds.) Structural Knowledge Transfer for Machine Learning: Papers from the ICML Workshop, Pittsburgh, PA (2006), http://orca.st.usm.edu/~banerjee/icmlws06/
  31. Sukthankar, G., Molineaux, M., Aha, D.W.: Recognizing and exploiting opponent intent in Rush Football (Technical Note AIC-09-062). Naval Research Laboratory, Navy Center for Applied Research in Artificial Intelligence, Washington (2008)Google Scholar
  32. Sutton, R., Barto, A.: Reinforcement learning: An introduction. MIT Press, Cambridge (1998)Google Scholar
  33. Thorndike, E.L., Woodworth, R.S.: The influence of improvement in one mental function upon the efficiency of other functions (I). Psychological Review 8, 247–261 (1901)CrossRefGoogle Scholar
  34. Vapnik, V.: Statistical learning theory. Wiley & Sons, New York (1998)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • David W. Aha
    • 1
  • Matthew Molineaux
    • 2
  • Gita Sukthankar
    • 3
  1. 1.Naval Research LaboratoryNavy Center for Applied Research in Artificial IntelligenceWashington
  2. 2.Knexus Research CorporationSpringfield
  3. 3.School of Electrical Engineering and Computer ScienceUniversity of Central FloridaOrlando

Personalised recommendations