Skip to main content
SpringerLink
Log in

Generalization and Transfer Learning with Qualitative Spatial Abstraction

  • Chapter
  • First Online:
Qualitative Spatial Abstraction in Reinforcement Learning

Part of the book series: Cognitive Technologies ((COGTECH))

  • 857 Accesses

Abstract

In this chapter we will investigate the properties of observation space representations achieved by qualitative abstraction with respect to generalization and transfer learning capabilities. Section 5.1 describes the importance of structural similarity for knowledge reuse.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Baird, L.: Residual algorithms: Reinforcement learning with function approximation. In: Proceedingsof the Twelfth International Conference on Machine Learning (ICML), pp. 30–37. MorganKaufmann, San Francisco, CA (1995)

    Google Scholar 

  • Frommberger, L.: A qualitative representation of structural spatial knowledge for robot navigationwith reinforcement learning. In: Proceedings of the ICMLWorkshop on Structural KnowledgeTransfer for Machine earning. Pittsburgh, PA, USA (2006)

    Google Scholar 

  • Frommberger, L.: Generalization and transfer learning in noise-affected robot navigation tasks. In: Neves, J.M., Santos, M.F., Machado, J.M. (eds.) Progress in Artificial Intelligence: Proceedingsof EPIA 2007, Lecture Notes in Artificial Intelligence, vol. 4874, pp. 508–519. Springer-VerlagBerlin Heidelberg, Guimarães, Portugal (2007a)

    Google Scholar 

  • Frommberger, L.: A generalizing spatial representation for robot navigation with reinforcementlearning. In: Proceedings of the Twentieth International Florida Artificial Intelligence ResearchSociety Conference (FLAIRS), pp. 586–591. AAAI Press, Key West, FL, USA (2007b)

    Google Scholar 

  • Frommberger, L.: Learning to behave in space: A qualitative spatial representation for robot navigationwith reinforcement learning. International Journal on Artificial Isntelligence Tools 17(3), 465–482 (2008a)

    Article  Google Scholar 

  • Frommberger, L.: Representing and selecting landmarks in autonomous learning of robot navigation. In: Xiong, C., Liu, H., Huang, Y., Xiong, Y. (eds.) Intelligent Robotics and Applications:First International Conference (ICIRA 2008), Part I, Lecture Notes in Artificial Intelligence,vol. 5314,pp. 488–497. Springer Verlag Berlin Heidelberg (2008b)

    Google Scholar 

  • Frommberger, L.: Situation dependent spatial abstraction in reinforcement learning based on structuralknowledge. In: Proceedings of the ICML/UAI/COLT Workshop on Abstraction in ReinforcementLearning. Montreal, Canada (2009)

    Google Scholar 

  • Gordon, G.J.: Stable function approximation on dynamic programming. In: Proceedings of theTwelfth International Conference on Machine Learning (ICML), pp. 261–268. Morgan Kaufmann,San Francisco (1995)

    Google Scholar 

  • Konidaris, G.D.: A framework for transfer in reinforcement learning. In: Proceedings of the ICML Workshop on Structural Knowledge Transfer for Machine Learning. Pittsburgh, PA, USA (2006)

    Google Scholar 

  • Konidaris, G.D., Barto, A.G.: Autonomous shaping: Knowledge transfer in reinforcement learning. In: Proceedings of the Twenty Third International Conference on Machine Learning (ICML), pp. 489–49. Pittsburgh, PA (2006)

    Google Scholar 

  • Roberts, F.S.: Tolerance geometry. Notre Dame Journal of Formal Logic 14(1), 68–76 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  • Sutton, R.S.: Generalization in reinforcement learning: Successful examples using sparse tile coding. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Advances in Neural InformationProcessing Systems: Proceedings of the 1995 Conference, vol. 8, pp. 1038–1044. MIT Press, Cambridge, MA (1996)

    Google Scholar 

  • Timmer, S., Riedmiller, M.: Fitted Q-iteration with CMACs. In: Proceedings of the IEEE InternationalSymposium on Approximate Dynamic Programming and Reinforcement Learning, pp.1–8. Honolulu, HI (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cognitive Systems Group, Department of Mathematics and Informatics, University of Bremen, P.O. Box 330 440, 28334, Bremen, Germany

    Dr.-Ing. Lutz Frommberger

Authors
  1. Dr.-Ing. Lutz Frommberger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lutz Frommberger .

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Frommberger, L. (2010). Generalization and Transfer Learning with Qualitative Spatial Abstraction. In: Qualitative Spatial Abstraction in Reinforcement Learning. Cognitive Technologies. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16590-0_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-16590-0_5

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-16589-4

  • Online ISBN: 978-3-642-16590-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation